mirror of
https://github.com/JoinMarket-Org/joinmarket-clientserver.git
synced 2026-08-13 12:33:40 +02:00
Onion-based message channels with directory nodes
Joinmarket bots run their own onion services allowing inbound connections. Both takers and makers connect to other makers at the mentioned onion services, over Tor. Directory nodes run persistent onion services allowing peers to find other (maker) peers to connect to, and also forwarding messages where necessary. This is implemented as an alternative to IRC, i.e. a new implementation of the abstract class MessageChannel, in onionmc.py. Note that using both this *and* IRC servers is supported; Joinmarket supports multiple, redundant different communication methods, simultaneously. Messaging is done with a derived class of twisted's LineReceiver, and there is an additional layer of syntax, similar to but not the same as the IRC syntax for ensuring that messages are passed with the same J5.. nick as is used on IRC. This allows us to keep the message signing logic the same as before. As well as Joinmarket line messages, we use additional control messages to communicate peer lists, and to manage connections. Peers which send messages not conforming to the syntax are dropped. See https://github.com/JoinMarket-Org/JoinMarket-Docs/pull/12 for documentation of the syntax. Connections to directory nodes are robust as for IRC servers, in that we use a ReconnectingClientFactory to keep trying to re-establish broken connections with exponential backoff. Connections to maker peers do not require this feature, as they will often disconnect in normal operation. Multiple directory nodes can and should be configured by bots.
This commit is contained in:
Adam Gibson
Adam Gibson
parent
bb8cd0074d
commit
fd550ee564
17 changed files with 1945 additions and 94 deletions
173
docs/onion-message-channels.md
Normal file
173
docs/onion-message-channels.md
Normal file
|
|
@ -0,0 +1,173 @@
|
|||
# HOW TO SETUP ONION MESSAGE CHANNELS IN JOINMARKET
|
||||
|
||||
### Contents
|
||||
|
||||
1. [Overview](#overview)
|
||||
|
||||
2. [Testing, configuring for signet](#testing)
|
||||
|
||||
4. [Directory nodes](#directory)
|
||||
|
||||
<a name="overview" />
|
||||
|
||||
## Overview
|
||||
|
||||
This is a new way for Joinmarket bots to communicate, namely by serving and connecting to Tor onion services. This does not
|
||||
introduce any new requirements to your Joinmarket installation, technically, because the use of Payjoin already required the need
|
||||
to service such onion services, and connecting to IRC used a SOCKS5 proxy (by default, and used by almost all users) over Tor to
|
||||
a remote onion service.
|
||||
|
||||
The purpose of this new type of message channel is as follows:
|
||||
|
||||
* less reliance on any service external to Joinmarket
|
||||
* most of the transaction negotiation will be happening directly peer to peer, not passed over a central server (
|
||||
albeit it was and remains E2E encrypted data, in either case)
|
||||
* the above can lead to better scalability at large numbers
|
||||
* a substantial increase in the speed of transaction negotiation; this is mostly related to the throttling of high bursts of traffic on IRC
|
||||
|
||||
The configuration for a user is simple; in their `joinmarket.cfg` they will add a messaging section like this:
|
||||
|
||||
```
|
||||
[MESSAGING:onion1]
|
||||
type = onion
|
||||
onion_serving_port = 8082
|
||||
# This is a comma separated list (comma can be omitted if only one item).
|
||||
# Each item has format host:port
|
||||
directory_nodes = rr6f6qtleiiwic45bby4zwmiwjrj3jsbmcvutwpqxjziaydjydkk5iad.onion:80
|
||||
```
|
||||
|
||||
Here, I have deliberately omitted the several other settings in this section which will almost always be fine as default;
|
||||
see `jmclient/jmclient/configure.py` for what those defaults are, and the extensive comments explaining.
|
||||
|
||||
The main point is the list of **directory nodes** (the one shown here is one being run on signet, right now), which will
|
||||
be comma separated if multiple directory nodes are configured (we expect there will be 2 or 3 as a normal situation).
|
||||
The `onion_serving_port` is on which port on the local machine the onion service is served.
|
||||
The `type` field must always be `onion` in this case, and distinguishes it from IRC message channels and others.
|
||||
|
||||
### Can/should I still run IRC message channels?
|
||||
|
||||
In short, yes.
|
||||
|
||||
### Do I need to configure Tor, and if so, how?
|
||||
|
||||
These message channels use both outbound and inbound connections to onion services (or "hidden services").
|
||||
|
||||
As previously mentioned, both of these features were already in use in Joinmarket. If you never served an
|
||||
onion service before, it should work fine as long as you have the Tor service running in the background,
|
||||
and the default control port 9051 (if not, change that value in the `joinmarket.cfg`, see above.
|
||||
|
||||
#### Why not use Lightning based onions?
|
||||
|
||||
(*Feel free to skip this section if you don't know what "Lightning based onions" refers to!*). The reason this architecture is
|
||||
proposed as an alternative to the previously suggested Lightning-node-based network (see
|
||||
[this PR](https://github.com/JoinMarket-Org/joinmarket-clientserver/pull/1000)), is mostly that:
|
||||
|
||||
* the latter has a bunch of extra installation and maintenance dependencies (just one example: pyln-client requires coincurve, which we just
|
||||
removed)
|
||||
* the latter requires establishing a new node "identity" which can be refreshed, but that creates more concern
|
||||
* longer term ideas to integrate Lightning payments to the coinjoin workflow (and vice versa!) are not realizable yet
|
||||
* using multi-hop onion messaging in the LN network itself is also a way off, and a bit problematic
|
||||
|
||||
So the short version is: the Lightning based alternative is certainly feasible, but has a lot more baggage that can't really be justified
|
||||
unless we're actually using it for something.
|
||||
|
||||
|
||||
<a name="testing" />
|
||||
|
||||
## Testing, and configuring for signet.
|
||||
|
||||
This testing section focuses on signet since that will be the less troublesome way of getting involved in tests for
|
||||
the non-hardcore JM developer :)
|
||||
|
||||
(For the latter, please use the regtest setup by running `test/e2e-coinjoin-test.py` under `pytest`,
|
||||
and pay attention to the settings in `regtest_joinmarket.cfg`.)
|
||||
|
||||
There is no separate/special configuration for signet other than the configuration that is already needed for running
|
||||
Joinmarket against a signet backend (so e.g. RPC port of 38332).
|
||||
|
||||
Add the `[MESSAGING:onion1]` message channel section to your `joinmarket.cfg`, as listed above, including the
|
||||
signet directory node listed above (rr6f6qtleiiwic45bby4zwmiwjrj3jsbmcvutwpqxjziaydjydkk5iad.onion:80), and,
|
||||
for the simplest test, remove the other `[MESSAGING:*]` sections that you have.
|
||||
|
||||
Then just make sure your bot has some signet coins and try running as maker or taker or both.
|
||||
|
||||
<a name="directory" />
|
||||
|
||||
## Directory nodes
|
||||
|
||||
**This last section is for people with a lot of technical knowledge in this area,
|
||||
who would like to help by running a directory node. You can ignore it if that does not apply.**.
|
||||
|
||||
This requires a long running bot. It should be on a server you can keep running permanently, so perhaps a VPS,
|
||||
but in any case, very high uptime. For reliability it also makes sense to configure to run as a systemd service.
|
||||
|
||||
A note: in this early stage, the usage of Lightning is only really network-layer stuff, and the usage of bitcoin, is none; feel free to add elements that remove any need for a backend bitcoin blockchain, but beware: future upgrades *could* mean that the directory node really does need the bitcoin backend.
|
||||
|
||||
#### Joinmarket-specific configuration
|
||||
|
||||
Add `hidden_service_dir` to your `[MESSAGING:onion1]` with a directory accessible to your user. You may want to lock this down
|
||||
a bit!
|
||||
The point to understand is: Joinmarket's `jmbase.JMHiddenService` will, if configured with a non-empty `hidden_service_dir`
|
||||
field, actually start an *independent* instance of Tor specifically for serving this, under the current user.
|
||||
(our tor interface library `txtorcon` needs read access to the Tor HS dir, so it's troublesome to do this another way).
|
||||
|
||||
##### Question: How to configure the `directory-nodes` list in our `joinmarket.cfg` for this directory node bot?
|
||||
|
||||
Answer: **you must only enter your own node in this list!** (otherwise you may find your bot infinitely rebroadcasting messages).
|
||||
|
||||
|
||||
#### Suggested setup of a service:
|
||||
|
||||
You will need two components: bitcoind, and Joinmarket itself, which you can run as a yg.
|
||||
Since this task is going to be attempted by someone with significant technical knowledge,
|
||||
only an outline is provided here; several details will need to be filled in.
|
||||
Here is a sketch of how the systemd service files can be set up for signet:
|
||||
|
||||
If someone wants to put together a docker setup of this for a more "one-click install", that would be great.
|
||||
|
||||
1. bitcoin-signet.service
|
||||
|
||||
```
|
||||
[Unit]
|
||||
Description=bitcoind signet
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
ExecStart=/usr/local/bin/bitcoind -signet
|
||||
User=user
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
```
|
||||
|
||||
This is deliberately a super-basic setup (see above). Don't forget to setup your `bitcoin.conf` as usual,
|
||||
for the bitcoin user, and make it match (specifically in terms of RPC) what you set up for Lightning below.
|
||||
|
||||
|
||||
2.
|
||||
|
||||
```
|
||||
[Unit]
|
||||
Description=joinmarket directory node on signet
|
||||
Requires=bitcoin-signet.service
|
||||
After=bitcoin-signet.service
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
ExecStart=/bin/bash -c 'cd /path/to/joinmarket-clientserver && source jmvenv/bin/activate && cd scripts && echo -n "password" | python yg-privacyenhanced.py --wallet-password-stdin --datadir=/custom/joinmarket-datadir some-signet-wallet.jmdat'
|
||||
User=user
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
```
|
||||
|
||||
To state the obvious, the idea here is that this second service will run the JM directory node and have a dependency on the previous one,
|
||||
to ensure they start up in the correct order.
|
||||
|
||||
Re: password echo, obviously this kind of password entry is bad;
|
||||
for now we needn't worry as these nodes don't need to carry any real coins (and it's better they don't!).
|
||||
Later we may need to change that (though of course you can use standard measures to protect the box).
|
||||
|
||||
TODO: add some material on network hardening/firewalls here, I guess.
|
||||
|
|
@ -128,16 +128,23 @@ def config_to_hs_ports(virtual_port, host, port):
|
|||
class JMHiddenService(object):
|
||||
""" Wrapper class around the actions needed to
|
||||
create and serve on a hidden service; an object of
|
||||
type Resource must be provided in the constructor,
|
||||
which does the HTTP serving actions (GET, POST serving).
|
||||
type either Resource or server.ProtocolFactory must
|
||||
be provided in the constructor, which does the HTTP
|
||||
(GET, POST) or other protocol serving actions.
|
||||
"""
|
||||
def __init__(self, resource, info_callback, error_callback,
|
||||
onion_hostname_callback, tor_control_host,
|
||||
def __init__(self, proto_factory_or_resource, info_callback,
|
||||
error_callback, onion_hostname_callback, tor_control_host,
|
||||
tor_control_port, serving_host, serving_port,
|
||||
virtual_port = None,
|
||||
shutdown_callback = None):
|
||||
self.site = Site(resource)
|
||||
self.site.displayTracebacks = False
|
||||
virtual_port=None,
|
||||
shutdown_callback=None,
|
||||
hidden_service_dir=""):
|
||||
if isinstance(proto_factory_or_resource, Resource):
|
||||
# TODO bad naming, in this case it doesn't start
|
||||
# out as a protocol factory; a Site is one, a Resource isn't.
|
||||
self.proto_factory = Site(proto_factory_or_resource)
|
||||
self.proto_factory.displayTracebacks = False
|
||||
else:
|
||||
self.proto_factory = proto_factory_or_resource
|
||||
self.info_callback = info_callback
|
||||
self.error_callback = error_callback
|
||||
# this has a separate callback for convenience, it should
|
||||
|
|
@ -155,6 +162,13 @@ class JMHiddenService(object):
|
|||
# config object, so no default here:
|
||||
self.serving_host = serving_host
|
||||
self.serving_port = serving_port
|
||||
# this is used to serve an onion from the filesystem,
|
||||
# NB: Because of how txtorcon is set up, this option
|
||||
# uses a *separate tor instance* owned by the owner of
|
||||
# this script (because txtorcon needs to read the
|
||||
# HS dir), whereas if this option is "", we set up
|
||||
# an ephemeral HS on the global or pre-existing tor.
|
||||
self.hidden_service_dir = hidden_service_dir
|
||||
|
||||
def start_tor(self):
|
||||
""" This function executes the workflow
|
||||
|
|
@ -162,19 +176,31 @@ class JMHiddenService(object):
|
|||
"""
|
||||
self.info_callback("Attempting to start onion service on port: {} "
|
||||
"...".format(self.virtual_port))
|
||||
if str(self.tor_control_host).startswith('unix:'):
|
||||
control_endpoint = UNIXClientEndpoint(reactor,
|
||||
self.tor_control_host[5:])
|
||||
if self.hidden_service_dir == "":
|
||||
if str(self.tor_control_host).startswith('unix:'):
|
||||
control_endpoint = UNIXClientEndpoint(reactor,
|
||||
self.tor_control_host[5:])
|
||||
else:
|
||||
control_endpoint = TCP4ClientEndpoint(reactor,
|
||||
self.tor_control_host, self.tor_control_port)
|
||||
d = txtorcon.connect(reactor, control_endpoint)
|
||||
d.addCallback(self.create_onion_ep)
|
||||
d.addErrback(self.setup_failed)
|
||||
# TODO: add errbacks to the next two calls in
|
||||
# the chain:
|
||||
d.addCallback(self.onion_listen)
|
||||
d.addCallback(self.print_host)
|
||||
else:
|
||||
control_endpoint = TCP4ClientEndpoint(reactor,
|
||||
self.tor_control_host, self.tor_control_port)
|
||||
d = txtorcon.connect(reactor, control_endpoint)
|
||||
d.addCallback(self.create_onion_ep)
|
||||
d.addErrback(self.setup_failed)
|
||||
# TODO: add errbacks to the next two calls in
|
||||
# the chain:
|
||||
d.addCallback(self.onion_listen)
|
||||
d.addCallback(self.print_host)
|
||||
ep = "onion:" + str(self.virtual_port) + ":localPort="
|
||||
ep += str(self.serving_port)
|
||||
# endpoints.TCPHiddenServiceEndpoint creates version 2 by
|
||||
# default for backwards compat (err, txtorcon needs to update that ...)
|
||||
ep += ":version=3"
|
||||
ep += ":hiddenServiceDir="+self.hidden_service_dir
|
||||
onion_endpoint = serverFromString(reactor, ep)
|
||||
d = onion_endpoint.listen(self.proto_factory)
|
||||
d.addCallback(self.print_host_filesystem)
|
||||
|
||||
|
||||
def setup_failed(self, arg):
|
||||
# Note that actions based on this failure are deferred to callers:
|
||||
|
|
@ -195,7 +221,8 @@ class JMHiddenService(object):
|
|||
serverstring = "tcp:{}:interface={}".format(self.serving_port,
|
||||
self.serving_host)
|
||||
onion_endpoint = serverFromString(reactor, serverstring)
|
||||
return onion_endpoint.listen(self.site)
|
||||
print("created the onion endpoint, now calling listen")
|
||||
return onion_endpoint.listen(self.proto_factory)
|
||||
|
||||
def print_host(self, ep):
|
||||
""" Callback fired once the HS is available
|
||||
|
|
@ -204,6 +231,14 @@ class JMHiddenService(object):
|
|||
"""
|
||||
self.onion_hostname_callback(self.onion.hostname)
|
||||
|
||||
def print_host_filesystem(self, port):
|
||||
""" As above but needed to respect slightly different
|
||||
callback chain for this case (where we start our own tor
|
||||
instance for the filesystem-based onion).
|
||||
"""
|
||||
self.onion = port.onion_service
|
||||
self.onion_hostname_callback(self.onion.hostname)
|
||||
|
||||
def shutdown(self):
|
||||
self.tor_connection.protocol.transport.loseConnection()
|
||||
self.info_callback("Hidden service shutdown complete")
|
||||
|
|
|
|||
|
|
@ -24,7 +24,7 @@ from .cryptoengine import (BTCEngine, BTC_P2PKH, BTC_P2SH_P2WPKH, BTC_P2WPKH, En
|
|||
TYPE_P2PKH, TYPE_P2SH_P2WPKH, TYPE_P2WPKH, detect_script_type)
|
||||
from .configure import (load_test_config, process_shutdown,
|
||||
load_program_config, jm_single, get_network, update_persist_config,
|
||||
validate_address, is_burn_destination, get_irc_mchannels,
|
||||
validate_address, is_burn_destination, get_mchannels,
|
||||
get_blockchain_interface_instance, set_config, is_segwit_mode,
|
||||
is_native_segwit_mode, JMPluginService, get_interest_rate, get_bondless_makers_allowance)
|
||||
from .blockchaininterface import (BlockchainInterface,
|
||||
|
|
|
|||
|
|
@ -15,7 +15,7 @@ import os
|
|||
import sys
|
||||
from jmbase import (get_log, EXIT_FAILURE, hextobin, bintohex,
|
||||
utxo_to_utxostr, bdict_sdict_convert)
|
||||
from jmclient import (jm_single, get_irc_mchannels,
|
||||
from jmclient import (jm_single, get_mchannels,
|
||||
RegtestBitcoinCoreInterface,
|
||||
SNICKERReceiver, process_shutdown)
|
||||
import jmbitcoin as btc
|
||||
|
|
@ -434,7 +434,7 @@ class JMMakerClientProtocol(JMClientProtocol):
|
|||
"blockchain_source")
|
||||
#needed only for channel naming convention
|
||||
network = jm_single().config.get("BLOCKCHAIN", "network")
|
||||
irc_configs = get_irc_mchannels()
|
||||
irc_configs = self.factory.get_mchannels()
|
||||
#only here because Init message uses this field; not used by makers TODO
|
||||
minmakers = jm_single().config.getint("POLICY", "minimum_makers")
|
||||
maker_timeout_sec = jm_single().maker_timeout_sec
|
||||
|
|
@ -601,7 +601,7 @@ class JMTakerClientProtocol(JMClientProtocol):
|
|||
"blockchain_source")
|
||||
#needed only for channel naming convention
|
||||
network = jm_single().config.get("BLOCKCHAIN", "network")
|
||||
irc_configs = get_irc_mchannels()
|
||||
irc_configs = self.factory.get_mchannels()
|
||||
minmakers = jm_single().config.getint("POLICY", "minimum_makers")
|
||||
maker_timeout_sec = jm_single().maker_timeout_sec
|
||||
|
||||
|
|
@ -795,6 +795,14 @@ class JMClientProtocolFactory(protocol.ClientFactory):
|
|||
def buildProtocol(self, addr):
|
||||
return self.protocol(self, self.client)
|
||||
|
||||
def get_mchannels(self):
|
||||
""" A transparent wrapper that allows override,
|
||||
so that a script can return a customised set of
|
||||
message channel configs; currently used for testing
|
||||
multiple bots on regtest.
|
||||
"""
|
||||
return get_mchannels()
|
||||
|
||||
def start_reactor(host, port, factory=None, snickerfactory=None,
|
||||
bip78=False, jm_coinjoin=True, ish=True,
|
||||
daemon=False, rs=True, gui=False): #pragma: no cover
|
||||
|
|
|
|||
|
|
@ -140,6 +140,9 @@ rpc_wallet_file =
|
|||
## SERVER 1/3) Darkscience IRC (Tor, IP)
|
||||
################################################################################
|
||||
[MESSAGING:server1]
|
||||
# by default the legacy format without a `type` field is
|
||||
# understood to be IRC, but you can, optionally, add it:
|
||||
# type = irc
|
||||
channel = joinmarket-pit
|
||||
port = 6697
|
||||
usessl = true
|
||||
|
|
@ -154,24 +157,47 @@ socks5 = false
|
|||
#socks5_host = localhost
|
||||
#socks5_port = 9050
|
||||
|
||||
## SERVER 2/3) hackint IRC (Tor, IP)
|
||||
################################################################################
|
||||
[MESSAGING:server2]
|
||||
channel = joinmarket-pit
|
||||
[MESSAGING:onion1]
|
||||
# onion based message channels must have the exact type 'onion'
|
||||
# (while the section name above can be MESSAGING:whatever), and there must
|
||||
# be only ONE such message channel configured (note the directory servers
|
||||
# can be multiple, below):
|
||||
type = onion
|
||||
|
||||
# For traditional IP (default):
|
||||
host = irc.hackint.org
|
||||
port = 6697
|
||||
usessl = true
|
||||
socks5 = false
|
||||
socks5_host = localhost
|
||||
socks5_port = 9050
|
||||
|
||||
# For Tor (recommended as clearnet alternative):
|
||||
#host = ncwkrwxpq2ikcngxq3dy2xctuheniggtqeibvgofixpzvrwpa77tozqd.onion
|
||||
#port = 6667
|
||||
#usessl = false
|
||||
#socks5 = true
|
||||
#socks5_host = localhost
|
||||
#socks5_port = 9050
|
||||
# the tor control configuration.
|
||||
# for most people running the tor daemon
|
||||
# on Linux, no changes are required here:
|
||||
tor_control_host = localhost
|
||||
# or, to use a UNIX socket
|
||||
# tor_control_host = unix:/var/run/tor/control
|
||||
tor_control_port = 9051
|
||||
|
||||
# the host/port actually serving the hidden service
|
||||
# (note the *virtual port*, that the client uses,
|
||||
# is hardcoded to 80):
|
||||
onion_serving_host = 127.0.0.1
|
||||
onion_serving_port = 8080
|
||||
|
||||
# directory node configuration
|
||||
#
|
||||
# This is mandatory for directory nodes (who must also set their
|
||||
# own *.onion:port as the only directory in directory_nodes, below),
|
||||
# but NOT TO BE USED by non-directory nodes (which is you, unless
|
||||
# you know otherwise!), as it will greatly degrade your privacy.
|
||||
# (note the default is no value, don't replace it with "").
|
||||
hidden_service_dir =
|
||||
#
|
||||
# This is a comma separated list (comma can be omitted if only one item).
|
||||
# Each item has format host:port ; both are required, though port will
|
||||
# be 80 if created in this code.
|
||||
directory_nodes = rr6f6qtleiiwic45bby4zwmiwjrj3jsbmcvutwpqxjziaydjydkk5iad.onion:80
|
||||
|
||||
# This setting is ONLY for developer regtest setups,
|
||||
# running multiple bots at once. Don't alter it otherwise
|
||||
regtest_count = 0,0
|
||||
|
||||
## SERVER 3/3) ILITA IRC (Tor - disabled by default)
|
||||
################################################################################
|
||||
|
|
@ -484,7 +510,7 @@ def set_config(cfg, bcint=None):
|
|||
global_singleton.bc_interface = bcint
|
||||
|
||||
|
||||
def get_irc_mchannels():
|
||||
def get_mchannels():
|
||||
SECTION_NAME = 'MESSAGING'
|
||||
# FIXME: remove in future release
|
||||
if jm_single().config.has_section(SECTION_NAME):
|
||||
|
|
@ -495,16 +521,30 @@ def get_irc_mchannels():
|
|||
return _get_irc_mchannels_old()
|
||||
|
||||
SECTION_NAME += ':'
|
||||
irc_sections = []
|
||||
sections = []
|
||||
for s in jm_single().config.sections():
|
||||
if s.startswith(SECTION_NAME):
|
||||
irc_sections.append(s)
|
||||
assert irc_sections
|
||||
sections.append(s)
|
||||
assert sections
|
||||
|
||||
req_fields = [("host", str), ("port", int), ("channel", str), ("usessl", str)]
|
||||
irc_fields = [("host", str), ("port", int), ("channel", str), ("usessl", str),
|
||||
("socks5", str), ("socks5_host", str), ("socks5_port", str)]
|
||||
onion_fields = [("type", str), ("directory_nodes", str), ("regtest_count", str),
|
||||
("socks5_host", str), ("socks5_port", int),
|
||||
("tor_control_host", str), ("tor_control_port", int),
|
||||
("onion_serving_host", str), ("onion_serving_port", int),
|
||||
("hidden_service_dir", str)]
|
||||
|
||||
configs = []
|
||||
for section in irc_sections:
|
||||
|
||||
# processing the IRC sections:
|
||||
for section in sections:
|
||||
if jm_single().config.has_option(section, "type"):
|
||||
# legacy IRC configs do not have "type" but just
|
||||
# in case, we'll allow the "irc" type:
|
||||
if not jm_single().config.get(section, "type").lower(
|
||||
) == "irc":
|
||||
break
|
||||
server_data = {}
|
||||
|
||||
# check if socks5 is enabled for tor and load relevant config if so
|
||||
|
|
@ -516,13 +556,30 @@ def get_irc_mchannels():
|
|||
server_data["socks5_host"] = jm_single().config.get(section, "socks5_host")
|
||||
server_data["socks5_port"] = jm_single().config.get(section, "socks5_port")
|
||||
|
||||
for option, otype in req_fields:
|
||||
for option, otype in irc_fields:
|
||||
val = jm_single().config.get(section, option)
|
||||
server_data[option] = otype(val)
|
||||
server_data['btcnet'] = get_network()
|
||||
configs.append(server_data)
|
||||
return configs
|
||||
|
||||
# processing the onion sections:
|
||||
for section in sections:
|
||||
if not jm_single().config.has_option(section, "type") or \
|
||||
not jm_single().config.get(section, "type").lower() == "onion":
|
||||
continue
|
||||
onion_data = {}
|
||||
for option, otype in onion_fields:
|
||||
try:
|
||||
val = jm_single().config.get(section, option)
|
||||
except NoOptionError:
|
||||
continue
|
||||
onion_data[option] = otype(val)
|
||||
onion_data['btcnet'] = get_network()
|
||||
# Just to allow a dynamic set of var:
|
||||
onion_data["section-name"] = section
|
||||
configs.append(onion_data)
|
||||
|
||||
return configs
|
||||
|
||||
def _get_irc_mchannels_old():
|
||||
fields = [("host", str), ("port", int), ("channel", str), ("usessl", str),
|
||||
|
|
@ -651,28 +708,6 @@ def load_program_config(config_path="", bs=None, plugin_services=[]):
|
|||
"settings and restart joinmarket.", "info")
|
||||
sys.exit(EXIT_FAILURE)
|
||||
|
||||
#These are left as sanity checks but currently impossible
|
||||
#since any edits are overlays to the default, these sections/options will
|
||||
#always exist.
|
||||
# FIXME: This check is a best-effort attempt. Certain incorrect section
|
||||
# names can pass and so can non-first invalid sections.
|
||||
for s in required_options: #pragma: no cover
|
||||
# check for sections
|
||||
avail = None
|
||||
if not global_singleton.config.has_section(s):
|
||||
for avail in global_singleton.config.sections():
|
||||
if avail.startswith(s):
|
||||
break
|
||||
else:
|
||||
raise Exception(
|
||||
"Config file does not contain the required section: " + s)
|
||||
# then check for specific options
|
||||
k = avail or s
|
||||
for o in required_options[s]:
|
||||
if not global_singleton.config.has_option(k, o):
|
||||
raise Exception("Config file does not contain the required "
|
||||
"option '{}' in section '{}'.".format(o, k))
|
||||
|
||||
loglevel = global_singleton.config.get("LOGGING", "console_log_level")
|
||||
try:
|
||||
set_logging_level(loglevel)
|
||||
|
|
@ -742,6 +777,11 @@ def load_program_config(config_path="", bs=None, plugin_services=[]):
|
|||
if not os.path.exists(plogsdir):
|
||||
os.makedirs(plogsdir)
|
||||
p.set_log_dir(plogsdir)
|
||||
# Check if a onion message channel was configured, and if so,
|
||||
# check there is only 1; multiple directory nodes will be inside the config.
|
||||
chans = get_mchannels()
|
||||
onion_chans = [x for x in chans if "type" in x and x["type"] == "onion"]
|
||||
assert len(onion_chans) < 2
|
||||
|
||||
def load_test_config(**kwargs):
|
||||
if "config_path" not in kwargs:
|
||||
|
|
|
|||
|
|
@ -159,6 +159,9 @@ class JMWalletDaemon(Service):
|
|||
# can be shut down cleanly:
|
||||
self.coinjoin_connection = None
|
||||
|
||||
def get_client_factory(self):
|
||||
return JMClientProtocolFactory(self.taker)
|
||||
|
||||
def activate_coinjoin_state(self, state):
|
||||
""" To be set when a maker or taker
|
||||
operation is initialized; they cannot
|
||||
|
|
@ -420,7 +423,8 @@ class JMWalletDaemon(Service):
|
|||
walletname=self.wallet_name,
|
||||
token=self.cookie)
|
||||
|
||||
def taker_finished(self, res, fromtx=False, waittime=0.0, txdetails=None):
|
||||
def taker_finished(self, res, fromtx=False,
|
||||
waittime=0.0, txdetails=None):
|
||||
# This is a slimmed down version compared with what is seen in
|
||||
# the CLI code, since that code encompasses schedules with multiple
|
||||
# entries; for now, the RPC only supports single joins.
|
||||
|
|
@ -1003,13 +1007,13 @@ class JMWalletDaemon(Service):
|
|||
self.taker = Taker(self.services["wallet"], schedule,
|
||||
max_cj_fee = max_cj_fee,
|
||||
callbacks=(self.filter_orders_callback,
|
||||
None, self.taker_finished))
|
||||
None, self.taker_finished))
|
||||
# TODO ; this makes use of a pre-existing hack to allow
|
||||
# selectively disabling the stallMonitor function that checks
|
||||
# if transactions went through or not; here we want to cleanly
|
||||
# destroy the Taker after an attempt is made, successful or not.
|
||||
self.taker.testflag = True
|
||||
self.clientfactory = JMClientProtocolFactory(self.taker)
|
||||
self.clientfactory = self.get_client_factory()
|
||||
|
||||
dhost, dport = self.check_daemon_ready()
|
||||
|
||||
|
|
|
|||
|
|
@ -4,6 +4,7 @@ from .protocol import *
|
|||
from .enc_wrapper import as_init_encryption, decode_decrypt, \
|
||||
encrypt_encode, init_keypair, init_pubkey, get_pubkey, NaclError
|
||||
from .irc import IRCMessageChannel
|
||||
from .onionmc import OnionMessageChannel
|
||||
from jmbase.support import get_log
|
||||
from .message_channel import MessageChannel, MessageChannelCollection
|
||||
from .orderbookwatch import OrderbookWatch
|
||||
|
|
|
|||
|
|
@ -7,8 +7,9 @@ from .enc_wrapper import (as_init_encryption, init_keypair, init_pubkey,
|
|||
from .protocol import (COMMAND_PREFIX, ORDER_KEYS, NICK_HASH_LENGTH,
|
||||
NICK_MAX_ENCODED, JM_VERSION, JOINMARKET_NICK_HEADER,
|
||||
COMMITMENT_PREFIXES)
|
||||
from .irc import IRCMessageChannel
|
||||
|
||||
from .irc import IRCMessageChannel
|
||||
from .onionmc import OnionMessageChannel
|
||||
from jmbase import (is_hs_uri, get_tor_agent, JMHiddenService,
|
||||
get_nontor_agent, BytesProducer, wrapped_urlparse,
|
||||
bdict_sdict_convert, JMHTTPResource)
|
||||
|
|
@ -527,10 +528,15 @@ class JMDaemonServerProtocol(amp.AMP, OrderbookWatch):
|
|||
self.mc_shutdown()
|
||||
self.irc_configs = irc_configs
|
||||
self.restart_mc_required = True
|
||||
mcs = [IRCMessageChannel(c,
|
||||
daemon=self,
|
||||
realname='btcint=' + bcsource)
|
||||
for c in self.irc_configs]
|
||||
mcs = []
|
||||
for c in self.irc_configs:
|
||||
if "type" in c and c["type"] == "onion":
|
||||
mcs.append(OnionMessageChannel(c, daemon=self))
|
||||
else:
|
||||
# default is IRC; TODO allow others
|
||||
mcs.append(IRCMessageChannel(c,
|
||||
daemon=self,
|
||||
realname='btcint=' + bcsource))
|
||||
self.mcc = MessageChannelCollection(mcs)
|
||||
OrderbookWatch.set_msgchan(self, self.mcc)
|
||||
#register taker-specific msgchan callbacks here
|
||||
|
|
@ -947,6 +953,7 @@ class JMDaemonServerProtocol(amp.AMP, OrderbookWatch):
|
|||
incomplete transaction is wiped.
|
||||
"""
|
||||
self.jm_state = 0 #uninited
|
||||
self.mcc.set_nick(nick)
|
||||
if self.restart_mc_required:
|
||||
self.mcc.run()
|
||||
self.restart_mc_required = False
|
||||
|
|
@ -954,7 +961,6 @@ class JMDaemonServerProtocol(amp.AMP, OrderbookWatch):
|
|||
#if we are not restarting the MC,
|
||||
#we must simulate the on_welcome message:
|
||||
self.on_welcome()
|
||||
self.mcc.set_nick(nick)
|
||||
|
||||
def transfer_commitment(self, commit):
|
||||
"""Send this commitment via privmsg to one (random)
|
||||
|
|
|
|||
|
|
@ -263,9 +263,9 @@ class MessageChannelCollection(object):
|
|||
#is supposed to be sent. There used to be an exception raise.
|
||||
#to prevent a crash (especially in makers), we just inform
|
||||
#the user about it for now
|
||||
log.error("Tried to communicate on this IRC server but "
|
||||
log.error("Tried to communicate on this message channel but "
|
||||
"failed: " + str(mc))
|
||||
log.error("You might have to comment out this IRC server "
|
||||
log.error("You might have to comment out this message channel"
|
||||
"in joinmarket.cfg and restart.")
|
||||
log.error("No action needed for makers / yield generators!")
|
||||
# todo: add logic to continue on other available mc
|
||||
|
|
@ -444,7 +444,7 @@ class MessageChannelCollection(object):
|
|||
if (not self.on_welcome_announce_id) and self.on_welcome:
|
||||
self.on_welcome_announce_id = reactor.callLater(60, self.on_welcome_setup_finished,)
|
||||
else:
|
||||
log.info("All IRC servers connected, starting execution.")
|
||||
log.info("All message channels connected, starting execution.")
|
||||
if self.on_welcome_announce_id:
|
||||
self.on_welcome_announce_id.cancel()
|
||||
self.on_welcome_setup_finished()
|
||||
|
|
|
|||
1179
jmdaemon/jmdaemon/onionmc.py
Normal file
1179
jmdaemon/jmdaemon/onionmc.py
Normal file
File diff suppressed because it is too large
Load diff
|
|
@ -7,7 +7,7 @@ from jmdaemon.daemon_protocol import JMDaemonServerProtocol
|
|||
from jmdaemon.protocol import NICK_HASH_LENGTH, NICK_MAX_ENCODED, JM_VERSION,\
|
||||
JOINMARKET_NICK_HEADER
|
||||
from jmbase import get_log
|
||||
from jmclient import (load_test_config, jm_single, get_irc_mchannels)
|
||||
from jmclient import (load_test_config, jm_single, get_mchannels)
|
||||
from twisted.python.log import msg as tmsg
|
||||
from twisted.python.log import startLogging
|
||||
from twisted.internet import protocol, reactor, task
|
||||
|
|
@ -59,7 +59,7 @@ class JMTestClientProtocol(JMBaseProtocol):
|
|||
|
||||
def clientStart(self):
|
||||
self.sigs_received = 0
|
||||
irc = get_irc_mchannels()
|
||||
irc = [get_mchannels()[0]]
|
||||
d = self.callRemote(JMInit,
|
||||
bcsource="dummyblockchain",
|
||||
network="dummynetwork",
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@ from twisted.trial import unittest
|
|||
from twisted.internet import reactor, task
|
||||
from jmdaemon import IRCMessageChannel, MessageChannelCollection
|
||||
#needed for test framework
|
||||
from jmclient import (load_test_config, get_irc_mchannels, jm_single)
|
||||
from jmclient import (load_test_config, get_mchannels, jm_single)
|
||||
|
||||
si = 1
|
||||
class DummyDaemon(object):
|
||||
|
|
@ -95,7 +95,7 @@ def junk_fill(mc):
|
|||
|
||||
def getmc(nick):
|
||||
dm = DummyDaemon()
|
||||
mc = DummyMC(get_irc_mchannels()[0], nick, dm)
|
||||
mc = DummyMC(get_mchannels()[0], nick, dm)
|
||||
mc.register_orderbookwatch_callbacks(on_order_seen=on_order_seen)
|
||||
mc.register_taker_callbacks(on_pubkey=on_pubkey)
|
||||
mc.on_connect = on_connect
|
||||
|
|
@ -108,7 +108,7 @@ class TrialIRC(unittest.TestCase):
|
|||
|
||||
def setUp(self):
|
||||
load_test_config()
|
||||
print(get_irc_mchannels()[0])
|
||||
print(get_mchannels()[0])
|
||||
jm_single().maker_timeout_sec = 1
|
||||
dm, mc, mcc = getmc("irc_publisher")
|
||||
dm2, mc2, mcc2 = getmc("irc_receiver")
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@ import pytest
|
|||
|
||||
from jmdaemon.orderbookwatch import OrderbookWatch
|
||||
from jmdaemon import IRCMessageChannel, fidelity_bond_cmd_list
|
||||
from jmclient import get_irc_mchannels, load_test_config
|
||||
from jmclient import get_mchannels, load_test_config
|
||||
from jmdaemon.protocol import JM_VERSION, ORDER_KEYS
|
||||
from jmbase.support import hextobin
|
||||
from jmclient.fidelity_bond import FidelityBondProof
|
||||
|
|
@ -24,7 +24,7 @@ def on_welcome(x):
|
|||
def get_ob():
|
||||
load_test_config()
|
||||
dm = DummyDaemon()
|
||||
mc = DummyMC(get_irc_mchannels()[0], "test", dm)
|
||||
mc = DummyMC(get_mchannels()[0], "test", dm)
|
||||
ob = OrderbookWatch()
|
||||
ob.on_welcome = on_welcome
|
||||
ob.set_msgchan(mc)
|
||||
|
|
|
|||
|
|
@ -44,7 +44,7 @@ if 'matplotlib' in sys.modules:
|
|||
import matplotlib.pyplot as plt
|
||||
|
||||
from jmclient import jm_single, load_program_config, calc_cj_fee, \
|
||||
get_irc_mchannels, add_base_options
|
||||
get_mchannels, add_base_options
|
||||
from jmdaemon import OrderbookWatch, MessageChannelCollection, IRCMessageChannel
|
||||
#TODO this is only for base58, find a solution for a client without jmbitcoin
|
||||
import jmbitcoin as btc
|
||||
|
|
@ -804,7 +804,7 @@ def main():
|
|||
(options, args) = parser.parse_args()
|
||||
load_program_config(config_path=options.datadir)
|
||||
hostport = (options.host, options.port)
|
||||
mcs = [ObIRCMessageChannel(c) for c in get_irc_mchannels()]
|
||||
mcs = [ObIRCMessageChannel(c) for c in get_mchannels()]
|
||||
mcc = MessageChannelCollection(mcs)
|
||||
mcc.set_nick(get_dummy_nick())
|
||||
taker = ObBasic(mcc, hostport)
|
||||
|
|
|
|||
364
test/e2e-coinjoin-test.py
Normal file
364
test/e2e-coinjoin-test.py
Normal file
|
|
@ -0,0 +1,364 @@
|
|||
#! /usr/bin/env python
|
||||
'''Creates wallets and yield generators in regtest,
|
||||
then runs both them and a JMWalletDaemon instance
|
||||
for the taker, injecting the newly created taker
|
||||
wallet into it and running sendpayment once.
|
||||
Number of ygs is configured in the joinmarket.cfg
|
||||
with `regtest-count` in the `ln-onion` type MESSAGING
|
||||
section.
|
||||
See notes below for more detail on config.
|
||||
Run it like:
|
||||
pytest \
|
||||
--btcroot=/path/to/bitcoin/bin/ \
|
||||
--btcpwd=123456abcdef --btcconf=/blah/bitcoin.conf \
|
||||
-s test/ln-ygrunner.py
|
||||
'''
|
||||
from twisted.internet import reactor, defer
|
||||
from twisted.web.client import readBody, Headers
|
||||
from common import make_wallets
|
||||
import pytest
|
||||
import random
|
||||
import json
|
||||
from datetime import datetime
|
||||
from jmbase import (get_nontor_agent, BytesProducer, jmprint,
|
||||
get_log, stop_reactor, hextobin, bintohex)
|
||||
from jmclient import (YieldGeneratorBasic, load_test_config, jm_single,
|
||||
JMClientProtocolFactory, start_reactor, SegwitWallet, get_mchannels,
|
||||
SegwitLegacyWallet, JMWalletDaemon)
|
||||
from jmclient.wallet_utils import wallet_gettimelockaddress
|
||||
from jmclient.wallet_rpc import api_version_string
|
||||
|
||||
log = get_log()
|
||||
|
||||
# For quicker testing, restrict the range of timelock
|
||||
# addresses to avoid slow load of multiple bots.
|
||||
# Note: no need to revert this change as ygrunner runs
|
||||
# in isolation.
|
||||
from jmclient import FidelityBondMixin
|
||||
FidelityBondMixin.TIMELOCK_ERA_YEARS = 2
|
||||
FidelityBondMixin.TIMELOCK_EPOCH_YEAR = datetime.now().year
|
||||
FidelityBondMixin.TIMENUMBERS_PER_PUBKEY = 12
|
||||
|
||||
wallet_name = "test-onion-yg-runner.jmdat"
|
||||
|
||||
mean_amt = 2.0
|
||||
|
||||
directory_node_indices = [1]
|
||||
|
||||
#
|
||||
def get_onion_messaging_config_regtest(run_num: int, dns=[1], hsd=""):
|
||||
""" Sets a onion messaging channel section for a regtest instance
|
||||
indexed by `run_num`. The indices to be used as directory nodes
|
||||
should be passed as `dns`, as a list of ints.
|
||||
"""
|
||||
def location_string(directory_node_run_num):
|
||||
return "127.0.0.1:" + str(
|
||||
8080 + directory_node_run_num)
|
||||
if run_num in dns:
|
||||
# means *we* are a dn, and dns currently
|
||||
# do not use other dns:
|
||||
dns_to_use = [location_string(run_num)]
|
||||
else:
|
||||
dns_to_use = [location_string(a) for a in dns]
|
||||
dn_nodes_list = ",".join(dns_to_use)
|
||||
log.info("For node: {}, set dn list to: {}".format(run_num, dn_nodes_list))
|
||||
cf = {"type": "onion",
|
||||
"socks5_host": "127.0.0.1",
|
||||
"socks5_port": 9050,
|
||||
"tor_control_host": "127.0.0.1",
|
||||
"tor_control_port": 9051,
|
||||
"onion_serving_host": "127.0.0.1",
|
||||
"onion_serving_port": 8080 + run_num,
|
||||
"hidden_service_dir": "",
|
||||
"directory_nodes": dn_nodes_list,
|
||||
"regtest_count": "1, 1"}
|
||||
if run_num in dns:
|
||||
# only directories need to use fixed hidden service directories:
|
||||
cf["hidden_service_dir"] = hsd
|
||||
return cf
|
||||
|
||||
|
||||
class RegtestJMClientProtocolFactory(JMClientProtocolFactory):
|
||||
i = 1
|
||||
def set_directory_nodes(self, dns):
|
||||
# a list of integers representing the directory nodes
|
||||
# for this test:
|
||||
self.dns = dns
|
||||
|
||||
def get_mchannels(self):
|
||||
# swaps out any existing lightning configs
|
||||
# in the config settings on startup, for one
|
||||
# that's indexed to the regtest counter var:
|
||||
default_chans = get_mchannels()
|
||||
new_chans = []
|
||||
onion_found = False
|
||||
hsd = ""
|
||||
for c in default_chans:
|
||||
if "type" in c and c["type"] == "onion":
|
||||
onion_found = True
|
||||
if c["hidden_service_dir"] != "":
|
||||
hsd = c["hidden_service_dir"]
|
||||
continue
|
||||
else:
|
||||
new_chans.append(c)
|
||||
if onion_found:
|
||||
new_chans.append(get_onion_messaging_config_regtest(
|
||||
self.i, self.dns, hsd))
|
||||
return new_chans
|
||||
|
||||
class JMWalletDaemonT(JMWalletDaemon):
|
||||
def check_cookie(self, request):
|
||||
if self.auth_disabled:
|
||||
return True
|
||||
return super().check_cookie(request)
|
||||
|
||||
class TWalletRPCManager(object):
|
||||
""" Base class for set up of tests of the
|
||||
Wallet RPC calls using the wallet_rpc.JMWalletDaemon service.
|
||||
"""
|
||||
# the port for the jmwallet daemon
|
||||
dport = 28183
|
||||
# the port for the ws
|
||||
wss_port = 28283
|
||||
|
||||
def __init__(self):
|
||||
# a client connnection object which is often but not always
|
||||
# instantiated:
|
||||
self.client_connector = None
|
||||
self.daemon = JMWalletDaemonT(self.dport, self.wss_port, tls=False)
|
||||
self.daemon.auth_disabled = True
|
||||
# because we sync and start the wallet service manually here
|
||||
# (and don't use wallet files yet), we won't have set a wallet name,
|
||||
# so we set it here:
|
||||
self.daemon.wallet_name = wallet_name
|
||||
|
||||
def start(self):
|
||||
r, s = self.daemon.startService()
|
||||
self.listener_rpc = r
|
||||
self.listener_ws = s
|
||||
|
||||
def get_route_root(self):
|
||||
addr = "http://127.0.0.1:" + str(self.dport)
|
||||
addr += api_version_string
|
||||
return addr
|
||||
|
||||
def stop(self):
|
||||
for dc in reactor.getDelayedCalls():
|
||||
dc.cancel()
|
||||
d1 = defer.maybeDeferred(self.listener_ws.stopListening)
|
||||
d2 = defer.maybeDeferred(self.listener_rpc.stopListening)
|
||||
if self.client_connector:
|
||||
self.client_connector.disconnect()
|
||||
# only fire if everything is finished:
|
||||
return defer.gatherResults([d1, d2])
|
||||
|
||||
@defer.inlineCallbacks
|
||||
def do_request(self, agent, method, addr, body, handler, token=None):
|
||||
if token:
|
||||
headers = Headers({"Authorization": ["Bearer " + self.jwt_token]})
|
||||
else:
|
||||
headers = None
|
||||
response = yield agent.request(method, addr, headers, bodyProducer=body)
|
||||
yield self.response_handler(response, handler)
|
||||
|
||||
@defer.inlineCallbacks
|
||||
def response_handler(self, response, handler):
|
||||
body = yield readBody(response)
|
||||
# these responses should always be 200 OK.
|
||||
#assert response.code == 200
|
||||
# handlers check the body is as expected; no return.
|
||||
yield handler(body)
|
||||
return True
|
||||
|
||||
def test_start_yg_and_taker_setup(setup_onion_ygrunner):
|
||||
"""Set up some wallets, for the ygs and 1 taker.
|
||||
Then start LN and the ygs in the background, then fire
|
||||
a startup of a wallet daemon for the taker who then
|
||||
makes a coinjoin payment.
|
||||
"""
|
||||
if jm_single().config.get("POLICY", "native") == "true":
|
||||
walletclass = SegwitWallet
|
||||
else:
|
||||
# TODO add Legacy
|
||||
walletclass = SegwitLegacyWallet
|
||||
|
||||
start_bot_num, end_bot_num = [int(x) for x in jm_single().config.get(
|
||||
"MESSAGING:onion1", "regtest_count").split(",")]
|
||||
num_ygs = end_bot_num - start_bot_num
|
||||
# specify the number of wallets and bots of each type:
|
||||
wallet_services = make_wallets(num_ygs + 1,
|
||||
wallet_structures=[[1, 3, 0, 0, 0]] * (num_ygs + 1),
|
||||
mean_amt=2.0,
|
||||
walletclass=walletclass)
|
||||
#the sendpayment bot uses the last wallet in the list
|
||||
wallet_service = wallet_services[end_bot_num - 1]['wallet']
|
||||
jmprint("\n\nTaker wallet seed : " + wallet_services[end_bot_num - 1]['seed'])
|
||||
# for manual audit if necessary, show the maker's wallet seeds
|
||||
# also (note this audit should be automated in future, see
|
||||
# test_full_coinjoin.py in this directory)
|
||||
jmprint("\n\nMaker wallet seeds: ")
|
||||
for i in range(start_bot_num, end_bot_num):
|
||||
jmprint("Maker seed: " + wallet_services[i - 1]['seed'])
|
||||
jmprint("\n")
|
||||
wallet_service.sync_wallet(fast=True)
|
||||
ygclass = YieldGeneratorBasic
|
||||
|
||||
# As per previous note, override non-default command line settings:
|
||||
options = {}
|
||||
for x in ["ordertype", "txfee_contribution", "txfee_contribution_factor",
|
||||
"cjfee_a", "cjfee_r", "cjfee_factor", "minsize", "size_factor"]:
|
||||
options[x] = jm_single().config.get("YIELDGENERATOR", x)
|
||||
ordertype = options["ordertype"]
|
||||
txfee_contribution = int(options["txfee_contribution"])
|
||||
txfee_contribution_factor = float(options["txfee_contribution_factor"])
|
||||
cjfee_factor = float(options["cjfee_factor"])
|
||||
size_factor = float(options["size_factor"])
|
||||
if ordertype == 'reloffer':
|
||||
cjfee_r = options["cjfee_r"]
|
||||
# minimum size is such that you always net profit at least 20%
|
||||
#of the miner fee
|
||||
minsize = max(int(1.2 * txfee_contribution / float(cjfee_r)),
|
||||
int(options["minsize"]))
|
||||
cjfee_a = None
|
||||
elif ordertype == 'absoffer':
|
||||
cjfee_a = int(options["cjfee_a"])
|
||||
minsize = int(options["minsize"])
|
||||
cjfee_r = None
|
||||
else:
|
||||
assert False, "incorrect offertype config for yieldgenerator."
|
||||
|
||||
txtype = wallet_service.get_txtype()
|
||||
if txtype == "p2wpkh":
|
||||
prefix = "sw0"
|
||||
elif txtype == "p2sh-p2wpkh":
|
||||
prefix = "sw"
|
||||
elif txtype == "p2pkh":
|
||||
prefix = ""
|
||||
else:
|
||||
assert False, "Unsupported wallet type for yieldgenerator: " + txtype
|
||||
|
||||
ordertype = prefix + ordertype
|
||||
|
||||
for i in range(start_bot_num, end_bot_num):
|
||||
cfg = [txfee_contribution, cjfee_a, cjfee_r, ordertype, minsize,
|
||||
txfee_contribution_factor, cjfee_factor, size_factor]
|
||||
wallet_service_yg = wallet_services[i - 1]["wallet"]
|
||||
|
||||
wallet_service_yg.startService()
|
||||
|
||||
yg = ygclass(wallet_service_yg, cfg)
|
||||
clientfactory = RegtestJMClientProtocolFactory(yg, proto_type="MAKER")
|
||||
# This ensures that the right rpc/port config is passed into the daemon,
|
||||
# for this specific bot:
|
||||
clientfactory.i = i
|
||||
# This ensures that this bot knows which other bots are directory nodes:
|
||||
clientfactory.set_directory_nodes(directory_node_indices)
|
||||
nodaemon = jm_single().config.getint("DAEMON", "no_daemon")
|
||||
daemon = True if nodaemon == 1 else False
|
||||
#rs = True if i == num_ygs - 1 else False
|
||||
start_reactor(jm_single().config.get("DAEMON", "daemon_host"),
|
||||
jm_single().config.getint("DAEMON", "daemon_port"),
|
||||
clientfactory, daemon=daemon, rs=False)
|
||||
reactor.callLater(1.0, start_test_taker, wallet_services[end_bot_num - 1]['wallet'], end_bot_num)
|
||||
reactor.run()
|
||||
|
||||
@defer.inlineCallbacks
|
||||
def start_test_taker(wallet_service, i):
|
||||
# this rpc manager has auth disabled,
|
||||
# and the wallet_service is set manually,
|
||||
# so no unlock etc.
|
||||
mgr = TWalletRPCManager()
|
||||
mgr.daemon.wallet_service = wallet_service
|
||||
# because we are manually setting the wallet_service
|
||||
# of the JMWalletDaemon instance, we do not follow the
|
||||
# usual flow of `initialize_wallet_service`, we do not set
|
||||
# the auth token or start the websocket; so we must manually
|
||||
# sync the wallet, including bypassing any restart callback:
|
||||
def dummy_restart_callback(msg):
|
||||
log.warn("Ignoring rescan request from backend wallet service: " + msg)
|
||||
mgr.daemon.wallet_service.add_restart_callback(dummy_restart_callback)
|
||||
mgr.daemon.wallet_name = wallet_name
|
||||
while not mgr.daemon.wallet_service.synced:
|
||||
mgr.daemon.wallet_service.sync_wallet(fast=True)
|
||||
mgr.daemon.wallet_service.startService()
|
||||
def get_client_factory():
|
||||
clientfactory = RegtestJMClientProtocolFactory(mgr.daemon.taker,
|
||||
proto_type="TAKER")
|
||||
clientfactory.i = i
|
||||
clientfactory.set_directory_nodes(directory_node_indices)
|
||||
return clientfactory
|
||||
|
||||
mgr.daemon.get_client_factory = get_client_factory
|
||||
# before preparing the RPC call to the wallet daemon,
|
||||
# we decide a coinjoin destination and amount. Choosing
|
||||
# a destination in the wallet is a bit easier because
|
||||
# we can query the mixdepth balance at the end.
|
||||
coinjoin_destination = mgr.daemon.wallet_service.get_internal_addr(4)
|
||||
cj_amount = 22000000
|
||||
# once the taker is finished we sanity check before
|
||||
# shutting down:
|
||||
def dummy_taker_finished(res, fromtx=False,
|
||||
waittime=0.0, txdetails=None):
|
||||
jmprint("Taker is finished")
|
||||
# check that the funds have arrived.
|
||||
mbal = mgr.daemon.wallet_service.get_balance_by_mixdepth()[4]
|
||||
assert mbal == cj_amount
|
||||
jmprint("Funds: {} sats successfully arrived into mixdepth 4.".format(cj_amount))
|
||||
stop_reactor()
|
||||
mgr.daemon.taker_finished = dummy_taker_finished
|
||||
mgr.start()
|
||||
agent = get_nontor_agent()
|
||||
addr = mgr.get_route_root()
|
||||
addr += "/wallet/"
|
||||
addr += mgr.daemon.wallet_name
|
||||
addr += "/taker/coinjoin"
|
||||
addr = addr.encode()
|
||||
body = BytesProducer(json.dumps({"mixdepth": "1",
|
||||
"amount_sats": cj_amount,
|
||||
"counterparties": "2",
|
||||
"destination": coinjoin_destination}).encode())
|
||||
yield mgr.do_request(agent, b"POST", addr, body,
|
||||
process_coinjoin_response)
|
||||
|
||||
def process_coinjoin_response(response):
|
||||
json_body = json.loads(response.decode("utf-8"))
|
||||
print("coinjoin response: {}".format(json_body))
|
||||
|
||||
def get_addr_and_fund(yg):
|
||||
""" This function allows us to create
|
||||
and publish a fidelity bond for a particular
|
||||
yield generator object after the wallet has reached
|
||||
a synced state and is therefore ready to serve up
|
||||
timelock addresses. We create the TL address, fund it,
|
||||
refresh the wallet and then republish our offers, which
|
||||
will also publish the new FB.
|
||||
"""
|
||||
if not yg.wallet_service.synced:
|
||||
return
|
||||
if yg.wallet_service.timelock_funded:
|
||||
return
|
||||
addr = wallet_gettimelockaddress(yg.wallet_service.wallet, "2021-11")
|
||||
print("Got timelockaddress: {}".format(addr))
|
||||
|
||||
# pay into it; amount is randomized for now.
|
||||
# Note that grab_coins already mines 1 block.
|
||||
fb_amt = random.randint(1, 5)
|
||||
jm_single().bc_interface.grab_coins(addr, fb_amt)
|
||||
|
||||
# we no longer have to run this loop (TODO kill with nonlocal)
|
||||
yg.wallet_service.timelock_funded = True
|
||||
|
||||
# force wallet to check for the new coins so the new
|
||||
# yg offers will include them:
|
||||
yg.wallet_service.transaction_monitor()
|
||||
|
||||
# publish a new offer:
|
||||
yg.offerlist = yg.create_my_orders()
|
||||
yg.fidelity_bond = yg.get_fidelity_bond_template()
|
||||
jmprint('updated offerlist={}'.format(yg.offerlist))
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def setup_onion_ygrunner():
|
||||
load_test_config()
|
||||
jm_single().bc_interface.tick_forward_chain_interval = 10
|
||||
jm_single().bc_interface.simulate_blocks()
|
||||
|
|
@ -16,6 +16,7 @@ network = testnet
|
|||
rpc_wallet_file = jm-test-wallet
|
||||
|
||||
[MESSAGING:server1]
|
||||
type = irc
|
||||
host = localhost
|
||||
hostid = localhost1
|
||||
channel = joinmarket-pit
|
||||
|
|
@ -26,6 +27,7 @@ socks5_host = localhost
|
|||
socks5_port = 9150
|
||||
|
||||
[MESSAGING:server2]
|
||||
type = irc
|
||||
host = localhost
|
||||
hostid = localhost2
|
||||
channel = joinmarket-pit
|
||||
|
|
@ -35,8 +37,46 @@ socks5 = false
|
|||
socks5_host = localhost
|
||||
socks5_port = 9150
|
||||
|
||||
[MESSAGING:onion1]
|
||||
# onion based message channels must have the exact type 'onion'
|
||||
# (while the section name above can be MESSAGING:whatever), and there must
|
||||
# be only ONE such message channel configured (note the directory servers
|
||||
# can be multiple, below):
|
||||
type = onion
|
||||
socks5_host = localhost
|
||||
socks5_port = 9050
|
||||
# the tor control configuration:
|
||||
tor_control_host = localhost
|
||||
# or, to use a UNIX socket
|
||||
# control_host = unix:/var/run/tor/control
|
||||
tor_control_port = 9051
|
||||
# the host/port actually serving the hidden service
|
||||
# (note the *virtual port*, that the client uses,
|
||||
# is hardcoded to 80):
|
||||
onion_serving_host = 127.0.0.1
|
||||
onion_serving_port = 8080
|
||||
# This is mandatory for directory nodes (who must also set their
|
||||
# own .onion:port as the only directory in directory_nodes, below),
|
||||
# but NOT TO BE USED by non-directory nodes (which is you, unless
|
||||
# you know otherwise!), as it will greatly degrade your privacy.
|
||||
#
|
||||
# Special handling on regtest, so just ignore and let the code handle it:
|
||||
hidden_service_dir = ""
|
||||
# This is a comma separated list (comma can be omitted if only one item).
|
||||
# Each item has format host:port
|
||||
# On regtest we are going to increment the port numbers served from, with
|
||||
# the value used here as the starting value:
|
||||
directory_nodes = localhost:8081
|
||||
# this is not present in default real config
|
||||
# and is specifically used to flag tests:
|
||||
# means we use indices 1,2,3,4,5:
|
||||
regtest_count=1,5
|
||||
|
||||
[TIMEOUT]
|
||||
maker_timeout_sec = 15
|
||||
maker_timeout_sec = 10
|
||||
|
||||
[LOGGING]
|
||||
console_log_level = DEBUG
|
||||
|
||||
[POLICY]
|
||||
# for dust sweeping, try merge_algorithm = gradual
|
||||
|
|
|
|||
|
|
@ -96,7 +96,7 @@ class DeterministicMaliciousYieldGenerator(YieldGeneratorBasic):
|
|||
"num_ygs, wallet_structures, fb_indices, mean_amt, malicious, deterministic",
|
||||
[
|
||||
# 1sp 3yg, honest makers, one maker has FB:
|
||||
(3, [[1, 3, 0, 0, 0]] * 4, [1, 2], 2, 0, False),
|
||||
(3, [[1, 3, 0, 0, 0]] * 4, [], 2, 0, False),
|
||||
# 1sp 3yg, malicious makers reject on auth and on tx 30% of time
|
||||
#(3, [[1, 3, 0, 0, 0]] * 4, 2, 30, False),
|
||||
# 1 sp 9 ygs, deterministically malicious 50% of time
|
||||
|
|
@ -173,6 +173,7 @@ def test_start_ygs(setup_ygrunner, num_ygs, wallet_structures, fb_indices,
|
|||
ygclass = DeterministicMaliciousYieldGenerator
|
||||
else:
|
||||
ygclass = MaliciousYieldGenerator
|
||||
|
||||
for i in range(num_ygs):
|
||||
cfg = [txfee_contribution, cjfee_a, cjfee_r, ordertype, minsize,
|
||||
txfee_contribution_factor, cjfee_factor, size_factor]
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue