Commit graph

43 commits

Author SHA1 Message Date
Calin Culianu
0bf3d4e2ae
refactor ChainInfo into its own struct 2019-11-27 12:12:16 +02:00
Calin Culianu
bb1d371d2e
Use getblockchaininfo rather than getblockcount, and more
We now query more information initially when synching, which can be
useful.

We also deal with initial block download by retrying every minute

and other miscellany
2019-11-27 12:03:12 +02:00
Calin Culianu
1743a4728e
Some nits. Use of an unordered_map and static_assert, simplified map.emplace, show hdr dl pct 2019-11-27 02:38:39 +02:00
Calin Culianu
cc8ce73c5a
Made bitcoind communication more efficient by use of ParseHexFast
When reading data from bitcoind, we can assume the hex data it gives us
is good and there is no need to validate on a per-character basis.

Thus, we use ParseHexFast to shave off cpu cycles (it's like 6x faster
than Qt's fromHex()).

This, along with the Util::ToHexFast() (which is 60% faster than
QByteArray::toHex()), will be used in performance critical code.
2019-11-26 22:04:45 +02:00
Calin Culianu
d9f335f1d4
memory/vector usage nits 2019-11-26 12:29:27 +02:00
Calin Culianu
9f2ba890f4
Tuned concurrency even better -- got performane to beat old scheme!
Also in this commit: small nits
2019-11-26 00:18:28 +02:00
Calin Culianu
b41957b811
improved parallelism / bitcoind utilization 2019-11-25 23:34:04 +02:00
Calin Culianu
e6642fa87a
nits 2019-11-25 22:56:46 +02:00
Calin Culianu
0eb50606b9
Added "CtlTask" mechanism to Controller class, plus nits & refactoring
This is more complex than what we had before -- but is more concurrent
and is more production-ready -- it can deal with failures better and now
we also poll bitcoind for new headers.

Still probably can be optimized somehwat -- but getting there!
2019-11-25 22:34:11 +02:00
Calin Culianu
a34681aa31
Added header download to our test code
It's pretty fast. Downloads all headers in 120 seconds or so.  Keeps
them all in memory.
2019-11-24 22:28:26 +02:00
Calin Culianu
b8a507615d
nit 2019-11-24 18:51:22 +02:00
Calin Culianu
7fa213e095
Added "bitcoind is warming up..." messaging
Just a small nit, really.
2019-11-24 18:23:34 +02:00
Calin Culianu
b823f97397
nits 2019-11-19 12:33:50 +02:00
Calin Culianu
abc1aaabd0
tweaks to header dl test code 2019-11-19 10:59:53 +02:00
Calin Culianu
d7e0ba16db
fixup for error parsing integer id: 1000000.. oops. 2019-11-19 10:44:51 +02:00
Calin Culianu
4b59e2233a Increased parallelism for the header sync
Set maxcur to 16, spinning up 3 bitcoind threads.

Seems to have brought header synch down from 170s to 71s on my mainnet
node here locally. Not bad!

Also fixed a bug where selecting bitcoind nodes was failing if >2 nodes.
2019-11-19 01:58:30 +02:00
Calin Culianu
819fa47990 wip 2019-11-18 21:22:30 +02:00
Calin Culianu
9213459817 wip 2019-11-18 21:09:21 +02:00
Calin Culianu
f64def2005
wip 2019-11-18 20:56:12 +02:00
Calin Culianu
0466fcf501
wip 2019-11-18 20:48:22 +02:00
Calin Culianu
89aeb474c5
WIP: playing with dling data from bitcoind in Controller class 2019-11-18 20:29:21 +02:00
Calin Culianu
f4091bb005
Devised a scheme for asynch messaging to bitcoind
And added some test code in Controller to use the scheme.

It's very asynchronous.  Needs some tuning long term to see how it deals
with load and perhaps throttle the requests if we know bitcoind is under
load.

But for right now, for testing, it should work ok.

Also in this commit small nits, plus bugfix to RPC::Message factory
methods (I forgot to populate some instance properties when constructing
the objects).

Additionally, we were registering the qMetaType wrong for RPC::Message
and RPC::Message::Id (this has been fixed).
2019-11-18 14:32:48 +02:00
Calin Culianu
0b6d6f0b70
Initial detection of bitcoinDs going up/down for Controller
Playing with lambdas. Detects when bitcoind goes down. Initial code --
can put hooks in the detection for eg. stopping block processing or
starting it, etc.
2019-11-18 09:47:18 +02:00
Calin Culianu
45e12158db
Added process function to controller, and hooks for bitcoind when ready
Also added the "Fatal" logging class which logs as Error but then
schedules an app exit when invoked.  Only to be used for truly terrible
errors.
2019-11-18 08:29:33 +02:00
Calin Culianu
742cee4090
a nit 2019-11-18 00:34:24 +02:00
Calin Culianu
c0ea7f75a0
Refactored things, added "Controller" class
Controller will manage BitcoinD and perhaps synching to bitcoind. It can
"see" all the servers as well as bitcoind.

Also refactored things a little bit and added some more startup sanity
checks.
2019-11-18 00:21:34 +02:00
Calin Culianu
43b7e915c4
Got rid of more ShuffleUp stuff, added CLI args 2019-11-07 23:55:13 +02:00
Calin Culianu
01d8d01ccd Added stats HTTP server. Also refactored TcpServer 2019-05-09 02:12:58 +03:00
Calin Culianu
386383d6d4 Made our JSON-RPC 2.0 implementation more closely match the spec
- It's still a little more restrictive than the spec allows -- doesn't
like non-positive-int 'id'
- We not made the shuffle.spec status report back 'notifications' to the
client rather than repeated 'result' messages (as those would be out of
spec). The first response to a 'shuffle.spec' request is either
result="pending", result="accepted", or an error response
- Subsequent shuffle.spec status changes come as a method notification
for "shuffle.spec" with params=["accepted"] or params=["some error
message"]

Overall I think our bare bones JSON-RPC will at least not break now as
interop with other impls.
2019-05-06 14:34:42 +03:00
Calin Culianu
212eb1169b Implemented onNewBlockHeight
- We check the address unspent cache and do cache upkeep
- Various other nits and corner cases handled in this commit
2019-05-05 12:45:26 +03:00
Calin Culianu
b40dfbeb20 Added 'CallOnObject' facility 2019-05-05 02:49:56 +03:00
Calin Culianu
0da480f3c5 Got Controller and EXMgr to use the TimersByNameMixin
This mixin should be useful as we move forward
2019-05-05 01:03:23 +03:00
Calin Culianu
e3471d743d nit 2019-05-04 23:32:10 +03:00
Calin Culianu
a0195a9ae0 Made the round-trip shuffle.spec command work and verify utxos
We are like 50% there to milestone 1. Next, implement shuffle logic,
python test client, and tx building/verification.

Who am I kidding.. we are still pretty far away. :)
2019-05-04 22:53:36 +03:00
Calin Culianu
d87720d110 Hooked Controller into more things from exMgr, etc 2019-05-04 16:54:17 +03:00
Calin Culianu
cb5def88a0 hooked the controller in further to tcpserver .. more to do on that front 2019-05-04 09:37:42 +03:00
Calin Culianu
31b891c006 hooked controller object in to app startup 2019-05-04 06:00:01 +03:00
Calin Culianu
f36d7d00c9 added unconfirmed utxo separation in AddressUnspentEntry 2019-05-04 05:13:35 +03:00
Calin Culianu
631b8d3e80 fixed bug in operator< for BTC::Address and BTC::UTXO 2019-05-03 21:39:25 +03:00
Calin Culianu
113efda25a Added spec rpc command from clients to us.. still need to do listunspent 2019-05-03 17:28:51 +03:00
Calin Culianu
8399ee0052 Added BTC::UTXO with tests
Faster encapsulation of UTXOs .. string based.  Copy-on-write.

interops with bitcoin::COutPoint
2019-05-03 14:37:07 +03:00
Calin Culianu
3f49dce1db Added some small goodies
- Util::RunInThread to run a lambda in a thread
- App exit handling cleanup
- EXMgr uses an id for clients now
- EXMgr keeps track of lagging clients
- EXMgr more aggressive with reconnects if no valid exclients
- Nits and refactorings and misc. cleanup
2019-04-27 12:38:50 +03:00
Calin Culianu
e65bacd7fd Petit refactoring
- Made the stale timeout be 2 mins now
- Check clients runs every 1 minute
- Added 'Controller' ABC which EXMgr is one of and the TcpServer will be
another.
2019-04-26 06:23:49 +03:00