We skip the utxo count check at startup unless the new -C (--checkdb)
flag is set, in which case we do a very thorough check of all utxos and
the corresponding scripthash_unspent entries.
We set a dirty flag during block addition / removal and if it's set, we
abort the app immediately and ask the user to do a full resynch.
Since adding/removing blocks isn't 100% ACID, this is needed until we
figure out how to redo the data to make everything ACID.
Also in this commit -- attempted to see if I could make the header
reading faster from the db by using MultiGet. It turns out it doesn't
help much.
This allows for an app-global threadpool to be used for work that may
take a little while to complete. The intended code that will use this
is the rpc server which will submit work that takes some time to
complete to the ThreadPool, so that the server's thread doesn't block
for very long while servicing client requests.
This ensures that the server remains responsive even if 1 or 2 clients
are issuing costly requests.
We plan on using this for get_history, listunspent, and the
merkle-related functions, to name a few.
We deleted the non-working merge operator for scripthash_unspent.. and
we do a read-modify-write for each new utxo/spend per scripthash.
It's very slow though.
I may have to defer updating this table until the very end from the utxo
set or something. This is unacceptably bad...
we weren't adding the right txidx to the history. Good thing I also
added debug code to this commit to check history.
Fixed!
Also various other nits and tweaks. Added the /debug endpoint for
sending params for debugging to the server.
RocksDB is super fast. Keeping it in memory was a waste of memory. This
is much faster and reduced memory consumption significantly.
TODO: Maybe do the same thing for headers as well!
Also: We need to save txNum to DB. We're not saving it now resulting in
correctness errors on subsequent reloads of same db.
Also in this commit various nits and subtle bugs prevented.
Refactored the 'Stats' mechanism to return a generic QVariant rather
than QVariantMap, which is more flexible.
Misc. other nits.
Controller will manage BitcoinD and perhaps synching to bitcoind. It can
"see" all the servers as well as bitcoind.
Also refactored things a little bit and added some more startup sanity
checks.
It works! I still haven't figured out how to provoke the dreaded
"Workdepth exceeded" error .. but we pretty much drop the connection
anyway on anything too unexpected from bitcoind.
When the bitcoind manager class is written it will back off and try
again if too many disconnects happen.
For now though, the code we have is fast and works well against
bitcoind.
I restricted the possible underlying types it can contain down to
qint64, QString, and nullptr_t. This should make it more type safe since
we don't want it to inadvertently end up holding lists or maps
(previously it was a straight up QVariant).
It has a conversion to QVariant so it should work nicely.
On Windows it still prints to console, however (without timestamps).
Note that syslog mode doesn't produce any debug output even if -d is
specified (at least on macOS) -- probably the sysadmin needs to
enable debug logging.
- Util::RunInThread to run a lambda in a thread
- App exit handling cleanup
- EXMgr uses an id for clients now
- EXMgr keeps track of lagging clients
- EXMgr more aggressive with reconnects if no valid exclients
- Nits and refactorings and misc. cleanup
- Already supports listening on as many sockets as you wish
- Each listening server has its own thread
- Will implement later: SSL server
Also refactored code a bit, created the generic "Controller" class, made
EXMgr and SrvMgr inherit from it.
ToDo: Figure out workflow for connections and clients, model client
sessions, etc.
- Made the stale timeout be 2 mins now
- Check clients runs every 1 minute
- Added 'Controller' ABC which EXMgr is one of and the TcpServer will be
another.
