This is to support writing raw ints directly to db keys / db values as a
space saving measure. QDataStream, while platform neutral, adds its own
marker/size bytes to the serialization which we do not need always.
Our DB format is now endian and architecture-specific, but that's ok.
Users can resynch if they switch machine architectures.
The PreProcessedBlock is intended to be a somewhat munged/mogrified form
of a CBlock which is somewhat ready for a db insert.
This data will get passed onto the Controller thread which will set
everything up to commit the SH history, header, etc to the db.
Did some initial testing and it's pretty fast and also.. I think it's
correct (as in I'm collating the data correctly).
TODO: Some more testing.
We now save the "chain" variable to the meta table. We also panic if
the headers we are reading from bitcoind are behind what we have (todo:
handle this elegantly if there actually was a reorg?)
Templates are fun. Decided to reduce boilerplate by making task
creation inside the Controller class be done by a template member
function which acts like a task factory. It surrounds the construction
of the task with some boilerplate (namely, inserting the task in the
tasks map, connecting an optional errored signal, and starting the
task).
Also in this commit various nits and subtle bugs prevented.
Refactored the 'Stats' mechanism to return a generic QVariant rather
than QVariantMap, which is more flexible.
Misc. other nits.
We now query more information initially when synching, which can be
useful.
We also deal with initial block download by retrying every minute
and other miscellany
When reading data from bitcoind, we can assume the hex data it gives us
is good and there is no need to validate on a per-character basis.
Thus, we use ParseHexFast to shave off cpu cycles (it's like 6x faster
than Qt's fromHex()).
This, along with the Util::ToHexFast() (which is 60% faster than
QByteArray::toHex()), will be used in performance critical code.
This is more complex than what we had before -- but is more concurrent
and is more production-ready -- it can deal with failures better and now
we also poll bitcoind for new headers.
Still probably can be optimized somehwat -- but getting there!
Set maxcur to 16, spinning up 3 bitcoind threads.
Seems to have brought header synch down from 170s to 71s on my mainnet
node here locally. Not bad!
Also fixed a bug where selecting bitcoind nodes was failing if >2 nodes.
And added some test code in Controller to use the scheme.
It's very asynchronous. Needs some tuning long term to see how it deals
with load and perhaps throttle the requests if we know bitcoind is under
load.
But for right now, for testing, it should work ok.
Also in this commit small nits, plus bugfix to RPC::Message factory
methods (I forgot to populate some instance properties when constructing
the objects).
Additionally, we were registering the qMetaType wrong for RPC::Message
and RPC::Message::Id (this has been fixed).
Playing with lambdas. Detects when bitcoind goes down. Initial code --
can put hooks in the detection for eg. stopping block processing or
starting it, etc.
