In block processing and block downloading, HaveBlock is used to check if
the block data already exists. It was ok to just check for the
existence of the blockNode but we now we also need to check if the data
exists as the blockNode may be present for just the block header.
On flushes to the database, we check that the blockNodes we have for
the downloaded block headers are not flushed to the disk unless the
block data is stored as well for backwards compatibility.
With older btcd clients, they rely on the fact that the blockNode is
present to check if the block data is also present. Since we now
store blockNodes for just the block headers, this is no longer true.
Because of this, we don't flush the blockNodes if there's no
accompanying block data for it. This results in downloading and
verifying the headers again if the node were to restart but since the
header data is small and the verification is quick, it's not a big
downside.
As an optimization, flushToDB now skips opening a write transaction
entirely when every dirty node is header-only. This avoids a no-op
write transaction on every ProcessBlockHeader call during header sync.
maybeAcceptBlock unconditionally created a new blockNode, overwriting the
index entry. If maybeAcceptBlockHeader had already processed the header,
the pointer held by bestHeader's chainView became orphaned, breaking
bestHeader.Contains and downstream checks like IsValidHeader.
Check for an existing node first and upgrade its status to
statusDataStored rather than replacing it.
ProcessBlockHeader performs chain selection and context-free &
contextual validation for the given block header. The function allows
a header-first downloading of blocks even without checkpoints.
maybeAcceptHeader performs checks to accept block headers into the
header chain. This function allows for a true headers-first download
where we only accept block headers for downloading new blocks.
We add a chainview of bestHeaders so that we'll be able to keep track of
headers separately from the bestChain. This is needed as we're getting
headers for new block annoucements instead of invs.
This commit introduces the concept of `AlwaysActiveHeight` to the
deployment mechanism, allowing a deployment to be forced into the active
state if the next block's height meets or exceeds this threshold.
This is intended primarily to be used alongside the new Testnet4
deployment, as the past major soft forks are meant to be active from the
very first block height.
Add robust property-based tests for the assertNoTimeWarp function using
the rapid testing library. The tests verify the following scenarios:
- Basic property tests:
- Only retarget blocks (block height divisible by blocksPerRetarget) are checked
- Valid timestamps (within maxTimeWarp of previous block) pass validation
- Invalid timestamps (too early) fail with appropriate ErrTimewarpAttack
- Correct boundary behavior (exactly at maxTimeWarp limit)
- Invariant tests:
- Function never panics with valid inputs
- Non-retarget blocks always return nil regardless of timestamps
- Security tests:
- All retarget blocks are protected from timewarp attacks
- Non-retarget blocks are not affected by the timewarp check
ReconsiderBlock reconsiders the validity of the block for the passed
in blockhash. The behavior of the function mimics that of Bitcoin Core.
The invalid status of the block nodes are reset and if the chaintip that
is being reconsidered has more cumulative work, then we'll validate the
blocks and reorganize to it. If the cumulative work is lesser than the
current active chain tip, then nothing else will be done.
reorganizeChain() used to handle the following:
1: That the blocknodes being disconnected/connected indeed to connect
properly without errors.
2: Perform the actual disconnect/connect of the blocknodes.
The functionality of 1, the validation that the disconnects/connects can
happen without errors are now refactored out into
verifyReorganizationValidity.
This is an effort made so that ReconsiderBlock() can call
verifyReorganizationValidity and set the block status of the
reconsidered chain and return nil even when an error returns as it's ok
to get an error when reconsidering an invalid branch.
InvalidateBlock() invalidates a given block and marks all its
descendents as invalid as well. The active chain tip changes if the
invalidated block is part of the best chain.
The block generating functions here allow for a test to create mock
blocks. This is useful for testing invalidateblock and reconsiderblock
methods on blockchain that will be added in later commits.
createSpendTx is moved to testhelper so that the function can be used
for callers in package blockchain without introducing import cycles.
The test code for invalidateblock and reconsiderblock that are going to
be added in later commits make use of this code.
createCoinbaseTx's code is refactored out and placed in testhelper
package and is exported so that callers in package blockchain can reuse
the code without introducing import cycles. The test code for
invalidateblock and reconsiderblock that'll be added in later commits
make use of this code.
standardCoinbaseScript is moved to testhelper and is exported. This
allows test code in package blockchain to reuse the code without
introducing an import cycle. This code is used in the testing code
for invalidateblock and reconsiderblock that's added in the later
commits.
uniqueOpReturnScript is moved to testhelper and is exported so that the
code and be reused in package blockchain without introducing import
cycles. The test code for invalidateblock and reconsiderblock that are
gonna be added in later commits uses the functions.
The variables are moved to testhelper so that they can be reused in the
blockchain package without introducing an import cycle. The testing
code for invalidateblock and reconsiderblock that will be added in later
commits will be using these variables.
solveBlock is moved to testhelper and is exported. This is done so that
the code can be reused without introducing import cycles. The testing
code to be added in alter commits for invalidateblock and reconsider
block will use SolveBlock.
spendableOut and the functions related to it are is moved to package
testhelper and are exported. This is done to make the code reusable
without introducing an import cycle when the testing code for
invalidateblock and reconsiderblock are added in follow up commits.
Some of the functions in difficulty.go are not dependent on any external
functions and they are needed to introduce testing code for the
invalidateblock and reconsiderblock methods that are to be added on in
later commits. Having the workmath package let's us reuse the code and
avoid dependency cycles.
The existing functions that were exported already (HashToBig,
CompactToBig, BigToCompact, CalcWork) are still kept in difficulty.go
to avoid breaking external code that depends on those exported
functions.
Duplicate entries are currently possible in the following scenario:
1: Add entries to the mapslice.
2: 1st map is full. Move onto the 2nd map.
3: Delete any entry in the first map.
4: Attempt to add an entry in the 2nd map.
When attempting (4), the entry should just be overwritten but a
duplicate gets added.
Allowing the caller to fetch from either the database or the cache
resulted in inconsistencies if the cache were ever to be dirty.
Removing this option eliminates this problem.
The assumption in the previous code was incorrect in that we were
assuming that the chainLock is held throughout the entire chain reorg.
This is not the case since the chainLock is let go of during the
callback to the subscribers.
Because of this, we need to ensure that the utxo set is consistent on
each block disconnect. To achieve this, additional flushes are added
during block disconnects.
Also the utxocache is no longer avoided during block connects and when
we're checking for the validity of the block connects and disconnects as
we can just use the cache instead of trying to avoid it.
calculations
Since BlockHeightByHash only returns the heights for blocks that are in
the main chain, when a block that is stale gets pruned, this will cause
an error in the block height lookup and cause an error in block
processing.
Look up the node directly from the index and if the node isn't found,
just skip that node. For utxoCache.lastFlushHash, if that isn't found,
just force a flush.
For various b.sendNotifcation() callbacks, if a runtime panic happens,
we don't get any useful debugging information since the error that
happens first is the "unlock of unlocked mutex" error.
This is because we temporarily unlock the chainLock for callbacks and
then relock them. However, since the relocking code is executed after
the completion of the callback, if an error happens during that
callback, we never relock the chainLock.
Switching to an anonymous function and having the unlock code as a
defer will ensure that the lock always relocks.
existance/non-existance
New test instance BlockDisconnectExpectUTXO tests that a utxo
exists/doesn't exist after a specific block has been disconnected.
