The first fuzztest takes a Simplicity program and a transaction and
directly calls the Simplicity interpreter with some context cobbled
together from the transaction. It also tries messing with the budget
and computes AMRs to check that the AMR-check works, even though on
the blockchain AMRs are never used.
It also attempts mangling programs to directly fuzz the parser, type
inference and CMR checking.
THIS test, on the other hand, takes a transaction, looks for Simplicity
programs (or witnesses which look like Simplicity programs), computes
their CMRs to produce a correct corresponding scriptPubKey, creates
scriptchecks, and executes them. This should do an end-to-end coverage
of the whole Simplicity consensus logic, including all the new branches
in interpreter.cpp.
To produce seeds for this, I have a a local fuzz target which uses
rust-simplicity and rust-elements to produce programs, deep Taproot trees,
and transactions. I run this to get high coverage, then dump the
resulting complete transactions to disk, where they can be used as
seeds for this test.
This fuzz target takes its seeds in a simple and well-defined format: a
four-byte LE budget, then a transaction, Simplicity program and witness,
each prefixed by a four-byte LE length. The fuzz target extracts any
additional randomness it needs from the txid of the first input of the
transaction, since this data is not interpreted in any other way we
therefore won't confuse the fuzzer.
The reason for this design, rather than a more typical "just query the
fuzzer when you need stuff", is to make it possible to fairly easily
generate test vectors from sources other than this fuzz test. (For
example, I have an alternate target which uses Rust code to generate
well-formed Simplicity programs, which quickly gets high coverage at the
expense of being an unmaintainable mess.)
This commit includes a .c file with a small function to comute the AMR
of a program. This is needed to pass a correct AMR to the Simplicity
interpreter, to exercise all the AMR-checking paths. In practice this is
not really necessary; Elements passes NULL to disable these AMR checks.
4d0ac72f3a [fuzz] Add fuzzing harness to compare both implementations of ChaCha20 (stratospher)
65ef93203c [fuzz] Add D. J. Bernstein's implementation of ChaCha20 (stratospher)
Pull request description:
This PR compares Bitcoin Core's implementation of ChaCha20 with D. J. Bernstein's in order to find implementation discrepancies if any.
ACKs for top commit:
laanwj:
Code review ACK 4d0ac72f3a
Tree-SHA512: f826144b4db61b9cbdd7efaaca8fa9cbb899953065bc8a26820a566303b2ab6a17431e7c114635789f0a63fbe3b65cb0bf2ab85baf882803a5ee172af4881544
d8ee8f3cd3 refactor: Make CWalletTx sync state type-safe (Russell Yanofsky)
Pull request description:
Current `CWalletTx` state representation makes it possible to set inconsistent states that won't be handled correctly by wallet sync code or serialized & deserialized back into the same form.
For example, it is possible to call `setConflicted` without setting a conflicting block hash, or `setConfirmed` with no transaction index. And it's possible update individual `m_confirm` and `fInMempool` data fields without setting an overall consistent state that can be serialized and handled correctly.
Fix this without changing behavior by using `std::variant`, instead of an enum and collection of fields, to represent sync state, so state tracking code is safer and more legible.
This is a first step to fixing state tracking bugs https://github.com/bitcoin-core/bitcoin-devwiki/wiki/Wallet-Transaction-Conflict-Tracking, by adding an extra margin of safety that can prevent new bugs from being introduced as existing bugs are fixed.
ACKs for top commit:
laanwj:
re-ACK d8ee8f3cd3
jonatack:
Code review ACK d8ee8f3cd3
Tree-SHA512: b9f15e9d99dbdbdd3ef7a76764e11f66949f50e6227e284126f209e4cb106af6d55e9a9e8c7d4aa216ddc92c6d5acc6f4aa4746f209bbd77f03831b51a2841c3
faba1abe46 Sort file list after rename (MarcoFalke)
fa8f60e311 scripted-diff: Move minisketchwrapper to src/node (MarcoFalke)
Pull request description:
The newly added wrapper is currently in the node library, but not placed in the node directory. While it is possible to use the wrapper outside of a node context (for example in a utility), it seems unlikely. Either way, I think the wrapper should either be moved to the util lib+dir or the node lib+dir, not something in-between.
Also, fix incorrect comment `BITCOIN_DBWRAPPER_H`.
ACKs for top commit:
fanquake:
ACK faba1abe46. I saw the comment in #21515, however given there hasn't been any new activity there, I'm going to merge this now.
Tree-SHA512: fccc0cfd1fee661152a1378587b96795ffb7a7eceb6d2c27ea5401993fd8b9c0a92579fdba61203917ae6565269cb28d0973464fb6201dabf72a5143495d3e77
Current CWalletTx state representation makes it possible to set
inconsistent states that won't be handled correctly by wallet sync code
or serialized & deserialized back into the same form.
For example, it is possible to call setConflicted without setting a
conflicting block hash, or setConfirmed with no transaction index. And
it's possible update individual m_confirm and fInMempool data fields
without setting an overall consistent state that can be serialized and
handled correctly.
Fix this without changing behavior by using std::variant, instead of an
enum and collection of fields, to represent sync state, so state
tracking code is safer and more legible.
This is a first step to fixing state tracking bugs
https://github.com/bitcoin-core/bitcoin-devwiki/wiki/Wallet-Transaction-Conflict-Tracking,
by adding an extra margin of safety that can prevent new bugs from being
introduced as existing bugs are fixed.
29173d6c6c ubsan: add minisketch exceptions (Cory Fields)
54b5e1aeab Add thin Minisketch wrapper to pick best implementation (Pieter Wuille)
ee9dc71c1b Add basic minisketch tests (Pieter Wuille)
0659f12b13 Add minisketch dependency (Gleb Naumenko)
0eb7928ab8 Add MSVC build configuration for libminisketch (Pieter Wuille)
8bc166d5b1 build: add minisketch build file and include it (Cory Fields)
b2904ceb85 build: add configure checks for minisketch (Cory Fields)
b6487dc4ef Squashed 'src/minisketch/' content from commit 89629eb2c7 (fanquake)
Pull request description:
This takes over #21859, which has [recently switched](https://github.com/bitcoin/bitcoin/pull/21859#issuecomment-921899200) to my integration branch. A few more build issues came up (and have been fixed) since, and after discussing with sipa it was decided I would open a PR to shepherd any final changes through.
> This adds a `src/minisketch` subtree, taken from the master branch of https://github.com/sipa/minisketch, to prepare for Erlay implementation (see #21515). It gets configured for just supporting 32-bit fields (the only ones we're interested in in the context of Erlay), and some code on top is added:
> * A very basic unit test (just to make sure compilation & running works; actual correctness checking is done through minisketch's own tests).
> * A wrapper in `minisketchwrapper.{cpp,h}` that runs a benchmark to determine which field implementation to use.
Only changes since my last update to the branch in the previous PR have been rebasing on master and fixing an issue with a header in an introduced file.
ACKs for top commit:
naumenkogs:
ACK 29173d6c6c
Tree-SHA512: 1217d3228db1dd0de12c2919314e1c3626c18a416cf6291fec99d37e34fb6eec8e28d9e9fb935f8590273b8836cbadac313a15f05b4fd9f9d3024c8ce2c80d02
14cd7bf793 [test] call CheckPackage for package sanitization checks (glozow)
6876378365 MOVEONLY: move package unit tests to their own file (glozow)
c9b1439ca9 MOVEONLY: mempool checks to their own functions (glozow)
9e910d8152 scripted-diff: clean up MemPoolAccept aliases (glozow)
fd92b0c398 document workspace members (glozow)
3d3e4598b6 [validation] cache iterators to mempool conflicts (glozow)
36a8441912 [validation/rpc] cache + use vsize calculated in PreChecks (glozow)
8fa2936b34 [validation] re-introduce bool for whether a transaction is RBF (glozow)
cbb3598b5c [validation/refactor] store precomputed txdata in workspace (glozow)
0a79eaba72 [validation] case-based constructors for ATMPArgs (glozow)
Pull request description:
This contains the refactors and moves within #22674. There are no behavior changes, so it should be simpler to review.
ACKs for top commit:
ariard:
Code Review ACK 14cd7bf
jnewbery:
Code review ACK 14cd7bf793
laanwj:
Code review ACK 14cd7bf793, thanks for adding documentation and clarifying the code
t-bast:
Code Review ACK https://github.com/bitcoin/bitcoin/pull/23381/commits/14cd7bf793547fa5143acece564482271f5c30bc
Tree-SHA512: 580ed48b43713a3f9d81cd9b573ef6ac44efe5df2fc7b7b7036c232b52952b04bf5ea92940cf73739f4fbd54ecf980cef58032e8a2efe05229ad0b3c639de8a0
This addresses issues like the one in #12467, where some of our compiler flags
end up being dropped during the subconfigure of Univalue. Specifically, we're
still using the compiler-default c++ version rather than forcing c++17.
We can drop the need subconfigure completely in favor of a tighter build
integration, where the sources are listed separately from the build recipes,
so that they may be included directly by upstream projects. This is
similar to the way leveldb build integration works in Core.
Core benefits of this approach include:
- Better caching (for ex. ccache and autoconf)
- No need for a slow subconfigure
- Faster autoconf
- No more missing compile flags
- Compile only the objects needed
There are no benefits to Univalue itself that I can think of. These changes
should be a no-op there, and to downstreams as well until they take advantage
of the new sources.mk.
This also removes the option to use an external univalue to avoid similar ABI
issues with mystery binaries.
Co-authored-by: fanquake <fanquake@gmail.com>
Messy merge conflicts because this PR backported some ad-hoc stuff from
upstream while keeping a few things that upstream deleted. Hopefully
reviewing is easier than doing this in the first place, since ultimately
all I did was delete code from one side or another of the conflicts.
(Ok, I also changed some boost optional stuff to std::optional, and had
to patch up a test file for test framework changes.)
When reviewing the detailed crypto, bear in mind that taptweaks, like all
hashes are the kind of crypto that cannot be subtly wrong -- it will either
fail very hard or be correct. And we have independent implementations in
C++ and Python that cross-check each other, si it's unlikely to be the former.
