We already use this in the blockfilter code, so not sure we need to maintain two
different ways of testing for the same functionality. Consolidate on testing
for __SIZEOF_INT128__, which we already use, is supported by the compilers we
care about, and is also used by libsecp256k1.
Messy merge conflicts because this PR backported some ad-hoc stuff from
upstream while keeping a few things that upstream deleted. Hopefully
reviewing is easier than doing this in the first place, since ultimately
all I did was delete code from one side or another of the conflicts.
(Ok, I also changed some boost optional stuff to std::optional, and had
to patch up a test file for test framework changes.)
When reviewing the detailed crypto, bear in mind that taptweaks, like all
hashes are the kind of crypto that cannot be subtly wrong -- it will either
fail very hard or be correct. And we have independent implementations in
C++ and Python that cross-check each other, si it's unlikely to be the former.
This is part of the "UTXO Set Statistics" project in Core, which aims
to make things like "checking coin supply" faster. This stuff, and
related things in coinstats.cpp, have always been broken for Elements
in the sense that we consider only explicit outputs and we ignore
assets (so everything just gets added together to get a meaningless
total). It probably wouldn't be too hard to restrict this to only
consider policyAsset, but it's out of scope for a rebase IMO.
Also, I think this situation is fine .. I don't understand the motivation
for this or why Core is merging this when they refuse to merge an address
index .. but I guess we'll see if there are users who care about this data
and who care about it being meaningful on Elements.
Also, apologies for the big diff -- there were some mechanical changes
to deal with CT amounts, but most of the changes related to the difference
in how fees are accounted for. While I'm not thrilled with this PR, its
functional test is really good! So I think what I eventually came up with
is internally consistent.
BIP324 mentions K1 is used for the associated data and K2 is used for
the payload. The code does the opposite. This is not a security problem
but will be a problem across implementations based on the HKDF key
derivations.
Division of MuHash objects are very expensive and multiplication relatively cheap. The whole idea of introducing and tracking numerator and denominators seperately as a representation of the internal state was so that divisions would be rare. So using divison in the Remove method did not make any sense and was just a silly mistake which is corrected here.
9815332d51 test: Change MuHash Python implementation to match cpp version again (Fabian Jahr)
01297fb3ca fuzz: Add MuHash consistency fuzz test (Fabian Jahr)
b111410914 test: Add MuHash3072 fuzz test (Fabian Jahr)
c122527385 bench: Add Muhash benchmarks (Fabian Jahr)
7b1242229d test: Add MuHash3072 unit tests (Fabian Jahr)
adc708c98d crypto: Add MuHash3072 implementation (Fabian Jahr)
0b4d290bf5 crypto: Add Num3072 implementation (Fabian Jahr)
589f958662 build: Check for 128 bit integer support (Fabian Jahr)
Pull request description:
This is the first split of #18000 which implements the Muhash algorithm and uses it to calculate the UTXO set hash in `gettxoutsetinfo`.
ACKs for top commit:
laanwj:
Code review ACK 9815332d51
Tree-SHA512: 4bc090738f0e3d80b74bdd8122e24a8ce80121120fd37c7e4335a73e7ba4fcd7643f2a2d559e2eebf54b8e3a3bd5f12cfb27ba61ded135fda210a07a233eae45
7be6ff6187 net: recognize TORv3/I2P/CJDNS networks (Vasil Dimov)
e0d73573a3 net: CNetAddr: add support to (un)serialize as ADDRv2 (Vasil Dimov)
fe42411b4b test: move HasReason so it can be reused (Vasil Dimov)
d2bb681f96 util: move HasPrefix() so it can be reused (Vasil Dimov)
Pull request description:
(chopped off from #19031 to ease review)
Add an optional support to serialize/unserialize `CNetAddr` in ADDRv2 format (BIP155). The new serialization is engaged by ORing a flag into the stream version.
So far this is only used in tests to ensure the new code works as expected.
ACKs for top commit:
Sjors:
re-tACK 7be6ff6187
sipa:
re-utACK 7be6ff6187
eriknylund:
ACK 7be6ff6187 I built the PR on macOS Catalina 10.15.6, ran both tests and functional tests. I've reviewed the code and think the changes look good and according to BIP155. I verified that the added Base32 encoding test looks as proposed and working. I've run a node for a week only with Onion addresses `-onlynet=onion` without issues and I can connect to other peer reviewers running TorV3 on their nodes and I can connect both of my test nodes to each other.
jonatack:
re-ACK 7be6ff6187 per `git diff b9c46e0 7be6ff6`, debug build, ran/running bitcoind with this change and observed the log and `-netinfo` peer connections while connected as a tor v2 service to both tor v2 peers and also five tor v3 peers.
hebasto:
ACK 7be6ff6187, tested on Linux Mint 20 (x86_64): on top of this pull and #19031 I'm able to connect to onion v3 addresses, and jonatack is able to connect to my created onion v3 address.
Tree-SHA512: dc621411ac4393993aa3ccad10991717ec5f9f2643cae46a24a89802df0a33d6042994fc8ff2f0f397a3dbcd1c0e58fe4724305a2f9eb64d9342c3bdf784d9be
