Commit graph

1490 commits

Author SHA1 Message Date
James Dorfman
438cb545b7 Merge 2364d17a31 into merged_master (Bitcoin PR bitcoin/bitcoin#25480) 2024-09-19 15:22:38 +00:00
James Dorfman
88a98fd4b3 Merge 2fe27029f5 into merged_master (Bitcoin PR bitcoin/bitcoin#25404) 2024-09-17 18:36:02 +00:00
glozow
2bf721e76a
Merge bitcoin/bitcoin#30661: fuzz: Test headers pre-sync through p2p
a97f43d63a fuzz: Add harness for p2p headers sync (marcofleon)
a0eaa4749f Add FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION in PoW check (marcofleon)
a3f6f5acd8 build: Automatically define FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION for fuzz builds (marcofleon)
0c02d4b2bd net_processing: Make MAX_HEADERS_RESULTS a PeerManager option (marcofleon)

Pull request description:

  This PR reopens https://github.com/bitcoin/bitcoin/pull/28043. It's a regression fuzz test for https://github.com/bitcoin/bitcoin/pull/26355 and [a couple bugs](https://github.com/bitcoin/bitcoin/pull/25717/commits/ed6cddd98e32263fc116a4380af6d66da20da990) that were addressed in https://github.com/bitcoin/bitcoin/pull/25717. This should help us move forward with the [removal of mainnet checkpoints](https://github.com/bitcoin/bitcoin/pull/25725).

  It seems like the main concern in https://github.com/bitcoin/bitcoin/pull/28043 was the global mock function for proof of work. This PR aims to be an improvement by replacing the previous approach with a fuzz build configured using `FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION`. This ensures that the simplified test code will never be in a release binary. If we agree this is the way to go, there are some other places (for future targets) where this method could be used.

  In this target, PoW isn't being tested, so the goal is to bypass the check and let the fuzzer do its thing. In the other harnesses where PoW is actually being fuzzed, `CheckProofOfWork` is now `CheckProofOfWorkImpl`. So, the only change to that function is in the name.

  More about `FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION` can be found at https://llvm.org/docs/LibFuzzer.html#fuzzer-friendly-build-mode and https://github.com/AFLplusplus/AFLplusplus/blob/stable/docs/fuzzing_in_depth.md#d-modifying-the-target.

ACKs for top commit:
  naumenkogs:
    ACK a97f43d63a
  dergoegge:
    reACK a97f43d63a
  instagibbs:
    tested ACK a97f43d63a
  brunoerg:
    ACK a97f43d63a

Tree-SHA512: 60b0bc6aadd8ca4c39db9cbba2da2debaaf68afcb6a8dd75c1ce48ca9e3996948fda8020930b6771a424e0f7c41b0b1068db4aa7dbe517f8fc152f1f712058ad
2024-09-16 13:59:22 -04:00
James Dorfman
fb17f731bd Merge 50a3921c96 into merged_master (Bitcoin PR bitcoin/bitcoin#25144) 2024-09-16 06:44:24 +00:00
James Dorfman
1c386ceb80 Merge e05564d706 into merged_master (Bitcoin PR bitcoin/bitcoin#25446) 2024-09-16 05:06:26 +00:00
Sergi Delgado Segura
07f4cebe57 refactor: move m_is_inbound out of CNodeState
`m_is_inbound` cannot be changed throughout the life of a `Peer`. However, we
are currently storing it in `CNodeState`, which requires locking `cs_main` in
order to access it. This can be moved to the outside scope and only require
`m_peer_mutex`.

This is a refactor in preparation for Erlay reworks.
2024-09-12 11:20:44 -04:00
James Dorfman
fe8d041f9f Merge 489b587669 into merged_master (Bitcoin PR bitcoin/bitcoin#25215) 2024-09-10 20:19:35 +00:00
James Dorfman
115b62e5ec Merge ede9089096 into merged_master (Bitcoin PR bitcoin/bitcoin#25156) 2024-09-10 19:08:58 +00:00
James Dorfman
c472e94b84 Merge 8f3ab9a1b1 into merged_master (Bitcoin PR bitcoin/bitcoin#24931) 2024-09-09 21:13:17 +00:00
James Dorfman
caa3ef8844 Merge 2ab4a80480 into merged_master (Bitcoin PR bitcoin/bitcoin#25254)
This commit moved DEFAULT_MIN_RELAY_TX_FEE to a different file.
Please confirm that the new value is equal to the old one.
2024-09-09 18:21:18 +00:00
James Dorfman
033f17b1df Merge 00ce8543f1 into merged_master (Bitcoin PR bitcoin/bitcoin#24171)
Please review the changes to src/net_processing.cpp carefully, I'm not sure
if I adapted new IBD logic to work properly for elements.
2024-09-09 16:33:14 +00:00
James Dorfman
1aff5fa9e9 Merge 5f65afff9c into merged_master (Bitcoin PR bitcoin/bitcoin#24178) 2024-09-06 18:21:45 +00:00
Ava Chow
d4b5553849
Merge bitcoin/bitcoin#30742: kernel: Use spans instead of vectors for passing block headers to validation functions
a2955f0979 validation: Use span for ImportBlocks paths (TheCharlatan)
20515ea3f5 validation: Use span for CalculateClaimedHeadersWork (TheCharlatan)
52575e96e7 validation: Use span for ProcessNewBlockHeaders (TheCharlatan)

Pull request description:

  Makes it friendlier for potential future users of the kernel library if they do not store the headers in a std::vector, but can guarantee contiguous memory.

  Take this opportunity to also change the argument of ImportBlocks previously taking a `std::vector` to a `std::span`.

ACKs for top commit:
  stickies-v:
    re-ACK a2955f0979 - no changes except further walking the ~file~ path of modernizing variable names.
  maflcko:
    ACK a2955f0979 🕑
  achow101:
    ACK a2955f0979
  danielabrozzoni:
    ACK a2955f0979

Tree-SHA512: 8b07f4ad26e270b65600d1968cd78847b85caca5bfbb83fd9860389f26656b1d9a40b85e0990339f50403d18cedcd2456990054f3b8b0bedce943e50222d2709
2024-09-03 15:40:40 -04:00
marcofleon
0c02d4b2bd net_processing: Make MAX_HEADERS_RESULTS a PeerManager option 2024-09-02 15:42:44 +01:00
merge-script
d4cc0c6845
Merge bitcoin/bitcoin#30750: scripted-diff: LogPrint -> LogDebug
fa09cb41f5 refactor: Remove unused LogPrint (MarcoFalke)
3333415890 scripted-diff: LogPrint -> LogDebug (MarcoFalke)

Pull request description:

  `LogPrint` has many issues:

  * It seems to indicate that something is being "printed", however config options such as `-printtoconsole` actually control what and where something is logged.
  * It does not mention the log severity (debug).
  * It is a deprecated alias for `LogDebug`, according to the dev notes.
  * It wastes review cycles, because reviewers sometimes point out that it is deprecated.
  * It makes the code inconsistent, when both are used, possibly even in lines right next to each other (like in `InitHTTPServer`)

  Fix all issues by removing the deprecated alias.

  I checked all conflicting pull requests and at the time of writing there are no conflicts, except in pull requests that are marked as draft, are yet unreviewed, or are blocked on feedback for other reasons. So I think it is fine to do now.

ACKs for top commit:
  stickies-v:
    ACK fa09cb41f5
  danielabrozzoni:
    utACK fa09cb41f5
  TheCharlatan:
    ACK fa09cb41f5

Tree-SHA512: 14270f4cfa3906025a0b994cbb5b2e3c8c2427c0beb19c717a505a2ccbfb1fd1ecf2fd03f6c52d22cde69a8d057e50d2207119fab2c2bc8228db3f10d4288d0f
2024-09-02 11:59:56 +01:00
TheCharlatan
20515ea3f5
validation: Use span for CalculateClaimedHeadersWork
Makes it friendlier for potential future users of the kernel library if
they do not store the headers in a std::vector, but can guarantee
contiguous memory.
2024-08-30 10:17:26 +02:00
TheCharlatan
52575e96e7
validation: Use span for ProcessNewBlockHeaders
Makes it friendlier for potential future users of the kernel library if
they do not store the headers in a std::vector, but can guarantee
contiguous memory.
2024-08-30 10:17:09 +02:00
MarcoFalke
3333415890
scripted-diff: LogPrint -> LogDebug
-BEGIN VERIFY SCRIPT-
 sed -i 's/\<LogPrint\>/LogDebug/g' $( git grep -l '\<LogPrint\>'  -- ./contrib/ ./src/ ./test/ ':(exclude)src/logging.h' )
-END VERIFY SCRIPT-
2024-08-29 13:49:57 +02:00
Hodlinator
50bc017040
refactor: Hand-replace some ParseHex -> ""_hex
The following scripted-diff commit will replace ParseHex("...") with "..."_hex_u8, but this replacement will not work in cases where vectors are needed instead of arrays, and is not ideal in cases where std::byte is accepted.

For example, it is currently necessary to use _hex_v_u8 when calling CScript operator<< because that operator does not currently support std::array or std::byte.

Conversely, it is incorrect to use _hex_v instead of _hex in net_processing.cpp for the MakeAndPushMessage argument, because if the argument is a std::vector it is considered variable-length and serialized with a size prefix, but if the argument is a std::array or Span is it considered fixed length and serialized without a prefix.

By the same logic, it is also safe to change the NUMS_H constant in pubkey.cpp from a std::vector to std::array because it is never serialized.
2024-08-28 19:11:59 +02:00
James Dorfman
9cb0f04edb Merge aac99faa66 into merged_master (Bitcoin PR bitcoin/bitcoin#25175) 2024-08-13 20:39:38 +00:00
James Dorfman
66bc45020f Merge 4d0c00dffd into merged_master (Bitcoin PR bitcoin/bitcoin#25168) 2024-08-13 20:10:21 +00:00
James Dorfman
0e02561a73 Merge fdb82a30be into merged_master (Bitcoin PR bitcoin/bitcoin#25147) 2024-08-13 05:07:06 +00:00
James Dorfman
cef04ed2a5 Merge 986bae8e72 into merged_master (Bitcoin PR bitcoin/bitcoin#22778) 2024-08-13 04:41:52 +00:00
James Dorfman
57c149aa06 Merge 0be1dc1f56 into merged_master (Bitcoin PR bitcoin/bitcoin#24062) 2024-08-12 04:44:23 +00:00
James Dorfman
09739a82e2 Merge aa3200d896 into merged_master (Bitcoin PR bitcoin/bitcoin#25109) 2024-08-12 03:41:59 +00:00
James Dorfman
ab65a98a04 Merge dc0ee57373 into merged_master (Bitcoin PR bitcoin/bitcoin#20799) 2024-08-12 03:28:06 +00:00
James Dorfman
5854eae590 Merge 25dd4d8513 into merged_master (Bitcoin PR bitcoin/bitcoin#24595) 2024-08-12 02:29:42 +00:00
Ava Chow
c2d15d993e
Merge bitcoin/bitcoin#29519: p2p: For assumeutxo, download snapshot chain before background chain
49d569cb1f p2p: For assumeutxo, download snapshot chain before background chain (Martin Zumsande)
7a885518d5 p2p: Restrict downloading of blocks for snapshot chain (Martin Zumsande)

Pull request description:

  After loading a snapshot, `pindexLastCommonBlock` is usually already set to some block for existing peers. That means we'd continue syncing the background chain from those peers instead of prioritising the snapshot chain, which defeats the purpose of doing assumeutxo in the first place. Only existing peers are affected by this bug.

ACKs for top commit:
  fjahr:
    re-ACK 49d569cb1f
  achow101:
    ACK 49d569cb1f
  Sjors:
    tACK 49d569cb1f

Tree-SHA512: 0eaebe1c29a8510d5ced57e14c09b128ccb34b491692815291df68bf12e2a15b52b1e7bf8d9f34808904e7f7bc20f70b0ad0f7e14df93bbdf456bd12cc02a5d2
2024-08-09 17:42:39 -04:00
James Dorfman
7effbfd139 Merge a2a8e919ee into merged_master (Bitcoin PR bitcoin/bitcoin#24925) 2024-08-09 21:08:52 +00:00
James Dorfman
56ea06cc37 Merge 5d53cf3878 into merged_master (Bitcoin PR bitcoin/bitcoin#24543) 2024-08-08 19:37:59 +00:00
Martin Zumsande
49d569cb1f p2p: For assumeutxo, download snapshot chain before background chain
After loading a snapshot, pindexLastCommonBlock is usually already set
to some block for existing peers. That means we'd continue syncing the
background chain from those peers instead of prioritising the snapshot
chain, which defeats the purpose of doing assumeutxo in the first place.
Only existing peers are affected by this bug.
2024-08-06 16:04:37 -04:00
Martin Zumsande
7a885518d5 p2p: Restrict downloading of blocks for snapshot chain
If the best chain of the peer doesn't include the snapshot
block, it is futile to download blocks from this chain,
because we couldn't reorg to it. We'd also crash
trying to reorg because this scenario is not handled.
2024-08-06 16:04:37 -04:00
James Dorfman
e7759695ba Merge f0a834e2f1 into merged_master (Bitcoin PR bitcoin/bitcoin#18642) 2024-08-06 18:37:18 +00:00
stickies-v
2925bd537c
refactor: use c++20 std::views::reverse instead of reverse_iterator.h
Use std::ranges::views::reverse instead of the implementation in
reverse_iterator.h, and remove it as it is no longer used.
2024-08-06 00:23:38 +01:00
James Dorfman
f86857195f Merge 43bb106613 into merged_master (Bitcoin PR bitcoin/bitcoin#24213) 2024-08-01 20:20:30 +00:00
James Dorfman
d4aa6eba2f Merge dbdc83ae01 into merged_master (Bitcoin PR bitcoin/bitcoin#24909) 2024-08-01 18:03:20 +00:00
dergoegge
a90ab4aec9 scripted-diff: Rename lazily initialized bloom filters
-BEGIN VERIFY SCRIPT-
sed -i 's/m_recent_confirmed_transactions/m_lazy_recent_confirmed_transactions/g' $(git grep -l 'm_recent_confirmed_transactions')
sed -i 's/m_recent_rejects_reconsiderable/m_lazy_recent_rejects_reconsiderable/g' $(git grep -l 'm_recent_rejects_reconsiderable')
sed -i 's/m_recent_rejects/m_lazy_recent_rejects/g' $(git grep -l 'm_recent_rejects')
-END VERIFY SCRIPT-
2024-07-31 13:23:46 +01:00
dergoegge
82de1bc478 [net processing] Lazily initialize m_recent_confirmed_transactions 2024-07-31 13:09:55 +01:00
dergoegge
fa0c87f19c [net processing] Lazily initialize m_recent_rejects_reconsiderable 2024-07-31 13:09:44 +01:00
dergoegge
662e8db2d3 [net processing] Lazily initialize m_recent_rejects 2024-07-31 13:08:20 +01:00
glozow
7c29e556c5 m_tx_download_mutex followups
- add AssertLockNotHeld(m_tx_download_mutex) in net_processing
- move doc about m_tx_download_mutex and mempool mutex to ActiveTipChange
2024-07-25 11:01:22 +01:00
glozow
e543c657da release m_tx_download_mutex before MakeAndPushMessage GETDATA 2024-07-25 11:01:22 +01:00
glozow
bce5f37c7b [refactor] change ActiveTipChange to use CBlockIndex ref instead of ptr 2024-07-25 11:01:22 +01:00
glozow
6f49548670 [refactor] combine block vtx loops in BlockConnected
Now that m_txrequest and m_recent_confirmed_transactions are guarded by
the same mutex, there is no benefit to processing them separately.
Instead, just loop through pblock->vtx once.
2024-07-24 10:38:34 +01:00
glozow
61745c7451 lock m_recent_confirmed_transactions using m_tx_download_mutex 2024-07-16 10:21:41 +01:00
glozow
723ea0f9a5 remove obsoleted hashRecentRejectsChainTip
This also means AlreadyHaveTx no longer needs cs_main held.
2024-07-16 10:21:41 +01:00
glozow
18a4355250 update recent_rejects filters on ActiveTipChange
Resetting m_recent_rejects once per block is more efficient than
comparing hashRecentRejectsChainTip with the chain tip every time we
call AlreadyHaveTx. We keep hashRecentRejectsChainTip for now to assert
that updates happen correctly; it is removed in the next commit.
2024-07-16 10:21:41 +01:00
glozow
3eb1307df0 guard TxRequest and rejection caches with new mutex
We need to synchronize between various tx download structures.
TxRequest does not inherently need cs_main for synchronization, and it's
not appropriate to lock all of the tx download logic under cs_main.
2024-07-16 10:01:24 +01:00
glozow
35dddbccf1
Merge bitcoin/bitcoin#30394: net: fix race condition in self-connect detection
16bd283b3a Reapply "test: p2p: check that connecting to ourself leads to disconnect" (Sebastian Falbesoner)
0dbcd4c148 net: prevent sending messages in `NetEventsInterface::InitializeNode` (Sebastian Falbesoner)
66673f1c13 net: fix race condition in self-connect detection (Sebastian Falbesoner)

Pull request description:

  This PR fixes a recently discovered race condition in the self-connect detection (see #30362 and #30368).

  Initiating an outbound network connection currently involves the following steps after the socket connection is established (see [`CConnman::OpenNetworkConnection`](bd5d1688b4/src/net.cpp (L2923-L2930)) method):
  1. set up node state
  2. queue VERSION message (both steps 1 and 2 happen in [`InitializeNode`](bd5d1688b4/src/net_processing.cpp (L1662-L1683)))
  3. add new node to vector `m_nodes`

  If we connect to ourself, it can happen that the sent VERSION message (step 2) is received and processed locally *before* the node object is added to the connection manager's `m_nodes` vector (step 3). In this case, the self-connect remains undiscovered, as the detection doesn't find the outbound peer in `m_nodes` yet (see `CConnman::CheckIncomingNonce`).

  Fix this by swapping the order of 2. and 3., by taking the `PushNodeVersion` call out of `InitializeNode` and doing that in the `SendMessages` method instead, which is only called for `CNode` instances in `m_nodes`.

  The temporarily reverted test introduced in #30362 is readded. Fixes #30368.

  Thanks go to vasild, mzumsande and dergoegge for suggestions on how to fix this (see https://github.com/bitcoin/bitcoin/issues/30368#issuecomment-2200625017 ff. and https://github.com/bitcoin/bitcoin/pull/30394#discussion_r1668290789).

ACKs for top commit:
  naiyoma:
    tested ACK [https://github.com/bitcoin/bitcoin/pull/30394/commits/16bd283b3ad05daa41259a062aee0fc05b463fa6](https://github.com/bitcoin/bitcoin/pull/30394/commits/16bd283b3ad05daa41259a062aee0fc05b463fa6),  built and tested locally,  test passes successfully.
  mzumsande:
    ACK 16bd283b3a
  tdb3:
    ACK 16bd283b3a
  glozow:
    ACK 16bd283b3a
  dergoegge:
    ACK 16bd283b3a

Tree-SHA512: 5b8aced6cda8deb38d4cd3fe4980b8af505d37ffa0925afaa734c5d81efe9d490dc48a42e1d0d45dd2961c0e1172a3d5b6582ae9a2d642f2592a17fbdc184445
2024-07-16 09:40:53 +01:00
Ava Chow
394651ff10
Merge bitcoin/bitcoin#29996: Assumeutxo: bugfix on loadtxoutset with a divergent chain + test
5b7f70ba26 test: loadtxoutset in divergent chain with less work (Alfonso Roman Zubeldia)
d35efe1efc p2p: Start downloading historical blocks from common ancestor (Martin Zumsande)

Pull request description:

  This PR adds a test to cover the scenario of loading an assumeutxo snapshot when the current chain tip is not an ancestor of the snapshot block but has less work.

  During the review process, a bug was discovered where blocks between the last common ancestor and the background tip were not being requested if the background tip was not an ancestor of the snapshot block. mzumsande suggested a fix (65343ec49a6b73c4197dfc38e1c2f433b0a3838a) to start downloading historical blocks from the last common ancestor to address this issue. This fix has been incorporated into the PR with a slight modification.

  Related to https://github.com/bitcoin/bitcoin/issues/28648

ACKs for top commit:
  fjahr:
    tACK 5b7f70ba26
  achow101:
    ACK 5b7f70ba26
  mzumsande:
    Code Review ACK 5b7f70ba26

Tree-SHA512: f8957349686a6a1292165ea9e0fd8c912d21466072632a10f8ef9d852a5f430bc6b2a531e6884a4dbf2e3adb28b3d512b25919e78f5804a67320ef54c3b1aaf6
2024-07-10 15:18:33 -04:00