elements/src
fanquake 0a55bcd299
Merge bitcoin/bitcoin#27981: Fix potential network stalling bug
3388e523a1 Rework receive buffer pushback (Pieter Wuille)

Pull request description:

  See https://github.com/ElementsProject/elements/issues/1233. There, it has been observed that if both sides of a P2P connection have a significant amount of data to send, a stall can occur, where both try to drain their own send queue before trying to receive. The same issue seems to apply to the current Bitcoin Core codebase, though I don't know whether it's a frequent issue for us.

  The core issue is that whenever our optimistic send fails to fully send a message, we do subsequently not even select() for receiving; if it then turns out that sending is not possible either, no progress is made at all. To address this, the solution used in this PR is to still select() for both sending and receiving when an optimistic send fails, but skip receiving if sending succeeded, and (still) doesn't fully drain the send queue.

  This is a significant reduction in how aggressive the "receive pushback" mechanism is, because now it will only mildly push back while sending progress is made; if the other side stops receiving entirely, the pushback disappears. I don't think that's a serious problem though:
  * We still have a pushback mechanism at the application buffer level (when the application receive buffer overflows, receiving is paused until messages in the buffer get processed; waiting on our own net_processing thread, not on the remote party).
  * There are cases where the existing mechanism is too aggressive; e.g. when the send queue is non-empty, but tiny, and can be sent with a single send() call. In that case, I think we'd prefer to still receive within the same processing loop of the network thread.

ACKs for top commit:
  ajtowns:
    ACK 3388e523a1
  naumenkogs:
    ACK 3388e523a1
  mzumsande:
    Tested ACK 3388e523a1

Tree-SHA512: 28960feb3cd2ff3dfb39622510da62472612f88165ea98fc9fb844bfcb8fa3ed3633f83e7bd72bdbbbd37993ef10181b2e1b34836ebb8f0d83fd1c558921ec17
2023-08-17 13:15:42 +01:00
..
bench Merge bitcoin/bitcoin#28244: Break up script/standard.{h/cpp} 2023-08-17 12:54:16 +01:00
common Rename script/standard.{cpp/h} to script/solver.{cpp/h} 2023-08-14 17:39:49 -04:00
compat Sort includes in compat.h 2023-08-08 17:50:41 +02:00
config
consensus Use int32_t type for most transaction size/weight values 2023-06-12 19:47:19 +01:00
crc32c Update crc32c subtree to latest upstream master 2022-08-13 13:55:38 +01:00
crypto refactor: add missing headers for BIP324 ciphersuite 2023-08-15 07:30:48 +05:30
index Merge bitcoin/bitcoin#28186: kernel: Prune leveldb headers 2023-08-07 22:31:46 +02:00
init init: remove config option names from translated -loglevel strings 2023-06-15 10:27:56 -06:00
interfaces Move CTxDestination to its own file 2023-08-14 17:38:27 -04:00
ipc refactor: Move system from util to common library 2023-05-20 12:08:13 +02:00
kernel Merge bitcoin/bitcoin#28244: Break up script/standard.{h/cpp} 2023-08-17 12:54:16 +01:00
leveldb Update leveldb-subtree subtree to latest upstream 2022-09-29 16:43:30 +01:00
logging Use steady clock for logging timer 2023-01-31 18:48:50 +01:00
minisketch Update minisketch subtree to latest upstream 2022-10-23 15:03:04 +01:00
node Merge bitcoin/bitcoin#28244: Break up script/standard.{h/cpp} 2023-08-17 12:54:16 +01:00
policy Rename script/standard.{cpp/h} to script/solver.{cpp/h} 2023-08-14 17:39:49 -04:00
primitives scripted-diff: Bump copyright headers 2022-12-24 23:49:50 +00:00
qt Rename script/standard.{cpp/h} to script/solver.{cpp/h} 2023-08-14 17:39:49 -04:00
rpc Merge bitcoin/bitcoin#28244: Break up script/standard.{h/cpp} 2023-08-17 12:54:16 +01:00
script Rename script/standard.{cpp/h} to script/solver.{cpp/h} 2023-08-14 17:39:49 -04:00
secp256k1 Update secp256k1 subtree to latest upstream master 2023-07-18 15:25:05 +01:00
support Don't derive secure_allocator from std::allocator 2023-07-24 22:33:40 -07:00
test Merge bitcoin/bitcoin#28244: Break up script/standard.{h/cpp} 2023-08-17 12:54:16 +01:00
univalue Remove unused raw-pointer read helper from univalue 2023-07-27 14:24:52 +02:00
util Clean up things that include script/standard.h 2023-08-14 17:38:27 -04:00
wallet Rename script/standard.{cpp/h} to script/solver.{cpp/h} 2023-08-14 17:39:49 -04:00
zmq p2p, refactor: return std::optional<CNetAddr> in LookupHost 2023-05-26 13:41:07 -03:00
.bear-tidy-config ci: filter all subtrees from tidy output 2023-06-29 11:59:31 +01:00
.clang-format
.clang-tidy tidy: Integrate bicoin-tidy clang-tidy plugin 2023-08-03 17:52:24 +01:00
addrdb.cpp script: appease spelling linter 2023-06-29 16:14:07 -06:00
addrdb.h fix includes for touched header files (iwyu) 2023-05-29 13:26:31 +02:00
addresstype.cpp Rename script/standard.{cpp/h} to script/solver.{cpp/h} 2023-08-14 17:39:49 -04:00
addresstype.h Move CTxDestination to its own file 2023-08-14 17:38:27 -04:00
addrman.cpp addrman: change asserts to Assumes 2023-05-26 15:47:55 -07:00
addrman.h doc: update Select function description 2023-05-24 12:03:18 -07:00
addrman_impl.h addrman: change asserts to Assumes 2023-05-26 15:47:55 -07:00
arith_uint256.cpp scripted-diff: Bump copyright headers 2022-12-24 23:49:50 +00:00
arith_uint256.h Fix clang-tidy readability-const-return-type violations 2023-02-01 11:33:35 +01:00
attributes.h Introduce platform-agnostic ALWAYS_INLINE macro 2023-05-04 20:57:51 +01:00
banman.cpp refactor: Move system from util to common library 2023-05-20 12:08:13 +02:00
banman.h refactor: Move fs.* to util/fs.* 2023-03-23 12:55:18 +01:00
base58.cpp scripted-diff: Bump copyright headers 2022-12-24 23:49:50 +00:00
base58.h scripted-diff: Bump copyright headers 2022-12-24 23:49:50 +00:00
bech32.cpp scripted-diff: Bump copyright headers 2022-12-24 23:49:50 +00:00
bech32.h Make Bech32 LocateErrors return error list rather than using out-arg 2021-12-06 14:17:41 +13:00
bip324.cpp refactor: add missing headers for BIP324 ciphersuite 2023-08-15 07:30:48 +05:30
bip324.h refactor: add missing headers for BIP324 ciphersuite 2023-08-15 07:30:48 +05:30
bitcoin-chainstate.cpp kernel: Remove StartShutdown calls from validation code 2023-07-11 12:30:56 -04:00
bitcoin-cli-res.rc
bitcoin-cli.cpp refactor: Move system from util to common library 2023-05-20 12:08:13 +02:00
bitcoin-tx-res.rc
bitcoin-tx.cpp kernel: Remove Univalue from kernel library 2023-07-25 17:40:07 +02:00
bitcoin-util-res.rc
bitcoin-util.cpp refactor: Move system from util to common library 2023-05-20 12:08:13 +02:00
bitcoin-wallet-res.rc
bitcoin-wallet.cpp refactor: Move system from util to common library 2023-05-20 12:08:13 +02:00
bitcoind-res.rc
bitcoind.cpp Remove the syscall sandbox 2023-06-16 10:38:19 +01:00
blockencodings.cpp refactor: Fix logging.h includes 2023-08-05 10:42:56 +02:00
blockencodings.h script: remove out-of-date snprintf TODO 2023-02-15 14:42:28 -08:00
blockfilter.cpp refactor: use Span for SipHash::Write 2023-07-17 13:27:46 +02:00
blockfilter.h scripted-diff: Bump copyright headers 2022-12-24 23:49:50 +00:00
chain.cpp scripted-diff: Bump copyright headers 2022-12-24 23:49:50 +00:00
chain.h Documentation improvements for assumeutxo 2023-07-24 16:23:38 -04:00
chainparams.cpp test: Throw error when -signetchallenge is non-hex 2023-05-25 19:24:05 +02:00
chainparams.h test: Throw error when -signetchallenge is non-hex 2023-05-25 19:24:05 +02:00
chainparamsbase.cpp refactor: Use ChainType enum exhaustively 2023-05-10 10:39:58 +02:00
chainparamsbase.h refactor: Use ChainType enum exhaustively 2023-05-10 10:39:58 +02:00
chainparamsseeds.h p2p: update hardcoded mainnet seeds for 25.x 2023-04-20 06:08:22 -07:00
checkqueue.h Remove the syscall sandbox 2023-06-16 10:38:19 +01:00
clientversion.cpp scripted-diff: Bump copyright headers 2022-12-24 23:49:50 +00:00
clientversion.h scripted-diff: Bump copyright headers 2022-12-24 23:49:50 +00:00
coins.cpp Use PoolAllocator for CCoinsMap 2023-03-23 19:38:38 +01:00
coins.h Use PoolAllocator for CCoinsMap 2023-03-23 19:38:38 +01:00
compressor.cpp Clean up things that include script/standard.h 2023-08-14 17:38:27 -04:00
compressor.h scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
core_io.h kernel: Remove Univalue from kernel library 2023-07-25 17:40:07 +02:00
core_memusage.h
core_read.cpp kernel: Remove Univalue from kernel library 2023-07-25 17:40:07 +02:00
core_write.cpp Rename script/standard.{cpp/h} to script/solver.{cpp/h} 2023-08-14 17:39:49 -04:00
cuckoocache.h clang-tidy: Fix modernize-use-default-member-init in headers 2023-01-31 11:50:10 +00:00
dbwrapper.cpp Merge bitcoin/bitcoin#28186: kernel: Prune leveldb headers 2023-08-07 22:31:46 +02:00
dbwrapper.h refactor: Correct dbwrapper key naming 2023-08-05 10:45:19 +02:00
deploymentinfo.cpp Decouple RegTestChainParams from ArgsManager 2023-03-15 16:42:42 +01:00
deploymentinfo.h Decouple RegTestChainParams from ArgsManager 2023-03-15 16:42:42 +01:00
deploymentstatus.cpp scripted-diff: Bump copyright headers 2022-12-24 23:49:50 +00:00
deploymentstatus.h scripted-diff: Bump copyright headers 2022-12-24 23:49:50 +00:00
dummywallet.cpp move-only: Extract common/args and common/config.cpp from util/system 2023-04-19 10:48:30 +02:00
external_signer.cpp Fix clang-tidy performance-unnecessary-copy-initialization warnings 2023-05-09 18:48:52 +02:00
external_signer.h refactor: Move system from util to common library 2023-05-20 12:08:13 +02:00
flatfile.cpp refactor: Extract util/fs_helpers from util/system 2023-03-23 12:52:00 +01:00
flatfile.h Merge bitcoin/bitcoin#27254: refactor: Extract util/fs from util/system 2023-04-03 14:41:22 +01:00
hash.cpp scripted-diff: Bump copyright headers 2022-12-24 23:49:50 +00:00
hash.h doc: Remove comments that just repeat what the code does 2023-07-12 10:01:51 +02:00
headerssync.cpp [headerssync] Make m_commit_offset protected 2022-12-12 21:06:04 +00:00
headerssync.h [headerssync] Make m_commit_offset protected 2022-12-12 21:06:04 +00:00
httprpc.cpp scripted-diff: Use UniValue::find_value method 2023-05-09 18:47:14 +02:00
httprpc.h scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
httpserver.cpp Merge bitcoin/bitcoin#27949: http: update libevent workaround to correct version 2023-06-28 12:20:25 +01:00
httpserver.h scripted-diff: Bump copyright headers 2022-12-24 23:49:50 +00:00
i2p.cpp refactor: Move system from util to common library 2023-05-20 12:08:13 +02:00
i2p.h refactor: Move fs.* to util/fs.* 2023-03-23 12:55:18 +01:00
indirectmap.h
init.cpp Merge bitcoin/bitcoin#28244: Break up script/standard.{h/cpp} 2023-08-17 12:54:16 +01:00
init.h refactor: index, decouple 'Init' from 'Start' 2023-07-10 10:47:17 -03:00
key.cpp Add ElligatorSwift key creation and ECDH logic 2023-06-23 14:22:33 -04:00
key.h Add ElligatorSwift key creation and ECDH logic 2023-06-23 14:22:33 -04:00
key_io.cpp Rename script/standard.{cpp/h} to script/solver.{cpp/h} 2023-08-14 17:39:49 -04:00
key_io.h Clean up things that include script/standard.h 2023-08-14 17:38:27 -04:00
logging.cpp refactor: rename BCLog::BLOCKSTORE to BLOCKSTORAGE 2023-06-15 10:27:56 -06:00
logging.h refactor: rename BCLog::BLOCKSTORE to BLOCKSTORAGE 2023-06-15 10:27:56 -06:00
Makefile.am Merge bitcoin/bitcoin#28244: Break up script/standard.{h/cpp} 2023-08-17 12:54:16 +01:00
Makefile.bench.include Merge bitcoin/bitcoin#28008: BIP324 ciphersuite 2023-08-10 11:58:59 +02:00
Makefile.crc32c.include build: Create .la library for crc32c 2022-04-26 16:25:38 -04:00
Makefile.leveldb.include Use more specific path when including memenv.h header 2022-06-23 15:33:01 +02:00
Makefile.minisketch.include build: Build minisketch test in make check, not in make 2022-07-21 10:42:09 +01:00
Makefile.qt.include Build: Log when test -f fails in Makefile 2023-02-03 22:33:14 +01:00
Makefile.qt_locale.include qt: Periodic translation updates from Transifex 2023-02-27 13:53:29 +00:00
Makefile.qttest.include build: extract $(BOOST_CPPFLAGS) from $(BITCOIN_INCLUDES) 2022-09-13 17:15:17 +01:00
Makefile.test.include Merge bitcoin/bitcoin#28008: BIP324 ciphersuite 2023-08-10 11:58:59 +02:00
Makefile.test_fuzz.include [fuzz] Move ConsumeNetAddr to fuzz/util/net.h 2022-11-17 14:52:45 +00:00
Makefile.test_util.include Merge bitcoin/bitcoin#27425: test: move remaining rand code from util/setup_common to util/random 2023-07-19 10:26:11 +01:00
Makefile.univalue.include
mapport.cpp Remove the syscall sandbox 2023-06-16 10:38:19 +01:00
mapport.h Remove configure-time setting of DEFAULT_UPNP 2023-01-28 15:24:13 +00:00
memusage.h Calculate memory usage correctly for unordered_maps that use PoolAllocator 2023-03-23 19:38:38 +01:00
merkleblock.cpp
merkleblock.h scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
net.cpp Merge bitcoin/bitcoin#27981: Fix potential network stalling bug 2023-08-17 13:15:42 +01:00
net.h Merge bitcoin/bitcoin#27981: Fix potential network stalling bug 2023-08-17 13:15:42 +01:00
net_permissions.cpp Merge bitcoin/bitcoin#26261: p2p: cleanup LookupIntern, Lookup and LookupHost 2023-05-30 11:39:59 -04:00
net_permissions.h scripted-diff: Bump copyright headers 2022-12-24 23:49:50 +00:00
net_processing.cpp Merge bitcoin/bitcoin#27675: p2p: Drop m_recently_announced_invs bloom filter 2023-08-17 10:52:06 +01:00
net_processing.h net processing: clamp -blockreconstructionextratxn to uint32_t bounds 2023-07-25 21:51:20 +01:00
net_types.cpp scripted-diff: Bump copyright headers 2022-12-24 23:49:50 +00:00
net_types.h scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
netaddress.cpp net, refactor: pass reference for peer address in GetReachabilityFrom 2023-06-05 11:02:47 -04:00
netaddress.h refactor: use Span for SipHash::Write 2023-07-17 13:27:46 +02:00
netbase.cpp Remove now-unnecessary poll, fcntl includes from net(base).cpp 2023-06-28 16:35:45 -05:00
netbase.h p2p, refactor: return std::optional<CNetAddr> in LookupHost 2023-05-26 13:41:07 -03:00
netgroup.cpp scripted-diff: Bump copyright headers 2022-12-24 23:49:50 +00:00
netgroup.h [netgroupman] Remove NetGroupManager::GetAsmap() 2022-04-20 14:35:53 +01:00
netmessagemaker.h
noui.cpp scripted-diff: Bump copyright headers 2022-12-24 23:49:50 +00:00
noui.h
outputtype.cpp Clean up things that include script/standard.h 2023-08-14 17:38:27 -04:00
outputtype.h Move CTxDestination to its own file 2023-08-14 17:38:27 -04:00
pow.cpp scripted-diff: Bump copyright headers 2022-12-24 23:49:50 +00:00
pow.h scripted-diff: Bump copyright headers 2022-12-24 23:49:50 +00:00
prevector.h util: implement prevector's move ctor & move assignment 2023-03-26 15:49:52 +02:00
protocol.cpp refactor: Remove duplicate allNetMessageTypesVec 2023-07-11 19:26:52 +02:00
protocol.h doc: Remove unused NO_BLOOM_VERSION constant 2023-05-15 10:18:58 +02:00
psbt.cpp Move Taproot{SpendData/Builder} to signingprovider.{h/cpp} 2023-08-14 17:38:27 -04:00
psbt.h Merge bitcoin/bitcoin#25939: rpc: In utxoupdatepsbt also look for the tx in the txindex 2023-04-21 14:06:12 -04:00
pubkey.cpp Add ElligatorSwift key creation and ECDH logic 2023-06-23 14:22:33 -04:00
pubkey.h Add BIP324Cipher, encapsulating key agreement, derivation, and stream/AEAD ciphers 2023-07-26 17:09:23 -04:00
random.cpp Allow FastRandomContext::randbytes for all byte types 2023-06-30 12:09:44 +02:00
random.h Allow FastRandomContext::randbytes for all byte types 2023-06-30 12:09:44 +02:00
randomenv.cpp refactor: don't avoid sys/types.h on when building for Windows 2023-04-03 14:44:48 +01:00
randomenv.h scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
rest.cpp rest: bugfix, fix crash error when calling /deploymentinfo 2023-06-12 10:24:14 -03:00
rest.h Handle query string when parsing data format 2022-03-10 12:01:53 +01:00
reverse_iterator.h doc: remove usages of C++11 2023-01-12 13:42:44 +00:00
scheduler.cpp Remove the syscall sandbox 2023-06-16 10:38:19 +01:00
scheduler.h scripted-diff: Bump copyright headers 2022-12-24 23:49:50 +00:00
serialize.h refactor: use "if constexpr" in std::vector's Unserialize() 2023-08-03 10:35:22 +02:00
shutdown.cpp kernel: Add fatalError method to notifications 2023-06-28 09:52:33 +02:00
shutdown.h kernel: Add fatalError method to notifications 2023-06-28 09:52:33 +02:00
signet.cpp Clean up things that include script/standard.h 2023-08-14 17:38:27 -04:00
signet.h scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
span.h refactor: Use reinterpret_cast where appropriate 2023-07-24 15:32:35 +02:00
streams.cpp streams: Teach AutoFile how to XOR 2023-07-19 18:12:42 +02:00
streams.h streams: Teach AutoFile how to XOR 2023-07-19 18:12:42 +02:00
sync.cpp scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
sync.h doc: Export threadsafety.h from sync.h 2023-01-25 09:33:26 +01:00
threadsafety.h
timedata.cpp move-only: Extract common/args and common/config.cpp from util/system 2023-04-19 10:48:30 +02:00
timedata.h scripted-diff: Bump copyright headers 2022-12-24 23:49:50 +00:00
tinyformat.h clang-tidy: Fix modernize-use-default-member-init in headers 2023-01-31 11:50:10 +00:00
torcontrol.cpp Replace LocaleIndependentAtoi with ToIntegral 2023-08-08 16:04:22 +02:00
torcontrol.h iwyu on torcontrol 2023-08-08 16:03:40 +02:00
txdb.cpp kernel: Pass interrupt reference to chainman 2023-06-28 09:52:27 +02:00
txdb.h kernel: Pass interrupt reference to chainman 2023-06-28 09:52:27 +02:00
txmempool.cpp Merge bitcoin/bitcoin#27675: p2p: Drop m_recently_announced_invs bloom filter 2023-08-17 10:52:06 +01:00
txmempool.h Merge bitcoin/bitcoin#27675: p2p: Drop m_recently_announced_invs bloom filter 2023-08-17 10:52:06 +01:00
txorphanage.cpp scripted-diff: Following the C++ Standard rules for identifiers with _. 2023-06-20 10:23:08 +02:00
txorphanage.h scripted-diff: Following the C++ Standard rules for identifiers with _. 2023-06-20 10:23:08 +02:00
txrequest.cpp refactor: use Span for SipHash::Write 2023-07-17 13:27:46 +02:00
txrequest.h
uint256.cpp refactor: modernize the implementation of uint256.* 2022-12-10 14:34:44 -06:00
uint256.h Merge bitcoin/bitcoin#27927: util: Allow std::byte and char Span serialization 2023-06-28 15:12:12 -04:00
undo.h
validation.cpp Merge bitcoin/bitcoin#27675: p2p: Drop m_recently_announced_invs bloom filter 2023-08-17 10:52:06 +01:00
validation.h Merge bitcoin/bitcoin#27675: p2p: Drop m_recently_announced_invs bloom filter 2023-08-17 10:52:06 +01:00
validationinterface.cpp Fix clang-tidy readability-const-return-type violations 2023-02-01 11:33:35 +01:00
validationinterface.h refactor: add kernel/cs_main.* 2023-01-05 09:05:14 +00:00
version.h doc: Remove unused NO_BLOOM_VERSION constant 2023-05-15 10:18:58 +02:00
versionbits.cpp refactor: use braced init for integer constants instead of c style casts 2023-01-03 19:31:29 -06:00
versionbits.h scripted-diff: Bump copyright headers 2022-12-24 23:49:50 +00:00
walletinitinterface.h scripted-diff: Bump copyright headers 2022-12-24 23:49:50 +00:00
warnings.cpp refactor: Move system from util to common library 2023-05-20 12:08:13 +02:00
warnings.h scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00