elements/ci
Andrew Poelstra c0fda0cc05 Merge ddc0a600b3 into merged_master (Bitcoin PR #18617)
For reasons I do not really grok, this PR changes the timing behavior of the
functional tests to reliably expose a deadlock in the claimpegin RPC that has
existed since the 0.17 rebase.

The mechanism is: in `claimpegin` in src/wallet/rpcwallet.cpp:5873, we call
`AcceptToMemoryPoolWorker`. This requires cs_main to be locked, which it is
not (contrast Core's `testmempoolaccept` RPC, which similarly calls
`AcceptToMemoryPoolWorker` from the RPC thread, and locks cs_main immediately
before).

We do *say* that it is locked, in the `LockAssertion` one the line above, but
this was added in ad3d496d78 during the 0.17
rebase (PR #620), apparently to shut up some linter on OSX, and as near as I
can tell it was never true.

Anyway, `AcceptToMemoryPoolWorker` calls through a couple layers which assume
cs_main is locked, to `AcceptSingleTransaction`, which locks m_pool.cs on line
src/validation.cpp:1101. It then calls `PreChecks`, which on line 784 calls
::ChainActive(), which _actually_ locks cs_main, though only briefly. This
line is the deadlock, because we locked m_pool.cs followed by cs_main...

...meanwhile, in src/net_processing.cpp, we lock cs_main at the top of the
`PeerLogicValidation::SendMessages` loop (src/net_processing.cpp:3628). In the
same loop, in the `feefilter` message processing, we call CTxMemPool::GetMinFee
(src/net_processing.cpp:4137), which locks m_pool.cs. Deadlock.

Anyway, that explains the change to locking behavior that I added to an
otherwise test-only PR.
2020-11-26 01:08:36 +00:00
..
lint Merge 22d11187ee into merged_master (Bitcoin PR #17398) 2020-11-26 01:07:41 +00:00
retry build: update retry to current version 2019-10-30 18:49:57 -04:00
test Merge ddc0a600b3 into merged_master (Bitcoin PR #18617) 2020-11-26 01:08:36 +00:00
README.md [tests] check v0.17.1 and v0.18.1 backwards compatibility 2020-02-11 21:46:21 +01:00
test_run_all.sh ci: Add environment files for all settings 2019-08-16 08:46:37 -04:00

ci scripts

This directory contains scripts for each build step in each build stage.

Currently three stages lint, extended_lint and test are defined. Each stage has its own lifecycle, similar to the Travis CI lifecycle. Every script in here is named and numbered according to which stage and lifecycle step it belongs to.

Running a stage locally

Be aware that the tests will be built and run in-place, so please run at your own risk. If the repository is not a fresh git clone, you might have to clean files from previous builds or test runs first.

The ci needs to perform various sysadmin tasks such as installing packages or writing to the user's home directory. While most of the actions are done inside a docker container, this is not possible for all. Thus, cache directories, such as the depends cache, previous release binaries, or ccache, are mounted as read-write into the docker container. While it should be fine to run the ci system locally on you development box, the ci scripts can generally be assumed to have received less review and testing compared to other parts of the codebase. If you want to keep the work tree clean, you might want to run the ci system in a virtual machine with a Linux operating system of your choice.

To allow for a wide range of tested environments, but also ensure reproducibility to some extent, the test stage requires docker to be installed. To install all requirements on Ubuntu, run

sudo apt install docker.io bash

To run the default test stage,

./ci/test_run_all.sh

To run the test stage with a specific configuration,

FILE_ENV="./ci/test/00_setup_env_arm.sh" ./ci/test_run_all.sh