7f122a4188 fuzz: non-addrman fuzz tests: override-able check ratio (Vasil Dimov)
3bd83e273d fuzz: addrman fuzz tests: override-able check ratio (Vasil Dimov)
46b0fe7829 test: non-addrman unit tests: override-able check ratio (Vasil Dimov)
81e4d54d3a test: addrman unit tests: override-able check ratio (Vasil Dimov)
6dff6214be bench: put addrman check ratio in a variable (Vasil Dimov)
6f7c7567c5 fuzz: parse the command line arguments in fuzz tests (Vasil Dimov)
92a0f7e58d test: parse the command line arguments in unit tests (Vasil Dimov)
Pull request description:
Previously command line arguments passed to unit and fuzz tests would be ignored by the tests themselves. They would be used by the boost test framework (e.g. `--run_test="addrman_tests/*"`) or by the fuzzer (e.g. `-runs=1`). However both provide ways to pass down the extra arguments to the test itself. Use that, parse the arguments and make them available to the tests via `gArgs`.
This makes the tests more flexible as they can be run with any bitcoind config option specified on the command line.
When creating `AddrMan` objects in tests, use `-checkaddrman=` (if provided) instead of hardcoding the check ratio in many different places. See https://github.com/bitcoin/bitcoin/pull/20233#issuecomment-889813074 for further motivation for this.
ACKs for top commit:
mzumsande:
re-ACK 7f122a4188
josibake:
reACK 7f122a4188
Tree-SHA512: 3a05e61e4d70a0569bb67594bcce3aad8fdef63cdcc54e2823a3bc9f18679571985004412b6c332a210f67849bab32d8467b4115fbff8f5fac9834982e60dcf3
5574e6ed52 refactor: replace RecursiveMutex `m_callbacks_mutex` with Mutex (Sebastian Falbesoner)
3aa258109e scripted-diff: rename `m_cs_callbacks_pending` -> `m_callbacks_mutex` (Sebastian Falbesoner)
Pull request description:
This PR is related to #19303 and gets rid of the RecursiveMutex `m_cs_callbacks_pending`. All of the critical sections (6 in total) only directly access the guarded elements, i.e. it is not possible that within one section another one is called, and we can use a regular Mutex:
807169e10b/src/scheduler.cpp (L138-L145)807169e10b/src/scheduler.cpp (L152-L160)807169e10b/src/scheduler.cpp (L169-L172)807169e10b/src/scheduler.cpp (L184-L187)807169e10b/src/scheduler.cpp (L197-L199)807169e10b/src/scheduler.cpp (L203-L206)
Also, it is renamed to adapt to the (unwritten) naming convention to use the `_mutex` suffix rather than the `cs_` prefix.
ACKs for top commit:
hebasto:
ACK 5574e6ed52, I have reviewed the code and it looks OK, I agree it can be merged.
w0xlt:
crACK 5574e6e
Tree-SHA512: ba4b151d956582f4c7183a1d51702b269158fc5e2902c51e6a242aaeb1c72cfcdf398f9ffa42e3072f5aba21a8c493086a5fe7c450c271322da69bd54c37ed1f
30927cb530 refactor: replace RecursiveMutex `m_subver_mutex` with Mutex (Sebastian Falbesoner)
0639aba42a scripted-diff: rename `cs_SubVer` -> `m_subver_mutex` (Sebastian Falbesoner)
Pull request description:
This PR is related to #19303 and gets rid of the RecursiveMutex `cs_SubVer`. Both of the critical sections only directly access the guarded variable, i.e. it is not possible that within one section another one is called, and we can use a regular Mutex.
ACKs for top commit:
hebasto:
ACK 30927cb530, I have reviewed the code and it looks OK, I agree it can be merged.
Tree-SHA512: 2faead792ea0b2f79f9f7fe99acde5cf2bfcd2f15c51fbb6cb1099d4f81276001a492f7d46a5139055f4366c2d58a36a8ba19f21d56df20e0ed93af3141dbe11
fa99e108e7 Fix implicit-integer-sign-change in arith_uint256 (MarcoFalke)
Pull request description:
This refactor doesn't change behaviour, but clarifies that the numbers being dealt with aren't supposed to be negative. This helps when reading the code and allows to remove a sanitizer suppression for the whole file.
ACKs for top commit:
PastaPastaPasta:
utACK fa99e108e7
shaavan:
ACK fa99e108e7
Tree-SHA512: f227e2fd22021e39f0445ec041f4a299d13477c23cef0fc06c53fb3313cbe550cec329336224a7e8775d9045b8009423052b394e83d42a1e40772085dfcdd471
fac22fd36b log: Remove GetAdjustedTime from IBD header progress estimation (MarcoFalke)
Pull request description:
This is a "refactor" that shouldn't change behaviour, because the two times are most likely equal. A minimum of 5 outbound peers are needed to adjust the time. And if the time is adjusted, it will be by at most 70 minutes (`DEFAULT_MAX_TIME_ADJUSTMENT`). Thus, the progress estimate should differ by at most 7 blocks.
ACKs for top commit:
laanwj:
Code review ACK fac22fd36b
vincenzopalazzo:
ACK fac22fd36b
Tree-SHA512: bf9f5eef66db0110dd268cf6dbfab64b9c11ba776924f5b386ceae3f2d005272cceb87ebcc96e0c8b854c051514854a2a5af39ae43bad008fac685b5aafaabd0
fa7238300c fuzz: Limit fuzzed time to years 2000-2100 (MarcoFalke)
Pull request description:
It doesn't make sense to fuzz times in the past, as Bitcoin Core will refuse to start in the past.
Fix that and also remove a sanitizer suppression, which would be hit in net_processing in `ProcessMessage`:
```cpp
if (addr.nTime <= 100000000 || addr.nTime > nNow + 10 * 60)
addr.nTime = nNow - 5 * 24 * 60 * 60; // <-- Here
```
This changes the format of fuzz inputs. Previously a time value was (de)serialized as 40 bytes, now it is 32 bytes.
ACKs for top commit:
mzumsande:
Code Review ACK fa7238300c
Tree-SHA512: ca6e7233beec2d9ef9fd481d8f1331942a4d2c8fe518b857629bebcc53a4f42ae123b994cf5d359384a0a8022098ff5a9c146600bc2593c6d88734e25bc240ad
In each of the critical sections, only the the guarded variable is
accessed, without any chance that within one section another one is
called. Hence, we can use an ordinary Mutex instead of RecursiveMutex.
In each of the critical sections, only the the guarded variables are
accessed, without any chance that within one section another one is
called. Hence, we can use an ordinary Mutex instead of RecursiveMutex.
aa8a65e4a8 test: use MiniWallet for mempool_accept.py (Sebastian Falbesoner)
b24f6c6855 test: MiniWallet: support default `from_node` for creating txs (Sebastian Falbesoner)
f30041c914 test: create txs with current `nVersion` (2) by default (Sebastian Falbesoner)
2f79786822 test: refactor: add constant for sequence number `SEQUENCE_FINAL` (Sebastian Falbesoner)
Pull request description:
This PR enables one more of the non-wallet functional tests (mempool_accept.py) to be run even with the Bitcoin Core wallet disabled by using the MiniWallet instead, as proposed in #20078.
It also includes some other minor changes that came up while working on the replacement:
* [commit 1/4] replace magic number 0xffffffff for a tx's nSequence with a new constant `SEQUENCE_FINAL`
* [commit 2/4] create `CTransaction` instances with the current nVersion=2 by default, in order to use BIP68 for tests
* [commit 3/4] support default `from_node` parameter for creating txs (this is a stripped down version of PR #24025)
ACKs for top commit:
MarcoFalke:
re-ACK aa8a65e4a8📊
Tree-SHA512: 34cd085ea4147ad5bd3f3321c84204064ceb95f382664c7fe29062c1bbc79d9d9465c5e46d35e11c416f2f3cd46030c90a09b518c829c73ae40d060be5e4c9cb
c62d763fc3 Necessary improvements to make configure work without libevent installed (Perlover)
091ccc38c2 The evhttp_connection_get_peer function from libevent changes the type of the second parameter. Fixing the problem. (Perlover)
Pull request description:
The second parameter of evhttp_connection_get_peer in libevent already has type as `const char **`
The compilation of bitcoind with the fresh libevent occurs errors
Details: https://github.com/bitcoin/bitcoin/issues/23606
ACKs for top commit:
laanwj:
Code review ACK c62d763fc3
luke-jr:
tACK c62d763fc3
Tree-SHA512: d1c8062d90bd0d55c582dae2c3a7e5ee1b6c7ca872bf4aa7fe6f45a52ac4a8f59464215759d961f8efde0efbeeade31b08daf9387d7d50d7622baa1c06992d83
b5c9bb5cb9 util: Restore GetIntArg saturating behavior (James O'Beirne)
Pull request description:
The new locale-independent atoi64 method introduced in #20452 parses large integer values higher than maximum representable value as 0 instead of the maximum value, which breaks backwards compatibility. This commit restores compatibility and adds test coverage for this case in terms of the related GetIntArg and strtoll functions.
Specifically, command line or bitcoin.conf integer values greater than `9223372036854775807` (`2**63-1`) used to be parsed as `9223372036854775807` before #20452. Then #20452 caused them to be parsed as `0`. And after this PR they will be parsed as `9223372036854775807` again.
This change is a stripped-down alternative version of #23841 by jamesob
ACKs for top commit:
jamesob:
Github ACK b5c9bb5cb9
vincenzopalazzo:
ACK b5c9bb5cb9
MarcoFalke:
review ACK b5c9bb5cb9🌘
Tree-SHA512: 4e8abdbabf3cf4713cf5a7c5169539159f359ab4109a4e7e644cc2e9b2b0c3c532fad9f6b772daf015e1c5340ce59280cd9a41f2730afda6099cbf636b7d23ae
3c4f5d2a20 scripted-diff: Rename functions to drop mention of boost (Hennadii Stepanov)
Pull request description:
Nothing specific to Boost is in `GUIUtil::qstringToBoostPath` and `GUIUtil::boostPathToQString` functions.
Related to bitcoin/bitcoin#20744.
ACKs for top commit:
laanwj:
Concept and code review ACK 3c4f5d2a20
promag:
Code review ACK 3c4f5d2a20👋
Tree-SHA512: 0c8790979783a067811f7699b4ce4c204f6e0818d5f32986ec24b2c71583b4496d7a0e0c0361dd77c7e641a75d983fee35cd51ef722bbca9a5f13194efb3b4c0
a229451590 build: Point Guix to the current top of the "version-1.4.0" branch (Hennadii Stepanov)
Pull request description:
On master (c561f2f06e) the commit in Guix repo from bitcoin/bitcoin#23778 seems unavailable:
```
$ git checkout fa17abf1af09570708daa28dd40ffbc932ebe25c
fatal: reference is not a tree: fa17abf1af09570708daa28dd40ffbc932ebe25c
```
This PR points Guix to the current top of the "version-1.4.0" branch.
Fixes#24040.
ACKs for top commit:
MarcoFalke:
Approach ACK a229451590
fanquake:
ACK a229451590 - from what I've seen on the mailing list there shouldn't be any more force pushing.
Tree-SHA512: c58f846fb0afd51b5c2dd33034e9d593aec5d5b49e9f2232af70ae1224da8408ad4e05aa314609350d92a6400e354a816b988226e3572198c3f839ab33913164
ce95fb36af Remove cs_main lock annotation from ChainstateManager.m_blockman (Ryan Ofsky)
Pull request description:
`BlockManager` is a large data structure, and `cs_main` is not required to take its address or access every part of it. Individual `BlockManager` fields and methods which do require `cs_main` like `m_block_index` and `LookupBlockIndex` are already annotated separately, and these other annotations describe locking requirements more accurately and do a better job enforcing thread safety.
Since `cs_main` is not needed to access the address of the m_block object, this commit drops `cs_main` LOCK calls which were added pointlessly to satisfy this annotation in the past.
Code changes were made by dongcarl, I just wrote the commit description
ACKs for top commit:
MarcoFalke:
cr ACK ce95fb36af
dongcarl:
crACK ce95fb36af
jonatack:
ACK ce95fb36af per `git range-diff db1f04f 5a1c413 ce95fb3` change since last push is rebase and dropping braces on the member initialization
Tree-SHA512: b18a6ebcc70bea750485f04d4feb7bb28450fea2176e513be9cc242e9f63b24254c5659e74eb6d6045c706a3aaeb94688937b25b7ca7653f8aa3cf8c18845d5a
9d3e95d77c [bugfix] prevent UnicodeDecodeError errors when opening log file in feature_init.py (sogoagain)
Pull request description:
Should fix#23989
To fix a bug, I modified `feature_init.py` to open the log file as a byte stream when opening it.
thank you.
ACKs for top commit:
MarcoFalke:
review ACK 9d3e95d77c
Tree-SHA512: 6e3e57cac5f4865b3894ee4e9fcd9eb2690e824af20e34b4595722bd7043b0c3fe417cc1bfcff25fbab95c66418d3fce13434bd63d0244875a867d08853a5644
The new locale-independent atoi64 method introduced in #20452 parses
large integer values higher than maximum representable value as 0
instead of the maximum value, which breaks backwards compatibility.
This commit restores compatibility and adds test coverage for this case
in terms of the related GetIntArg and strtoll functions.
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
If no `from_node` parameter is passed explicitely to the
`create_self_transfer` method, the test node passed in the course
of creating the MiniWallet instance is used. This seems to
be the main use-case in most of the current functional
tests, i.e. in many instances the calls can be shortened.
Make it possible to override from the command line (without recompiling)
the addrman check ratio in non-addrman fuzz tests (connman and
deserialize) instead of hardcoding it to 0:
```
FUZZ=connman ./src/test/fuzz/fuzz --checkaddrman=5
```
Make it possible to override from the command line (without recompiling)
the addrman check ratio in addrman fuzz tests instead of hardcoding it
to 0:
```
FUZZ=addrman ./src/test/fuzz/fuzz --checkaddrman=5
```
Make it possible to override from the command line (without recompiling)
the addrman check ratio in the common `TestingSetup::m_node::addrman`
(used by all unit tests) instead of hardcoding it to 0:
```
test_bitcoin --run_test="transaction_tests/tx_valid" -- -checkaddrman=1
```
In addrman unit tests, make it possible to override the check ratio from
the command line, without recompiling:
```
test_bitcoin --run_test="addrman_tests/*" -- -checkaddrman=1
```
Also, make the arguments of the constructor of `AddrManTest` the
same as the arguments of `AddrMan`.
Retrieve the command line arguments from the fuzzer and save them for
later retrieval by `BasicTestingSetup` so that we gain extra flexibility
of passing any config options on the test command line, e.g.:
```
FUZZ=addrman ./src/test/fuzz/fuzz --checkaddrman=5
```
A fuzz test should call `MakeNoLogFileContext<>()` in its initialize
function in order to invoke the constructor of `BasicTestingSetup`,
which sets `gArgs`.
Retrieve the command line arguments from boost and pass them to
`BasicTestingSetup` so that we gain extra flexibility of passing any
config options on the test command line, e.g.:
```
test_bitcoin -- -printtoconsole=1 -checkaddrman=5
```
BlockManager is a large data structure, and cs_main is not required to
take its address or access every part of it. Individual BlockManager
fields and methods which do require cs_main like m_block_index and
LookupBlockIndex are already annotated separately, and these other
annotations describe locking requirements more accurately and do a
better job enforcing thread safety.
Since cs_main is not needed to access the address of the m_block object,
this commit drops cs_main LOCK calls which were added pointlessly to
satisfy this annotation in the past.
Co-authored-by: Carl Dong <contact@carldong.me>
e5b6aef612 Move CBlockFileInfo::ToString method where class is declared (Russell Yanofsky)
f7086fd8ff Add src/wallet/* code to wallet:: namespace (Russell Yanofsky)
90fc8b089d Add src/node/* code to node:: namespace (Russell Yanofsky)
Pull request description:
There are no code changes, this is just adding `namespace` and `using` declarations and `node::` or `wallet::` qualifiers in some places.
Motivations for this change are:
- To make it easier to see when node and wallet code is being accessed places where it shouldn't be. For example if GUI code is accessing node and wallet internals or if wallet and node code are referencing each other.
- To make source code organization clearer ([#15732](https://github.com/bitcoin/bitcoin/issues/15732)), being able to know that `wallet::` code is in `src/wallet/`, `node::` code is in `src/node/`, `init::` code is in `src/init/`, `util::` code is in `src/util/`, etc.
Reviewing with `git log -p -n1 -U0 --word-diff-regex=.` can be helpful to verify this is only updating declarations, not changing code.
ACKs for top commit:
achow101:
ACK e5b6aef612
MarcoFalke:
Concept ACK e5b6aef612🍨
Tree-SHA512: 3797745c90246794e2d55a2ee6e8b0ad5c811e4e03a242d3fdfeb68032f8787f0d48ed4097f6b7730f540220c0af99ef423cd9dbe7f76b2ec12e769a757a2c8d
3a45dc36a6 Change type of `backup_file` parameter in RestoreWallet/restoreWallet (Hennadii Stepanov)
213172c734 refactor: Block unsafe std::string fs::path conversion copy_file calls (Hennadii Stepanov)
Pull request description:
This PR is an optional prerequisite for bitcoin/bitcoin#20744 "Use std::filesystem. Remove Boost Filesystem & System" which:
- makes further code changes safer
- prevents [some](https://cirrus-ci.com/task/6525835388649472) test failures on native Windows
ACKs for top commit:
ryanofsky:
Code review ACK 3a45dc36a6. Looks great! Thanks for debugging and fixing this and making #20744 smaller!
Tree-SHA512: c6dfaef6b45b9c268bc9ee9b943b9d679152c9d565ca4f86da8d33f8eb9b3cdbe9ba6df7b7578eacc0d00db6551048beff97419f86eb4b1d3182c43e2b4eb9a5
`fs::path` looks more native than `std::string` for a parameter which
represents a backup file. This change eliminates back-and-forth type
conversions.
There is no change in behavior. This just helps prepare for the
transition from boost::filesystem to std::filesystem by avoiding
copy_file calls that will be unsafe after the transition to
std::filesystem to due lack of a boost::filesystem::path::imbue
equivalent and inability to set a predictable locale.
fa8e01a5b8 doc: Remove outdated scriptChange TODO comment (MarcoFalke)
Pull request description:
This was added in commit bf798734db (raw multisig). Raw multisig isn't a thing, so remove the TODO.
ACKs for top commit:
S3RK:
ACK fa8e01a5b8
achow101:
ACK fa8e01a5b8
Tree-SHA512: 01d521ca3605ab130c43531da4922ea85461ca5e7436267a34fb5df348009e086b3c66d85532c62255d9a0ba43db56424884808e773d0ef0177035dfb25d6a6c
efab28b06b Add FastRange32 function and use it throughout the codebase (Pieter Wuille)
96ecd6fa3e scripted-diff: rename MapIntoRange to FastRange64 (Pieter Wuille)
c6d15c45d9 [moveonly] Move MapIntoRange() to separate util/fastrange.h (Pieter Wuille)
Pull request description:
Several places in the codebase use the fast range mapping technique described in https://lemire.me/blog/2016/06/27/a-fast-alternative-to-the-modulo-reduction/, some for 32-bit ranges, some for 64-bit ones.
Move all of these to `util/fastrange.h`, and give them a consistent name.
ACKs for top commit:
Sjors:
ACK efab28b06b
shaavan:
reACK efab28b06b
MarcoFalke:
review ACK efab28b06b🍸
Tree-SHA512: 3190a25ef21d17f0ab2afcd9b8d5a1813fdfac0d93996878017e84ff62eee412c823d6149ae8e92cfc3214458641e83ace4b022b4a0fe0679f78dbaee21c6227
6200fbf54f build: rename --enable-ebpf to --enable-usdt (0xb10c)
e158a2a7aa build: add systemtap's sys/sdt.h as depends (0xb10c)
Pull request description:
There has been light conceptual agreement on including the Userspace, Statically Defined Tracing tracepoints in Bitcoin Core release builds. This, for example, enables user to hook into production deployments, if they need to. Binaries don't have to be switched out. This is possible because we don't do [expensive computations](https://github.com/bitcoin/bitcoin/blob/master/doc/tracing.md#no-expensive-computations-for-tracepoints) only needed for the tracepoints. The tracepoints are NOPs when not used.
Systemtap's `sys/sdt.h` header is required to build Bitcoin Core with USDT support. The header file defines the `DTRACE_PROBE` macros used in [`src/util/trace.h`](https://github.com/bitcoin/bitcoin/blob/master/src/util/trace.h). This PR adds Systemtap 4.5 (May 2021) as dependency. GUIX builds for Linux hosts now include the tracepoints.
Closes https://github.com/bitcoin/bitcoin/issues/23297.
ACKs for top commit:
fanquake:
ACK 6200fbf54f - tested enabling / disabling and with/without SDT from depends. We can follow up with #23819, #23907 and #23296, and if any serious issues arise before feature freeze, it is easy for us to flip depends such that USDT becomes opt-in, rather than opt-out, and thus, releases would be tracepoint free.
Tree-SHA512: 0263f44892bf8450e8a593e4de7a498243687f8d81269e1c3283fa8354922c7cf93fddef4b92cf5192d33798424aa5812e03e68ef8de31af078a32dd34021382
