049feabf28 Add missing optional.h include (Kristaps Kaupe)
29c66ace5c Silence false positive GCC warning (Kristaps Kaupe)
Pull request description:
Resolves#20381.
ACKs for top commit:
jnewbery:
utACK 049feabf28
practicalswift:
ACK 049feabf28: diagnostics signal to noise is increased by getting rid of false positives
hebasto:
ACK 049feabf28, I have reviewed the code and it looks OK, I agree it can be merged.
Tree-SHA512: 05d84f51521c3b843ed6bf284a83a91db015ad0cd4fcf8b602275812575c1f6b4899286a89d360fbd3caef184abdfb9d834e119842d8740919892f05a0f9e1f8
406097c810 ci: Use the previous build worker image in AppVeyor (Hennadii Stepanov)
Pull request description:
This is a workaround as the [recent](https://www.appveyor.com/updates/2020/11/14/) Visual Studio 2019 image update breaks our builds.
This PR is alternative to #20392 due to its build [failure](https://ci.appveyor.com/project/DrahtBot/bitcoin/builds/36314660).
ACKs for top commit:
MarcoFalke:
review ACK 406097c810 also seems to pass
Tree-SHA512: f9b722d8e67dd7f0745de6da385064630adb27ecbc0a919be47f62217a2bb7a27a6fa00a7536a24bf17500a77160ca3b92b3c8619047171a6f5198b434015221
CIRRUS_BASE_BRANCH is a PR-specific variable and undocumented on non-PR builds.
In practice (at the moment), it seems to be HEAD, which in private repositories can be pretty much anything, causing CI to fail if it can't be cleanly merged.
By checking CIRRUS_PR first, we can reliably do CI builds of branches outside PRs.
440f8d3abe fix potential devision by 0 (Jonas Schnelli)
Pull request description:
#20344 removed the divide-by-zero sanitizer suppression in `wallet/wallet.cpp` but kept a potential devision by zero in `wallet.cpp`'s fee logging.
Detected here https://bitcoinbuilds.org/index.php?job=ffb7d59f-379f-4f27-a273-a5595b8c5f07
ACKs for top commit:
practicalswift:
ACK 440f8d3abe
laanwj:
Code review ACK 440f8d3abe
hebasto:
re-ACK 440f8d3abe
Tree-SHA512: 9f7903d1e567497c5f972d39e9629c059151e705dbed0a6b88f7c6650c50ecf820f78e3e0f3e629c661d45a938c5d7659faae7c61e47ca8b3bdb029661bca55a
d04a17a790 fuzz: Use ConsumeRandomLengthBitVector(...) in src/test/fuzz/connman and src/test/fuzz/net (practicalswift)
e6bb9fde85 tests: Add fuzzing harness for CAddrMan (practicalswift)
Pull request description:
Add fuzzing harness for `CAddrMan`.
~~Fill some fuzzing coverage gaps for functions in `addrdb.h`, `merkleblock.h` and `outputtype.h`.~~
See [`doc/fuzzing.md`](https://github.com/bitcoin/bitcoin/blob/master/doc/fuzzing.md) for information on how to fuzz Bitcoin Core. Don't forget to contribute any coverage increasing inputs you find to the [Bitcoin Core fuzzing corpus repo](https://github.com/bitcoin-core/qa-assets).
Happy fuzzing :)
ACKs for top commit:
MarcoFalke:
review ACK d04a17a790
Tree-SHA512: a6b627e3a0cb51e3a8cb02ad0f19088fc0e965ca34ab110b68d5822d0ea7f473207ae312b49fb217cb6cf2f9f211d00bb69c83bac9f50d79c9ed1e157e85775d
0ccb3addf6 tests: Remove no longer needed UBSan suppression (float-divide-by-zero in validation.cpp) (practicalswift)
Pull request description:
Remove no longer needed UBSan suppression.
The float divide-by-zero in `validation.cpp` was fixed by instagibbs in ec30a79f1c (#15283).
ACKs for top commit:
MarcoFalke:
ACK 0ccb3addf6
Tree-SHA512: 89a4f4b7371fa5725d9f801cee7ebbd17523f66017c9acfa813657dcb8d837f42209eff44ce9e5d48296a630bab9599d75f10024a0c7da7defb228f4eae3392a
38ada892ed addrman: ensure old versions don't parse peers.dat (Vasil Dimov)
Pull request description:
Even though the format of `peers.dat` was changed in a backwards
incompatible way, it is not guaranteed that old versions will fail to
parse it. There is a chance that old versions parse its contents as
garbage and use it.
Old versions expect the "key size" field to be 32 and fail the parsing
if it is not. Thus, we put something other than 32 in it. This will make
versions between 0.11.0 and 0.20.1 deterministically fail on the new
format. Versions prior to https://github.com/bitcoin/bitcoin/pull/5941
will still parse it as garbage.
Also, introduce a way to increment the `peers.dat` format in a way that
does not necessary make older versions refuse to read it.
ACKs for top commit:
jnewbery:
ACK 38ada892ed
laanwj:
Code review ACK 38ada892ed
MarcoFalke:
re-ACK 38ada892ed🥐
Tree-SHA512: 550bd660c5019dba0f9c334aca8a11c4a0463cfddf11efe7a4a5585ffb05549c82b95066fba5d073ae37893e0eccc158a7ffea9b33ea031d9be4a39e44f6face
241434200e refactor: qt: Use vQueueNotifications.clear() (João Barbosa)
989e579d07 qt: Make transaction notification queue wallet specific (João Barbosa)
7b3b2303f4 move-only: Define TransactionNotification before TransactionTablePriv (João Barbosa)
Pull request description:
Currently `vQueueNotifications` holds transactions of any wallet, but the queue is dispatched on a given wallet and it assumes notifications are of that wallet.
This means that some transactions can be missed if multiple wallets are loaded.
Fix this by having a queue for each wallet.
ACKs for top commit:
jonasschnelli:
utACK 241434200e
hebasto:
ACK 241434200e, I have reviewed the code and it looks OK, I agree it can be merged.
ryanofsky:
Code review ACK 241434200e. Only change is dropping one commit
Tree-SHA512: 61beac5a16ed659e3a25ad145dbceafcef963aaf8f9838355298949ec2324e2bd760f59353cd251d30cf0334d8dc1642a1f3821d8a9eec092533b581f6ce86db
79ef8324d4 tests: Add fuzzing harness for CConnman (practicalswift)
Pull request description:
Add fuzzing harness for `CConnman`.
See [`doc/fuzzing.md`](https://github.com/bitcoin/bitcoin/blob/master/doc/fuzzing.md) for information on how to fuzz Bitcoin Core. Don't forget to contribute any coverage increasing inputs you find to the [Bitcoin Core fuzzing corpus repo](https://github.com/bitcoin-core/qa-assets).
Happy fuzzing :)
ACKs for top commit:
MarcoFalke:
review ACK 79ef8324d4
Tree-SHA512: eb9ffae20e939b818f8b9def064544b9a8fcd127ca22d1a54af1afedf1d24143be42419f3a03d684be59a5ff07b29d8bfa34ef2aaf1d9f9f75c4c1aaa90a29a8
3c77b8009d fuzz: Improve coverage for CPartialMerkleTree fuzzing harness (practicalswift)
Pull request description:
Improve coverage for `CPartialMerkleTree` fuzzing harness.
See [`doc/fuzzing.md`](https://github.com/bitcoin/bitcoin/blob/master/doc/fuzzing.md) for information on how to fuzz Bitcoin Core. Don't forget to contribute any coverage increasing inputs you find to the [Bitcoin Core fuzzing corpus repo](https://github.com/bitcoin-core/qa-assets).
Happy fuzzing :)
ACKs for top commit:
MarcoFalke:
ACK 3c77b8009d
Tree-SHA512: a1fa0f7650a5ee5ff83f35e41b9faf6c34671fc304b9af00e5b83073f21d50bcbe91c2428fa64d05dc42a7c521bfd24031e307c7f4abf9ded469d69a55c5d64a
ee11a412a5 Avoid signed integer overflow when loading a mempool.dat file with a malformed time field (practicalswift)
Pull request description:
Avoid signed integer overflow when loading a `mempool.dat` file with a malformed time field.
Avoid the following signed integer overflow:
```
$ xxd -p -r > mempool.dat-crash-1 <<EOF
0100000000000000000000000004000000000000000000000000ffffffff
ffffff7f00000000000000000000000000
EOF
$ cp mempool.dat-crash-1 ~/.bitcoin/regtest/mempool.dat
$ UBSAN_OPTIONS="print_stacktrace=1:halt_on_error=1:report_error_type=1" src/bitcoind -regtest
validation.cpp:5079:23: runtime error: signed integer overflow: 9223372036854775807 + 1209600 cannot be represented in type 'long'
#0 0x5618d335197f in LoadMempool(CTxMemPool&) src/validation.cpp:5079:23
#1 0x5618d3350df3 in CChainState::LoadMempool(ArgsManager const&) src/validation.cpp:4217:9
#2 0x5618d2b9345f in ThreadImport(ChainstateManager&, std::vector<boost::filesystem::path, std::allocator<boost::filesystem::path> >, ArgsManager const&) src/init.cpp:762:33
#3 0x5618d2b92162 in AppInitMain(util::Ref const&, NodeContext&, interfaces::BlockAndHeaderTipInfo*)::$_14::operator()() const src/init.cpp:1881:9
```
This PR was broken out from PR #20089. Hopefully this PR is trivial to review.
Fixes a subset of #19278.
ACKs for top commit:
MarcoFalke:
review ACK ee11a412a5
Crypt-iQ:
crACK ee11a412a5
Tree-SHA512: 227ab95cd7d22f62f3191693b455eacfa8e36534961bee12c622fc9090957cfb29992eabafa74d806a336e03385aa8f98b7ce734f04b0b400e33aa187d353337
c82336c493 Remove references to CreateWalletFromFile (fanquake)
Pull request description:
`CWallet::CreateWalletFromFile()` was removed in 8b5e7297c0 but these references remain.
ACKs for top commit:
hebasto:
ACK c82336c493
Tree-SHA512: 3dd50fe0cd5a60bbc96d265107d4739f3e08f943435f3772038963ac4be9e4a87a863412ac0d571226ea66d71550b17b52f01b9d46a6282d49feae1508fd682e
24d2d3341d QA: wallet_multiwallet: Check that recursive symlink directory and wallet.dat loops are ignored (Luke Dashjr)
69f59af54d Bugfix: Wallet: Soft-fail exceptions within ListWalletDir file checks (Luke Dashjr)
Pull request description:
Previously, an exception would be thrown, which could kill the node in some circumstances.
Includes test changes to cause failure.
Review with `?w=1`
ACKs for top commit:
hebasto:
re-ACK 24d2d3341d, rebased only since my [previous](https://github.com/bitcoin/bitcoin/pull/19502#pullrequestreview-520552944) review.
promag:
Tested ACK 24d2d3341d, test change fails on master.
meshcollider:
utACK 24d2d3341d
Tree-SHA512: f701f81b3aa3d3e15cee52ac9e7c31a73c0d8166e56bf077235294507cbcee099829fedc432a1c4b6d8780885f4e37897b44b980b08125771de3c849c000499e
5e146022da wallet: fix scanning progress calculation for single block range (Sebastian Falbesoner)
Pull request description:
If the blockchain is rescanned for a single block (i.e. start and stop hashes are equal, and with that also the estimated start/stop verification progress values) the progress calculation could lead to a NaN value caused by a division by zero (0.0/0.0), resulting in an invalid JSON result for the `getwalletinfo` RPC. This PR fixes this behaviour by setting the progress to zero in that special case. Fixes#20297.
The behaviour can easily be reproduced by continuously running single block rescans in an endless loop, e.g. via
```bash
#!/bin/bash
while true
do
bitcoin-cli rescanblockchain $(bitcoin-cli getblockcount)
done
```
and at the same time perform some `getwalletinfo` RPCs.
On the master branch, this leads to frequent invalid responses (tested on mainchain):
```
$ bitcoin-cli getwalletinfo
error: couldn't parse reply from server
$ curl --user `cat ~/.bitcoin/.cookie` --data-binary '{"jsonrpc": "1.0", "id": "curltest", "method": "getwalletinfo", "params": []}' -H 'content-type: text/plain;' http://127.0.0.1:8332/
{"result":{"walletname":"","walletversion":169900,"format":"bdb","balance":0.00000000,"unconfirmed_balance":0.00000000,"immature_balance":0.00000000,"txcount":0,"keypoololdest":1603677276,"keypoolsize":1000,"hdseedid":"3196e33ecb47c7130e6ca60f2f895f9259860dca","keypoolsize_hd_internal":1000,"paytxfee":0.00000000,"private_keys_enabled":true,"avoid_reuse":false,"scanning":{"duration":0,"progress":},"descriptors":false},"error":null,"id":"curltest"}
```
(note that missing value for "progress" in the JSON result).
On the PR branch, the behaviour doesn't occur anymore.
ACKs for top commit:
MarcoFalke:
review ACK 5e146022da
promag:
Core review ACK 5e146022da.
Tree-SHA512: f0e6aad5a6cd08b36c5fe820fff0ef26663229b39169a4dbe757f3c795a41cf5c69c9dc90efe7515675ae1059307f8971123781a0514d10704123a6f28b125ab
Even though the format of `peers.dat` was changed in an incompatible
way (old software versions <0.21 cannot understand the new file format),
it is not guaranteed that old versions will fail to parse it. There is a
chance that old versions parse its contents as garbage and use it.
Old versions expect the "key size" field to be 32 and fail the parsing
if it is not. Thus, we put something other than 32 in it. This will make
versions between 0.11.0 and 0.20.1 deterministically fail on the new
format. Versions prior to https://github.com/bitcoin/bitcoin/pull/5941
(<0.11.0) will still parse it as garbage.
Also, introduce a way to increment the `peers.dat` format in a way that
does not necessary make older versions refuse to read it.
Create a fee_rate (sat/vB) RPC param and replace overloading the conf_target and
estimate_mode params in the following 6 RPCs with it:
- sendtoaddress
- sendmany
- send
- fundrawtransaction
- walletcreatefundedpsbt
- bumpfee
In RPC bumpfee, the previously existing fee_rate remains but the unit is changed
from BTC/kvB to sat/vB. This is a breaking change, but it should not be an
overly risky one, as the units change by a factor of 1e5 and any fees specified
in BTC/kvB after this commit will either be too low and raise an error or be 1
sat/vB and can be RBFed.
Update the test coverage for each RPC.
Co-authored-by: Murch <murch@murch.one>
fa949b3c13 test: Suppress epoll_ctl data race (MarcoFalke)
Pull request description:
Happens intermittently: https://cirrus-ci.com/task/5462892373868544?command=ci#L5385
ACKs for top commit:
hebasto:
ACK fa949b3c13, I have reviewed the code and it looks OK, I agree it can be merged.
Tree-SHA512: d5aa559fc105053da594531722f2a03d898eadeb4413c3a728fc5116cc4d1a2c16c49649a24c75ea810e4ec6bb9728b0bcd2ea991886bb9d206170218eddf6d2
fa92cf29d9 ci: Remove redundant valgrind fuzz task (MarcoFalke)
Pull request description:
This task has several issues:
* It slows down other tasks and times out: It needs a lot of resources (CPU, RAM, time), because it builds more than 100 fuzzers, clones a 2 GB repo with 100k seeds and pipes them all through valgrind
* It doesn't add a lot of value: Except for one issue in the boost time library, it hasn't found any issues that the existing fuzz,asan,ubsan fuzzer has already found
* It is redundant: It is already run in the bitcoin-core/qa-assets repo on every push of new seeds and once daily
Fix all issues by removing it here.
Top commit has no ACKs.
Tree-SHA512: 76d16a3e5afc79ba9d89cfeb915d7e66fd1ad4e2035ae4ccd30a21cc060bfba3fb6a904346b7c41606e69d972e86bf660df962673028689227cc26072bba24bd
If the blockchain is rescanned for a single block (i.e. start and stop hashes
are equal, and with that also the estimated verification progress) the progress
calculation could lead to a NaN value caused by a division by zero, resulting in
an invalid JSON result for the getwalletinfo RPC. Fixed by setting the progress
to zero in that special case.
Co-authored-by: MarcoFalke <falke.marco@gmail.com>
native_ds_store now takes advantage of Pythons ability to decode binary
plists (since 3.4), so we can drop its biplist dependency.
The call to biplist.Data() in custom_dsstore doesn't seem to do anything,
and from what I can tell can just be removed.
fa4234d877 test: Mock IBD in net_processing fuzzers (MarcoFalke)
Pull request description:
Without this the fuzzers fail to detect trivial crasher bugs, such as https://github.com/bitcoin/bitcoin/pull/20317#issuecomment-723047111
ACKs for top commit:
practicalswift:
Tested ACK fa4234d877
Tree-SHA512: ce5da5c0a604b7559805a98ffdde882b44ca4f91b003b493d6e1be230714ce4cccb11dbfc1fc175f9d8fc779551c0a4103ceb4b473552928207d7d78ae329e10
79b8f8d574 fuzz: Assert roundtrip equality for both addrv1 and addrv2 versions of CService (practicalswift)
0e3a78a8ab fuzz: Check for addrv1 compatibility before using addrv1 serializer/deserializer on CSubNet (practicalswift)
Pull request description:
Check for `addrv1` compatibility before using `addrv1` serializer/deserializer on `CSubNet`. As requested by MarcoFalke in https://github.com/bitcoin/bitcoin/pull/20289#issuecomment-724012969.
Assert roundtrip equality for both `addrv1` and `addrv2` versions of `CService`.
ACKs for top commit:
MarcoFalke:
review ACK 79b8f8d574
Tree-SHA512: 3f758aa89ab0c253b593fbe8fe9adc5c6db9afec8856facfe635053a32b4feb438c951323ae0c9e27f1d7e89d12a9b62d81f094dc96159233c12f64d4b95c290
Previously, when running ./configure:
1. With CONFIG_SITE pointed to our depends config.site.in, and
2. PYTHONPATH was not set either in the environment or by the user
The configure would output something like:
PYTHONPATH='depends/x86_64-pc-linux-gnu/share/../native/lib/python3/dist-packages:'
When we really mean:
PYTHONPATH='depends/x86_64-pc-linux-gnu/share/../native/lib/python3/dist-packages'
...without the colon
This change makes sure that:
1. There's no trailing colon, and
2. We use the $PATH_SEPARATOR variable instead of a colon
