241434200e refactor: qt: Use vQueueNotifications.clear() (João Barbosa)
989e579d07 qt: Make transaction notification queue wallet specific (João Barbosa)
7b3b2303f4 move-only: Define TransactionNotification before TransactionTablePriv (João Barbosa)
Pull request description:
Currently `vQueueNotifications` holds transactions of any wallet, but the queue is dispatched on a given wallet and it assumes notifications are of that wallet.
This means that some transactions can be missed if multiple wallets are loaded.
Fix this by having a queue for each wallet.
ACKs for top commit:
jonasschnelli:
utACK 241434200e
hebasto:
ACK 241434200e, I have reviewed the code and it looks OK, I agree it can be merged.
ryanofsky:
Code review ACK 241434200e. Only change is dropping one commit
Tree-SHA512: 61beac5a16ed659e3a25ad145dbceafcef963aaf8f9838355298949ec2324e2bd760f59353cd251d30cf0334d8dc1642a1f3821d8a9eec092533b581f6ce86db
79ef8324d4 tests: Add fuzzing harness for CConnman (practicalswift)
Pull request description:
Add fuzzing harness for `CConnman`.
See [`doc/fuzzing.md`](https://github.com/bitcoin/bitcoin/blob/master/doc/fuzzing.md) for information on how to fuzz Bitcoin Core. Don't forget to contribute any coverage increasing inputs you find to the [Bitcoin Core fuzzing corpus repo](https://github.com/bitcoin-core/qa-assets).
Happy fuzzing :)
ACKs for top commit:
MarcoFalke:
review ACK 79ef8324d4
Tree-SHA512: eb9ffae20e939b818f8b9def064544b9a8fcd127ca22d1a54af1afedf1d24143be42419f3a03d684be59a5ff07b29d8bfa34ef2aaf1d9f9f75c4c1aaa90a29a8
3c77b8009d fuzz: Improve coverage for CPartialMerkleTree fuzzing harness (practicalswift)
Pull request description:
Improve coverage for `CPartialMerkleTree` fuzzing harness.
See [`doc/fuzzing.md`](https://github.com/bitcoin/bitcoin/blob/master/doc/fuzzing.md) for information on how to fuzz Bitcoin Core. Don't forget to contribute any coverage increasing inputs you find to the [Bitcoin Core fuzzing corpus repo](https://github.com/bitcoin-core/qa-assets).
Happy fuzzing :)
ACKs for top commit:
MarcoFalke:
ACK 3c77b8009d
Tree-SHA512: a1fa0f7650a5ee5ff83f35e41b9faf6c34671fc304b9af00e5b83073f21d50bcbe91c2428fa64d05dc42a7c521bfd24031e307c7f4abf9ded469d69a55c5d64a
ee11a412a5 Avoid signed integer overflow when loading a mempool.dat file with a malformed time field (practicalswift)
Pull request description:
Avoid signed integer overflow when loading a `mempool.dat` file with a malformed time field.
Avoid the following signed integer overflow:
```
$ xxd -p -r > mempool.dat-crash-1 <<EOF
0100000000000000000000000004000000000000000000000000ffffffff
ffffff7f00000000000000000000000000
EOF
$ cp mempool.dat-crash-1 ~/.bitcoin/regtest/mempool.dat
$ UBSAN_OPTIONS="print_stacktrace=1:halt_on_error=1:report_error_type=1" src/bitcoind -regtest
validation.cpp:5079:23: runtime error: signed integer overflow: 9223372036854775807 + 1209600 cannot be represented in type 'long'
#0 0x5618d335197f in LoadMempool(CTxMemPool&) src/validation.cpp:5079:23
#1 0x5618d3350df3 in CChainState::LoadMempool(ArgsManager const&) src/validation.cpp:4217:9
#2 0x5618d2b9345f in ThreadImport(ChainstateManager&, std::vector<boost::filesystem::path, std::allocator<boost::filesystem::path> >, ArgsManager const&) src/init.cpp:762:33
#3 0x5618d2b92162 in AppInitMain(util::Ref const&, NodeContext&, interfaces::BlockAndHeaderTipInfo*)::$_14::operator()() const src/init.cpp:1881:9
```
This PR was broken out from PR #20089. Hopefully this PR is trivial to review.
Fixes a subset of #19278.
ACKs for top commit:
MarcoFalke:
review ACK ee11a412a5
Crypt-iQ:
crACK ee11a412a5
Tree-SHA512: 227ab95cd7d22f62f3191693b455eacfa8e36534961bee12c622fc9090957cfb29992eabafa74d806a336e03385aa8f98b7ce734f04b0b400e33aa187d353337
c82336c493 Remove references to CreateWalletFromFile (fanquake)
Pull request description:
`CWallet::CreateWalletFromFile()` was removed in 8b5e7297c0 but these references remain.
ACKs for top commit:
hebasto:
ACK c82336c493
Tree-SHA512: 3dd50fe0cd5a60bbc96d265107d4739f3e08f943435f3772038963ac4be9e4a87a863412ac0d571226ea66d71550b17b52f01b9d46a6282d49feae1508fd682e
24d2d3341d QA: wallet_multiwallet: Check that recursive symlink directory and wallet.dat loops are ignored (Luke Dashjr)
69f59af54d Bugfix: Wallet: Soft-fail exceptions within ListWalletDir file checks (Luke Dashjr)
Pull request description:
Previously, an exception would be thrown, which could kill the node in some circumstances.
Includes test changes to cause failure.
Review with `?w=1`
ACKs for top commit:
hebasto:
re-ACK 24d2d3341d, rebased only since my [previous](https://github.com/bitcoin/bitcoin/pull/19502#pullrequestreview-520552944) review.
promag:
Tested ACK 24d2d3341d, test change fails on master.
meshcollider:
utACK 24d2d3341d
Tree-SHA512: f701f81b3aa3d3e15cee52ac9e7c31a73c0d8166e56bf077235294507cbcee099829fedc432a1c4b6d8780885f4e37897b44b980b08125771de3c849c000499e
5e146022da wallet: fix scanning progress calculation for single block range (Sebastian Falbesoner)
Pull request description:
If the blockchain is rescanned for a single block (i.e. start and stop hashes are equal, and with that also the estimated start/stop verification progress values) the progress calculation could lead to a NaN value caused by a division by zero (0.0/0.0), resulting in an invalid JSON result for the `getwalletinfo` RPC. This PR fixes this behaviour by setting the progress to zero in that special case. Fixes#20297.
The behaviour can easily be reproduced by continuously running single block rescans in an endless loop, e.g. via
```bash
#!/bin/bash
while true
do
bitcoin-cli rescanblockchain $(bitcoin-cli getblockcount)
done
```
and at the same time perform some `getwalletinfo` RPCs.
On the master branch, this leads to frequent invalid responses (tested on mainchain):
```
$ bitcoin-cli getwalletinfo
error: couldn't parse reply from server
$ curl --user `cat ~/.bitcoin/.cookie` --data-binary '{"jsonrpc": "1.0", "id": "curltest", "method": "getwalletinfo", "params": []}' -H 'content-type: text/plain;' http://127.0.0.1:8332/
{"result":{"walletname":"","walletversion":169900,"format":"bdb","balance":0.00000000,"unconfirmed_balance":0.00000000,"immature_balance":0.00000000,"txcount":0,"keypoololdest":1603677276,"keypoolsize":1000,"hdseedid":"3196e33ecb47c7130e6ca60f2f895f9259860dca","keypoolsize_hd_internal":1000,"paytxfee":0.00000000,"private_keys_enabled":true,"avoid_reuse":false,"scanning":{"duration":0,"progress":},"descriptors":false},"error":null,"id":"curltest"}
```
(note that missing value for "progress" in the JSON result).
On the PR branch, the behaviour doesn't occur anymore.
ACKs for top commit:
MarcoFalke:
review ACK 5e146022da
promag:
Core review ACK 5e146022da.
Tree-SHA512: f0e6aad5a6cd08b36c5fe820fff0ef26663229b39169a4dbe757f3c795a41cf5c69c9dc90efe7515675ae1059307f8971123781a0514d10704123a6f28b125ab
fa949b3c13 test: Suppress epoll_ctl data race (MarcoFalke)
Pull request description:
Happens intermittently: https://cirrus-ci.com/task/5462892373868544?command=ci#L5385
ACKs for top commit:
hebasto:
ACK fa949b3c13, I have reviewed the code and it looks OK, I agree it can be merged.
Tree-SHA512: d5aa559fc105053da594531722f2a03d898eadeb4413c3a728fc5116cc4d1a2c16c49649a24c75ea810e4ec6bb9728b0bcd2ea991886bb9d206170218eddf6d2
fa92cf29d9 ci: Remove redundant valgrind fuzz task (MarcoFalke)
Pull request description:
This task has several issues:
* It slows down other tasks and times out: It needs a lot of resources (CPU, RAM, time), because it builds more than 100 fuzzers, clones a 2 GB repo with 100k seeds and pipes them all through valgrind
* It doesn't add a lot of value: Except for one issue in the boost time library, it hasn't found any issues that the existing fuzz,asan,ubsan fuzzer has already found
* It is redundant: It is already run in the bitcoin-core/qa-assets repo on every push of new seeds and once daily
Fix all issues by removing it here.
Top commit has no ACKs.
Tree-SHA512: 76d16a3e5afc79ba9d89cfeb915d7e66fd1ad4e2035ae4ccd30a21cc060bfba3fb6a904346b7c41606e69d972e86bf660df962673028689227cc26072bba24bd
If the blockchain is rescanned for a single block (i.e. start and stop hashes
are equal, and with that also the estimated verification progress) the progress
calculation could lead to a NaN value caused by a division by zero, resulting in
an invalid JSON result for the getwalletinfo RPC. Fixed by setting the progress
to zero in that special case.
Co-authored-by: MarcoFalke <falke.marco@gmail.com>
fa4234d877 test: Mock IBD in net_processing fuzzers (MarcoFalke)
Pull request description:
Without this the fuzzers fail to detect trivial crasher bugs, such as https://github.com/bitcoin/bitcoin/pull/20317#issuecomment-723047111
ACKs for top commit:
practicalswift:
Tested ACK fa4234d877
Tree-SHA512: ce5da5c0a604b7559805a98ffdde882b44ca4f91b003b493d6e1be230714ce4cccb11dbfc1fc175f9d8fc779551c0a4103ceb4b473552928207d7d78ae329e10
79b8f8d574 fuzz: Assert roundtrip equality for both addrv1 and addrv2 versions of CService (practicalswift)
0e3a78a8ab fuzz: Check for addrv1 compatibility before using addrv1 serializer/deserializer on CSubNet (practicalswift)
Pull request description:
Check for `addrv1` compatibility before using `addrv1` serializer/deserializer on `CSubNet`. As requested by MarcoFalke in https://github.com/bitcoin/bitcoin/pull/20289#issuecomment-724012969.
Assert roundtrip equality for both `addrv1` and `addrv2` versions of `CService`.
ACKs for top commit:
MarcoFalke:
review ACK 79b8f8d574
Tree-SHA512: 3f758aa89ab0c253b593fbe8fe9adc5c6db9afec8856facfe635053a32b4feb438c951323ae0c9e27f1d7e89d12a9b62d81f094dc96159233c12f64d4b95c290
538be4219a wallet: fix importdescriptor silent fail (Ivan Metlushko)
Pull request description:
Currently `importdescriptor` command will successfully import a descriptor with hardened derivations into a watch-only wallet while silently failing to expand the descriptor to fill the cache. This leads to a broken wallet state and failure to load such wallet due to missing cache on subsequent restart.
ACKs for top commit:
laanwj:
Code review ACK 538be4219a
achow101:
ACK 538be4219a
meshcollider:
utACK 538be4219a
Tree-SHA512: 4bdd0ab4437d55b3f1a79c3a300a0b186089155c020fe220a73d0cce274de47d90371d88918d39fd795f9fccf8db328f1e322d29a6062f9ce94a1c254398f004
77777c8b5e ci: Run windows ci config on cirrus (MarcoFalke)
3333d6942e ci: Run macos ci config on cirrus (MarcoFalke)
fa8b1114e6 ci: Run arm ci config on cirrus (MarcoFalke)
fa0795f54d ci: Replace TRAVIS_OS_NAME with CI_OS_NAME (MarcoFalke)
fafce1a13a ci: Move documentation to correct config file (MarcoFalke)
Pull request description:
ACKs for top commit:
hebasto:
ACK 77777c8b5e
Tree-SHA512: 581d3bdb2c3e3da20bd8492c6b23d90f3b4f7f9300ade384667e41de9fbbe42bbcef26dd99328839dc0b49be6e303c4c6d1b66ceb5d8009800c52c97685f9080
bd93fc9945 Fix change detection of imported internal descriptors (Andrew Chow)
Pull request description:
Import internal descriptors were having address book entries added which meant they would be detected as non-change. Fix this and add a test for it.
ACKs for top commit:
laanwj:
Code review ACK bd93fc9945
meshcollider:
utACK bd93fc9945
promag:
Code review ACK bd93fc9945.
Tree-SHA512: 8fa9e364be317627ec171eedffdb505976c0e7f1e55bc7e8cfdffa3aeea5db24d231f55166602cd0e97a5ba621acc871de0a765c75d0c65678f83e93c3b657c5
faa2f06f5e scripted-diff: [build] Ensure source tarball has leading directory name (MarcoFalke)
Pull request description:
This has been fixed in 0.20, so it needs to be fixed on master as well to avoid a regression
#18945
ACKs for top commit:
laanwj:
ACK faa2f06f5e
hebasto:
ACK faa2f06f5e, tested gitian builds only.
promag:
ACK faa2f06f5e.
Tree-SHA512: e3b025c29c45b025002abc35262bb5d771f6cbd807f1c256c477c243685e93cd43ad9f642b38e3cf218590912abe6ea0ddfec3bfbef36f99080aad74ed6cc0af
fa2c3c0d96 ci: Set LC_ALL=C to allow running the s390x tests in qemu (MarcoFalke)
fac0517836 travis: Remove s390x build (MarcoFalke)
Pull request description:
This has been discussed in the last meeting.
Refer to the commit body for more details.
Top commit has no ACKs.
Tree-SHA512: 8e0455286ce41c95ed2e5eb624ac534251bb4a321f13d26d14356497e0c39f841372e166373ffd4a0a9fa379636c2cfb535bd92534fff427cdcb827354e66b6c
66667acc53 cirrus: Skip tasks on the gui repo main branch (MarcoFalke)
Pull request description:
No need to run every build twice, once in the main repo and then in the read-only gui mirror repo
ACKs for top commit:
decryp2kanon:
ACK 66667ac
hebasto:
ACK 66667acc53, though still preferring `only_if` as showing skipped tasks as successful ones seems a bit confused.
Tree-SHA512: 0d35bd115152e06ba4dc5f364130ba5496167d960c44eac2c76192ff9bf7c51f46ab72e2d054dcc6a91818a18dffbbc262f8a4c4483857158c0af4f55dfe9b28
04a69c200e macOS deploy: use the new plistlib API (Jonas Schnelli)
Pull request description:
See https://docs.python.org/3/library/plistlib.html.
The old API was deprecated in 3.4 and removed in 3.9.
~~AFAIK the macdeployplus scripts is only used when calling `make deploy` locally (on macOS). The linux cross compile build (like gitian) are not affected by this PR.~~
ACKs for top commit:
fanquake:
ACK 04a69c200e - I checked that `make deploy` on macOS currently fails when building master and using Python 3.9. This PR fixes that, and it's fine to use (and backport) these changes as they only require Python 3.4. Related note: I think we could just about drop our native_biplist dependency entirely given some changes upstream.
practicalswift:
ACK 04a69c200e: patch looks correct
Tree-SHA512: c5bb60c5157b371d680c82e0978470a488f3edc58cd09e1be635fed59420f227dd113e901c28e15a463da6fe81dc64d08a701b1fdfeb4502f418785707dbebbc
568a1d7261 fix ecdsa verify in test framework (Stepan Snigirev)
Pull request description:
This PR fixes a small bug in the test framework in `verify_ecdsa` function.
`r` in ecdsa signature is modulo curve order, so if the point `R` calculated during verification has x-coordinate that is larger than the curve order, the verification will fail in the test framework but pass in libsecp256k1.
Example (all in hex):
public key: `0289d889551598a0263746c01e5882ccf9b7dc4ca5a37108482c9d80de40e0a8cf`
der signature: `3006020104020104` (r = 4, s = 4)
message: `3232323232323232323232323232323232323232323232323232323232323232`
libsecp256k1 returns `true`, test framework returns `false`.
ACKs for top commit:
sipa:
utACK 568a1d7261
Tree-SHA512: 9e9c58498f10085d2ad85e95caff6c92793799d2a40696ef43febcd7d313c8c3d5ecec715ca903cbb8432a8a96bd0065d86d060966d4ee651c3871ce16c252bf
faf5fa7413 wallet: Set DatabaseStatus::SUCCESS in MakeSQLiteDatabase (MarcoFalke)
Pull request description:
This is a refactor to set the status to `SUCCESS` (like it is done in `MakeBerkeleyDatabase`, too). It also happens to fix a false positive valgrind warning (tested with bionic-gcc and focal-clang):
```
node1 stderr ==28149== Conditional jump or move depends on uninitialised value(s)
==28149== at 0x464471: LoadWallets(interfaces::Chain&) (load.cpp:105)
==28149== by 0x44BFBA: interfaces::(anonymous namespace)::WalletClientImpl::load() (wallet.cpp:510)
==28149== by 0x1640F9: AppInitMain(util::Ref const&, NodeContext&, interfaces::BlockAndHeaderTipInfo*) (init.cpp:1815)
==28149== by 0x144F3F: AppInit (bitcoind.cpp:142)
==28149== by 0x144F3F: main (bitcoind.cpp:172)
==28149==
{
<insert_a_suppression_name_here>
Memcheck:Cond
fun:_Z11LoadWalletsRN10interfaces5ChainE
fun:_ZN10interfaces12_GLOBAL__N_116WalletClientImpl4loadEv
fun:_Z11AppInitMainRKN4util3RefER11NodeContextPN10interfaces21BlockAndHeaderTipInfoE
fun:AppInit
fun:main
}
TEST | STATUS | DURATION
wallet_hd.py --descriptors | ✖ Failed | 69 s
```
ACKs for top commit:
achow101:
ACK faf5fa7413
Tree-SHA512: e8cbac195d05518467f89725d413bdf226d74671eba1c1eb80b3a61d65724af75a1fe93bcb5c608eaa0d54eddce992738bd923e7d83e493f54c3f4c67b66408c
f1f433e8ca Make it easier to reason about node eviction by removing unused NodeEvictionCandidate::addr (CAddress) (practicalswift)
Pull request description:
Make it easier to reason about node eviction by removing unused `NodeEvictionCandidate::addr` (`CAddress`).
ACKs for top commit:
jnewbery:
utACK f1f433e8ca
Tree-SHA512: fef91d7b412b8a4f172370cff6c37eb8c3db0ba618f5daf2dcc8737c8fcef7b9b820d7ee99cd0a9eae7dd653a096cf83d5113776b0d1d9a324147581674e9ede
Without the fix, the tests immediately abort:
terminate called after throwing an instance of 'std::runtime_error'
what(): locale::facet::_S_create_c_locale name not valid
./qt/test/test_bitcoin-qt: line 2: 116150 Aborted
The build didn't find too many issues, and they are generally only
endianness related, so easy to fix if they do make it into the main
developement branch.
With travis migrating its service, it fails too often intermittently. It
might be more appropriate to run it manually once before every release.
