This is needed so that it can be used by RPCResult
Also,
* rename NAMED_ARG to NONE for generalization.
* change RPCArg constructors to initialize the members by moving values
5bad7921d0 [test] PSBT RPC: check that bip32_derivs are present by default (Sjors Provoost)
29a21c9061 [rpc] set default bip32derivs to true for psbt methods (Sjors Provoost)
Pull request description:
In https://github.com/bitcoin/bitcoin/pull/13557#pullrequestreview-135905054 I recommended not including bip32 deriviation by default in PSBTs:
> _Bit of a privacy issue_: let's say person A and B are about to spend from a multisig address, sending everything to person A. Person A gives their address to person B, their wallet wallet creates a PSBT, but doesn't sign it. Wallet A then calls `walletprocesspsbt` which signs it and _spontaneously adds the master_fingerprint and bip32 path_. Same issue with `walletcreatefundedpsbt`.
>
> Adding `bip32_derivs` should probably be opt-in.
In practice I find this default quite annoying because I forget it and end up with a confused hardware wallet.
More importantly, in the multisig example I provided, it's actually essential for the other side to know the derivation details (in addition to an xpub). This allows them to check that change is going to an address you can still co-sign for (because the spending policy is unchanged except for an index).
ACKs for top commit:
instagibbs:
utACK 5bad7921d0
jonatack:
ACK 5bad7921d0 code review, built, ran tests, inspected/messed around with/pprinted values from the new tests. Thanks for adding the tests.
meshcollider:
utACK 5bad7921d0
Tree-SHA512: 22ad71dda96856060a96758c4ae7aafa22d5e9efba30e0c8287c711e7579849bd72593cbc0f41a2e9e8821315d78bda04e848dbb006283b841b2795e2faebcfd
e193a84fb2 Refactor message hashing into a utility function (Jeffrey Czyz)
f8f0d9893d Deduplicate the message signing code (Vasil Dimov)
2ce3447eb1 Deduplicate the message verifying code (Vasil Dimov)
Pull request description:
The message signing and verifying logic was replicated in a few places
in the code. Consolidate in a newly introduced `MessageSign()` and
`MessageVerify()` and add unit tests for them.
ACKs for top commit:
Sjors:
re-ACK e193a84fb2
achow101:
ACK e193a84fb2
instagibbs:
utACK e193a84fb2
meshcollider:
utACK e193a84fb2
Tree-SHA512: b0e02a7d4623a98c8f8c77627af1725e6df07700de4630c2f75da6beacdf55414c38ba147bc6d2a757491ab07c827dddf93e8632fe600478760e255714ddab88
12a2f37718 util: Avoid potential uninitialized read in FormatISO8601DateTime(int64_t nTime) by checking gmtime_s/gmtime_r return value (practicalswift)
Pull request description:
Avoid potential uninitialized read in `FormatISO8601DateTime(int64_t)` by checking `gmtime_s`/`gmtime_r` return value.
Before this patch `FormatISO8601DateTime(67768036191676800)` resulted in:
```
==5930== Conditional jump or move depends on uninitialised value(s)
==5930== at 0x4F44C0A: std::ostreambuf_iterator<char, std::char_traits<char> > std::num_put<char, std::ostreambuf_iterator<char, std::char_traits<char> > >::_M_insert_int<long>(std::ostreambuf_iterator<char, std::char_traits<char> >, std::ios_base&, char, long) const (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.25)
==5930== by 0x4F511A4: std::ostream& std::ostream::_M_insert<long>(long) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.25)
==5930== by 0x4037C3: void tinyformat::formatValue<int>(std::ostream&, char const*, char const*, int, int const&) (tinyformat.h:358)
==5930== by 0x403725: void tinyformat::detail::FormatArg::formatImpl<int>(std::ostream&, char const*, char const*, int, void const*) (tinyformat.h:543)
==5930== by 0x402E02: tinyformat::detail::FormatArg::format(std::ostream&, char const*, char const*, int) const (tinyformat.h:528)
==5930== by 0x401B16: tinyformat::detail::formatImpl(std::ostream&, char const*, tinyformat::detail::FormatArg const*, int) (tinyformat.h:907)
==5930== by 0x4017AE: tinyformat::vformat(std::ostream&, char const*, tinyformat::FormatList const&) (tinyformat.h:1054)
==5930== by 0x401765: void tinyformat::format<int, int, int, int, int, int>(std::ostream&, char const*, int const&, int const&, int const&, int const&, int const&, int const&) (tinyformat.h:1064)
==5930== by 0x401656: std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > tinyformat::format<int, int, int, int, int, int>(char const*, int const&, int const&, int const&, int const&, int const&, int const&) (tinyformat.h:1073)
==5930== by 0x4014CC: FormatISO8601DateTime[abi:cxx11](long) (…)
```
The same goes for other very large positive and negative arguments.
Fix by simply checking the `gmtime_s`/`gmtime_r` return value :)
ACKs for top commit:
MarcoFalke:
ACK 12a2f37718
theStack:
re-ACK 12a2f37718
elichai:
re ACK 12a2f37718
Tree-SHA512: 066142670d9bf0944d41fa3f3c702b1a460b5471b93e76a619b1e818ff9bb9c09fe14c4c37e9536a04c99533f7f21d1b08ac141e1b829ff87ee54c80d0e61d48
bca8665d08 scripted-diff: Wallet: Rename incorrectly named *UsedDestination (Luke Dashjr)
Pull request description:
These functions are used to mark/check if a key of our own has been used to spend (and only for avoid-reuse wallets), which has nothing to do with the destination/address itself.
Give them more accurate names to avoid confusion.
-BEGIN VERIFY SCRIPT-
sed -i -e 's/UsedDestination/SpentKey/g' $(git grep -l 'UsedDestination' ./src)
-END VERIFY SCRIPT-
ACKs for top commit:
practicalswift:
ACK bca8665d08 -- patch looks correct and rationale makes sense
instagibbs:
ACK bca8665d08, much more meaningful name, thanks
kallewoof:
ACK bca8665d08
Tree-SHA512: ff13d9061ffa748e92eb41ba962c3ec262a43e4b6abd62408b38c6f650395d6ae5851554257d1900fb02767a88d08380d592a27210192ee9abb72d0945976686
faca8eff39 test: Remove incorrect assumptions in validation_flush_tests (MarcoFalke)
fa31eebfe9 test: Tabs to spaces in all tests (MarcoFalke)
Pull request description:
The tests assume standard library internals that may not hold on all supported archs or when the code is instrumented for sanitizer or debug use cases
Fixes#18111
ACKs for top commit:
jamesob:
ACK faca8eff39 pending passing tests
fjahr:
ACK faca8eff39
Tree-SHA512: 60a5ae824bdffb0762f82f67957b31b185385900be5e676fcb12c23d53f5eea734601680c2e3f0bdb8052ce90e7ca1911b1342affb67e43d91a506b111406f41
fac52dafa0 test: Set catch_system_errors=no on boost unit tests (MarcoFalke)
Pull request description:
Closes#16700
Can be tested by adding an `assert(0)` and then running either `make check` or `./src/test/test_bitcoin -t bla_tests --catch_system_errors=no/yes`
ACKs for top commit:
practicalswift:
ACK fac52dafa0
Empact:
Tested ACK fac52dafa0
Tree-SHA512: ec00636951b2c1137aaf43610739d78d16f823f7da76a726d47f93b8b089766fb66b21504b3c5413bcf8b6b5c3db0ad74027d677db24a44487d6d79a6bdee2e0
These functions are used to mark/check if a key of our own has been used to spend (and only for avoid-reuse wallets), which has nothing to do with the destination/address itself.
Give them more accurate names to avoid confusion.
-BEGIN VERIFY SCRIPT-
sed -i -e 's/UsedDestination/SpentKey/g' $(git grep -l 'UsedDestination' ./src)
-END VERIFY SCRIPT-
7e80f646b2 Get the OutputType for a descriptor (Andrew Chow)
Pull request description:
Adds a `GetOutputType()` method to get the OutputType of a descriptor. Some descriptors don't have a determinate OutputType, so we actually use an `Optional<OutputType>`. For descriptors with indeterminate OutputType, we return `nullopt`.
`addr()` and `raw()` use OutputTypes as determined by the CTxDestination they have. For simplicity, `ScriptHash` destinations are `LEGACY` even though they could be `P2SH_SEGWIT`.
`combo()`, `pk()`, and `multi()` are `nullopt` as they either don't have an OutputType or they have multiple. `DescriptorImpl` defaults to `nullopt`.
`pkh()` is `LEGACY` as expected
`wpkh()` and `wsh()` are `BECH32` as expected.
`sh()` checks whether the sub-descriptor is `BECH32`. If so, it is `P2SH_SEGWIT`. Otherwise it is `LEGACY`.
The descriptor tests are updated to check the OutputType too.
ACKs for top commit:
fjahr:
ACK 7e80f646b2
meshcollider:
utACK 7e80f646b2
instagibbs:
cursory ACK 7e80f646b2
Sjors:
Code review ACK 7e80f646b2
jonatack:
ACK 7e80f64 code review/build/tests
Tree-SHA512: c5a813447b62e982435e1c948066f8d6c148c9ebffb0a5eb5a9028b173b01d5ead2f076a5ca3f7f37698538baa346f82a977ee48f583d89cb4e5ebd9111b2341
d6d2602a32 add: test that transactions expire from mempool (0xb10c)
Pull request description:
This adds the functional test `mempool_expiry.py` covering mempool transaction expiry. Both the default `DEFAULT_MEMPOOL_EXPIRY` of 336 hours (two weeks, set in #9312) and the user definable mempool expiry via the `-mempoolexpiry=<n>` command line option are tested. The test checks that descendants of expired transactions are removed as well.
*Notes for reviewers*
- `LimitMempoolSize()` (which is the only caller of `CTxMemPool::Expire()`) is only called when a transaction is added to the mempool. In order to test expiry of a transaction-that-should-expire, the mocktime is set and a random transaction is broadcast to trigger `LimitMempoolSize()`. The transaction-that-should-expire is then checked for expiry. LMK if there is another way, but I don't think there is.
ACKs for top commit:
MarcoFalke:
ACK d6d2602a32
theStack:
ACK d6d2602a32
promag:
Code review ACK d6d2602a32.
Tree-SHA512: eb68cd9e2d870872b8e8e1522fed8954fb99cc9e4edda4b28bb2a4e41cddbc53fe6f7d9c090f1e0e98ab49beb24bf37ff3787a9e9801a95e8ae9ca9eb34fe6f0
This is safe because MilliSleep is never executed in a boost::thread,
the only type of thread that is interruptible.
* The RPC server uses std::thread
* The wallet is either executed in an RPC thread or the main thread
* bitcoin-cli, benchmarks and tests are only one thread (the main thread)
-BEGIN VERIFY SCRIPT-
sed -i --regexp-extended -e 's/MilliSleep\((\S+)\);/UninterruptibleSleep(std::chrono::milliseconds{\1});/g' $(git grep -l MilliSleep)
-END VERIFY SCRIPT-
63ce882760 doc: link to homebrew's troubleshooting page (Gastón I. Silva)
Pull request description:
A trivial documentation update.
When I was following the build steps for mac, I had some errors installing the dependencies. After searching on the Internet, and correcting the errors, I found that `brew doctor` had all the answers I needed. Could have skipped the Internet searches all together.
ACKs for top commit:
fanquake:
ACK 63ce882760 - a link to the troubleshooting page seems fine. I wouldn't really want our README to have anything more specific than that.
Tree-SHA512: 12c96cd9c9bd39ada21f3f27cbec3ed4bef4b8e74dec7872c892fc6a92a70418a5cc0882ff449883e91d96c01e1ca7104b076590917f397334c82931ec7fda1c
7f1475c711 rpc: update validateaddress RPCExamples to bech32 (Sebastian Falbesoner)
Pull request description:
Another small step to get rid of legacy addresses in the RPC help texts and by that encourage the use of bech32 addresses by default. The (invalid) address is the same as in the `getaddressinfo` RPC (see 2ee0cb3330, kudos to jonatack!), I don't think it adds any value to have a different example address per RPC.
ACKs for top commit:
fanquake:
ACK 7f1475c711
MarcoFalke:
ACK 7f1475c711
Tree-SHA512: 2350f61fa942a9053f9f5c860ea446965dc7209c71c81bdb98a859d03ca23b225ad72c9c506e4a55c8d8988823d9cfbe808c1a452a1eeadb70ab186b146dd4ca
f2472f6460 tests: Improve test runner output in case of target errors (practicalswift)
733bbec34f tests: Add --exclude integer,parse_iso8601 (temporarily) to make Travis pass until uninitialized read issue in FormatISO8601DateTime is fixed (practicalswift)
5ea81449f3 tests: Add support for excluding fuzz targets using -x/--exclude (practicalswift)
555236f769 tests: Remove -detect_leaks=0 from test/fuzz/test_runner.py - no longer needed (practicalswift)
a3b539a924 ci: Run fuzz testing test cases under valgrind (practicalswift)
Pull request description:
Run fuzz testing [test cases (bitcoin-core/qa-assets)](https://github.com/bitcoin-core/qa-assets) under `valgrind`.
This would have caught `util: Avoid potential uninitialized read in FormatISO8601DateTime(int64_t) by checking gmtime_s/gmtime_r return value` (#18162) and similar cases.
ACKs for top commit:
MarcoFalke:
ACK f2472f6460👼
Tree-SHA512: bb0879d40167cf6906bc0ed31bed39db83c39c7beb46026f7b0ee53f28ff0526ad6fabc3f4cb3f5f18d3b8cafdcbf5f30105b35919f4e83697c71e838ed71493
This tests that a mempool transaction expires after a given timeout
and its children are removed as well.
Both the default expiry timeout defied by DEFAULT_MEMPOOL_EXPIRY and
a user definable expiry timeout via the -mempoolexpiry=<n> command
line argument (<n> is the timeout in hours) are tested.
a304a3632f Revert "Store p2sh scripts in AddAndGetDestinationForScript" (Russell Yanofsky)
eb7d8a5b07 [test] check for addmultisigaddress regression (Sjors Provoost)
005f8a92cc wallet: Improve LegacyScriptPubKeyMan::CanProvide script recognition (Russell Yanofsky)
Pull request description:
Make `LegacyScriptPubKeyMan::CanProvide` method able to recognize p2sh scripts when the redeem script is present in the `mapScripts` map without the p2sh script also having to be added to the `mapScripts` map. This restores behavior prior to #17261, which I think broke backwards compatibility with old wallet files by no longer treating addresses created by `addmultisigaddress` calls before #17261 as solvable.
The reason why tests didn't fail with the CanProvide implementation in #17261 is because of a workaround added in 4a7e43e846 "Store p2sh scripts in AddAndGetDestinationForScript", which masked the problem for new `addmultisigaddress` RPC calls without fixing it for multisig addresses already created in old wallet files.
This change adds a lot of comments and allows reverting commit 4a7e43e846 "Store p2sh scripts in AddAndGetDestinationForScript", so the `AddAndGetDestinationForScript()` function, `CanProvide()` method, and `mapScripts` map should all be more comprehensible
ACKs for top commit:
Sjors:
re-ACK a304a3632f (rebase, slight text changes and my test)
achow101:
re-ACK a304a3632f
meshcollider:
utACK a304a3632f
Tree-SHA512: 03b625220c49684c376a8062d7646aeba0e5bfe043f977dc7dc357a6754627d594e070e4d458d12d2291888405d94c1dbe08c7787c318374cedd5755e724fb6e
Before, macros defined in `<event2/http.h>` have been used
for some HTTP status codes.
`<event2/http.h>` is included implicitly and the usage
of its status code macros is inconsistent with the majority
HTTP response implementations in this file.
Now, the `HTTPStatusCode` enum from `<rpc/protocol.h>` is
consistently used for all HTTP response implementations.
8bca30ea17 [rpc] expose ability to mock scheduler via the rpc (Amiti Uttarwar)
7c8b6e5b52 [lib] add scheduler to node context (Amiti Uttarwar)
930d837542 [test] add chainparams property to indicate chain allows time mocking (Amiti Uttarwar)
1cd43e83c6 [test] unit test for new MockForward scheduler method (Amiti Uttarwar)
a6f63598ad [util] allow scheduler to be mocked (Amiti Uttarwar)
Pull request description:
This PR is to support functional tests by allowing the scheduler to be mocked via the RPC.
It adds a `MockForward` method to the scheduler class that iterates through the task queue and reschedules them to be `delta_seconds` sooner.
This is currently used to support functional testing of the "unbroadcast" set tracking in #18038. If this patch is accepted, it would also be useful to simplify the code in #16698.
ACKs for top commit:
MarcoFalke:
ACK 8bca30ea17, only change is some style fixups 🕓
Tree-SHA512: 2a97fe8ade2b7fd1fb5cdfa1dcafb3227a377d7a847e3845a228bc119eb77824b4aefa43d922a06d583939b22725e223f308cf092961048079d36f6b1d9a639b
530d02addb build: pass -fno-ident in Windows gitian descriptor (fanquake)
Pull request description:
`-fno-ident` prevents compilers from emitting compiler name and version number information that can needlessly bloat binaries.
For example, in the `v0.19.0.1` Windows release binaries, there are > 1000 GCC compiler version strings embedded:
```bash
# GCC: (GNU) 7.3-posix 20180312... & GCC: (GNU) 6.3.0 20170415.......
strings bitcoind.exe | rg GCC | wc -l
1021
```
They end up collected in the end of the`.rdata` section, and cannot be removed by `strip`. i.e:
```bash
objdump --section=.rdata --full-contents bitcoind.exe
...
cfcc00 00000000 00000000 00000000 00000000 ................
cfcc10 00000000 00000000 00000000 00000000 ................
cfcc20 4743433a 2028474e 55292036 2e332e30 GCC: (GNU) 6.3.0
cfcc30 20323031 37303431 35000000 00000000 20170415.......
cfcc40 4743433a 2028474e 55292037 2e332d70 GCC: (GNU) 7.3-p
cfcc50 6f736978 20323031 38303331 32000000 osix 20180312...
cfcc60 4743433a 2028474e 55292037 2e332d70 GCC: (GNU) 7.3-p
cfcc70 6f736978 20323031 38303331 32000000 osix 20180312...
```
The flag is available for [Clang](https://clang.llvm.org/docs/ClangCommandLineReference.html#cmdoption-clang-qn) and [GCC](https://gcc.gnu.org/onlinedocs/gcc/Code-Gen-Options.html#index-fno-ident).
Relevant code in [GCC](https://github.com/gcc-mirror/gcc/blob/master/gcc/toplev.c#L565-L578):
```c
/* Attach a special .ident directive to the end of the file to identify
the version of GCC which compiled this code. The format of the .ident
string is patterned after the ones produced by native SVR4 compilers. */
if (!flag_no_ident)
{
const char *pkg_version = "(GNU) ";
char *ident_str;
if (strcmp ("(GCC) ", pkgversion_string))
pkg_version = pkgversion_string;
ident_str = ACONCAT (("GCC: ", pkg_version, version_string, NULL));
targetm.asm_out.output_ident (ident_str);
}
```
ACKs for top commit:
practicalswift:
ACK 530d02addb
laanwj:
ACK 530d02addb
Tree-SHA512: b3b28f43ec483dee28d1df8548fe72425bf00e750701825c256395f6aa7b23256eb27609b51779b86aed108b6eaa3912181a9d8282e23eebf9cee7784f9fabe0
4e9efac678 test: Check wallet name in -walletnotify script (João Barbosa)
9a5b5ee81f wallet: Replace %w by wallet name in -walletnotify script (João Barbosa)
Pull request description:
Fixes#13237.
ACKs for top commit:
laanwj:
ACK 4e9efac678
Tree-SHA512: 189dd1c785485f2e974d7c12531851b2a977778b3b954aa95efd527322ba3345924cfd587fb9c90b0fa979202af0ab2d90e53d125fe266a36c94f757e4176203
fad027fb0c scripted-diff: Add missing spaces in RPCResult, Fix type names (MarcoFalke)
Pull request description:
This makes the rendered diff smaller when the RPCResult is machine generated later on (Previous attempts: #14601 and #14459)
ACKs for top commit:
Sjors:
ACK fad027fb0c
Tree-SHA512: 48afd571b1cd349ca0b29bb444c1c7cda657e07dd96c610d479f931ccd938186aec98e533d0552b5b10afc9a3d7b911359260a49448e8e1106e3647b2c71f3ba
1b068c50dd tests: Add --valgrind option to test/fuzz/test_runner.py for running fuzzing test cases under valgrind (practicalswift)
Pull request description:
Add `--valgrind` option to `test/fuzz/test_runner.py` for running fuzzing test cases under `valgrind`.
Test this PR using:
```
$ make distclean
$ ./autogen.sh
$ CC=clang CXX=clang++ ./configure --enable-fuzz --with-sanitizers=fuzzer
$ make
$ git clone https://github.com/bitcoin-core/qa-assets
$ test/fuzz/test_runner.py --valgrind -l DEBUG qa-assets/fuzz_seed_corpus/
```
ACKs for top commit:
MarcoFalke:
ACK 1b068c50dd🌒
Tree-SHA512: e6eb99af1bceaa6f36f49092a05de415848099ccc1497cc098a62e925954c978cb37a46410b44ed5eef2c6464ca4ecb06397b75b5d35701f5a8525436e47b9fd
c98c26ee99 ci: use --enable-werror on more hosts (Sjors Provoost)
6ba617dbe2 build: add Wreturn-type to Werror flags (Sjors Provoost)
Pull request description:
I overlooked a missing `return false` in https://github.com/bitcoin/bitcoin/pull/17577#discussion_r379137776 and the warning only showed up on one Travis machine (`warning: control reaches end of non-void function [-Wreturn-type]`).
This PR promotes `Wreturn-type` to an error when configured with `--enable-werror`. I also added `--enable-werror` to the Travis machine that happened to catch this particular instance.
ACKs for top commit:
vasild:
ACK c98c26e.
practicalswift:
ACK c98c26ee99
Tree-SHA512: 64e86c67fef2c5048aab201a8400b7e4a6f27b93d626159ba0b2807b5f119d2b0a83e3372db88f692cb4b0d059722d6a642d130c74a4f991a27f3a6b21780b5f
