6544ea5035 refactor: Block unsafe fs::path std::string conversion calls (Russell Yanofsky)
b39a477ec6 refactor: Add fs::PathToString, fs::PathFromString, u8string, u8path functions (Russell Yanofsky)
Pull request description:
The `fs::path` class has a `std::string` constructor which will implicitly convert from strings. Implicit conversions like this are not great in general because they can hide complexity and inefficiencies in the code, but this case is especially bad, because after the transition from `boost::filesystem` to `std::filesystem` in #20744 the behavior of this constructor on windows will be more complicated and can mangle path strings. The `fs::path` class also has a `.string()` method which is inverse of the constructor and has the same problems.
Fix this by replacing the unsafe method calls with `PathToString` and `PathFromString` function calls, and by forbidding unsafe method calls in the future.
ACKs for top commit:
kiminuo:
ACK 6544ea5035
laanwj:
Code review ACK 6544ea5035
hebasto:
re-ACK 6544ea5035, only added `fsbridge_stem` test case, updated comment, and rebased since my [previous](https://github.com/bitcoin/bitcoin/pull/22937#pullrequestreview-765503126) review. Verified with the following command:
Tree-SHA512: c36324740eb4ee55151146626166c00d5ccc4b6f3df777e75c112bcb4d1db436c1d9cc8c29a1e7fb96051457d317961ab42e6c380c3be2771d135771b2b49fa0
6531599f42 test: Add check that newkeypool flushes change addresses too (Samuel Dobson)
84fa19c77a Add release notes for keypool flush changes (Samuel Dobson)
f9603ee4e0 Add test for flushing keypool with newkeypool (Samuel Dobson)
6f6f7bb36c Make legacy wallet upgrades from non-HD to HD always flush the keypool (Samuel Dobson)
2434b10781 Fix outdated keypool size default (Samuel Dobson)
22cc797ca5 Add newkeypool RPC to flush the keypool (Samuel Dobson)
Pull request description:
This PR makes two main changes:
1) Adds a new RPC `newkeypool` which will entirely flush and refill the keypool.
2) When upgradewallet is called on old, non-HD wallets upgrading them to HD, we now always flush the keypool and generate a new one, to immediately start using the HD generated keys.
This PR is motivated by a number of users with old, pre-compressed-key wallets upgrading them and being confused about why they still can't generate p2sh-segwit or bech32 addresses -- this is due to uncompressed keys remaining in the keypool post-upgrade and being illegal in these newer address formats. There is currently no easy way to flush the keypool other than to call `getnewaddress` a hundred/thousand times or an ugly hack of using a `sethdseed` call.
ACKs for top commit:
laanwj:
re-ACK 6531599f42
meshcollider:
Added new commit 6531599f42 to avoid invalidating previous ACKs.
instagibbs:
ACK 6531599f42
Tree-SHA512: 50c79c5d42dd27ab0ecdbfdc4071fdaa1b2dbb2f9195ed325b007106ff19226419ce57fe5b1539c0c24101b12f5e034bbcfb7bbb0451b766cb1071295383d774
aa69fd6caf build: Drop -Wno-unused-local-typedef (Hennadii Stepanov)
672e8c5d07 build: remove -Wunused-variable (fanquake)
5239af0574 build: remove -Wswitch (fanquake)
0375906e0a build: use loop-analysis over range-loop-analysis (fanquake)
12712fa2c4 build: remove -Wsign-compare (fanquake)
Pull request description:
This remove the addition of flags that are already part of other options, such as `-Wall` or `-Wextra`; see each commit message for details. All of the flags being removed here already exist as part of `-Wall` as of GCC 8, or, for Clang, all exist in `-Wmost` (included in `-Wall)`, or as part of `-Wextra` as of Clang 7. Both of which are our minimum required compilers.
Also cherry-picks one change from #21458.
To give an example of how GCCs `-Wall` has changed over the last few releases:
### 11.x to trunk (12.x)
Added:
```bash
-Wzero-length-bounds
-Wmismatched-dealloc
-Wmismatched-new-delete (only for C/C++)
```
### 10.x to 11.x
Added:
```bash
-Warray-parameter=2 (C and Objective-C only)
-Wrange-loop-construct (only for C++)
-Wsizeof-array-div
-Wvla-parameter (C and Objective-C only)
```
Removed:
```bash
-Wenum-conversion in C/ObjC;
```
### 9.x to 10.x
Added:
```bash
-Wenum-conversion in C/ObjC;
-Wformat-overflow
-Wformat-truncation
-Wzero-length-bounds
```
### 8.x to 9.x
Added:
```bash
-Wpessimizing-move
```
Removed:
```bash
-Wstringop-truncation
```
### 7.x to 8.x
Added:
```bash
-Wcatch-value (C++ and Objective-C++ only)
-Wmissing-attributes
-Wmultistatement-macros
-Wrestrict
-Wsizeof-pointer-div
-Wstringop-truncation
```
[Clang Warning Options](https://clang.llvm.org/docs/DiagnosticsReference.html)
[GCC Warning Options](https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html)
ACKs for top commit:
meshcollider:
utACK aa69fd6caf
Tree-SHA512: 34dde6bd773c864202c151eaa35f902d03fb531c27fe5e1ef659225da03acade2efe5df56df3efb4df5bbded3d395348ce03c25b837fce83be53af3352f0f2bc
b5950dd59c validation: put coins cache write log into bench debug log (Anthony Towns)
31b2b802b5 blockstorage: use debug log category (Anthony Towns)
da94ebc2fa validation: move header validation error logging to VALIDATION debug category (Anthony Towns)
1d7d835ec3 validation: include block hash when reporting prev block not found errors (Anthony Towns)
Pull request description:
Moves the following log messages into debug log categories:
* "AcceptBlockHeader: ..." to validation
* "Prune: deleted blk/rev" to new blockstorage log category
* "Leaving block file" moves from validation to blockstorage
* "write coins cache to disk" to bench
Also adds the hash of the block to the log message when AcceptBlockHeader is rejecting because of problems with the prev block.
ACKs for top commit:
practicalswift:
cr ACK b5950dd59c
Empact:
Code review ACK b5950dd59c
laanwj:
Code review ACK b5950dd59c
promag:
Code review ACK b5950dd59c.
meshcollider:
Code review ACK b5950dd59c
Tree-SHA512: a73fdbfe8d36da48a3e89c2d5e0b6a3c5045d280c1a57f61c38d0d21f4f198aece4bd85155be3439e179d5dabdb523bf15fa0395e0e3ceff19c878ba3112c840
1527b7e8a1 symbol-check: Check requested ELF interpreter (Carl Dong)
b96adcbfae guix: Fix powerpc64(le) dynamic linker name (Carl Dong)
Pull request description:
Fixes https://github.com/bitcoin/bitcoin/issues/23111
It would seem that I got the wrong default glibc-dynamic-linker path for PowerPC platforms. This means that for our currently released v22.0 binaries to be run on powerpc platforms, users would have to either:
1. Move `/lib64/ld64.so.?` to `/lib`, or
2. Invoke their linker-loader directly to start our binaries, e.g. `/lib64/ld64.so.? bitcoind`
This is my bad.
I've fixed the paths in this patchset, and also added a test to `symbol-check.py` so that this does not ever slip past our checks again.
ACKs for top commit:
laanwj:
Code review ACK 1527b7e8a1
Tree-SHA512: bc520c35f72a9d4a3804b53d211138724560bd2405bf2f592ef755d19073e72f114fc4b8a3747e0c8724ac46a60b6ca86ea7766d66acb88eed1ebe2abc2678b8
It is important that binaries request a standard interpreter location
where most distros would place the linker-loader. Otherwise, the user
would be met with a very confusing message:
bash: <path>/<to>/bitcoind: No such file or directory
When really it's the interpreter that's not found.
I used Guix's values for the powerpc64(le) dynamic linkers, and the
/lib-prefix seems to be a Guix-ism rather than standard. The standard
path for the linker-loaders start with /lib64.
I've taken the new loader values from SYSDEP_KNOWN_INTERPRETER_NAMES in
glibc's sysdeps/unix/sysv/linux/powerpc/ldconfig.h file.
For future reference, loader path values can also be found on glibc's
website: https://sourceware.org/glibc/wiki/ABIList?action=recall&rev=16
ce69e18947 scripts: remove pixie.py (fanquake)
00b85d0b13 scripts: only parse the binary once in security-check.py (fanquake)
cad40a5b16 scripts: use LIEF for ELF checks in security-check.py (fanquake)
8242ae230e scripts: only parse the binary once in symbol-check.py (fanquake)
309eac9019 scripts: use LIEF for ELF checks in symbol-check.py (fanquake)
610a8a8e39 test-*-check: Pass in *FLAGS and compile with them (Carl Dong)
Pull request description:
This finishes the transition to using LIEF for the ELF symbol and security checks.
Note that there's currently a work around used for identifying RISCV binaries (just checking the interpreter). I've sent a PR upstream, https://github.com/lief-project/LIEF/pull/562, and we should be able to drop that when using LIEF 0.12.0 and onwards.
ACKs for top commit:
dongcarl:
Code Review ACK ce69e18947
laanwj:
Code review ACK ce69e18947
Tree-SHA512: 911ba693cd9777ad1fc1f66dff6c4d3630a907351215380cbde5b14a4bbf5cf7eebf52eafa7e86b27deabd2d93d1b403f34aabd356b5ceaab3cc6e9941a01dd4
fa6f29de51 bitcoin-tx: Reject non-integral and out of range multisig numbers (MarcoFalke)
fafab8ea5e bitcoin-tx: Reject non-integral and out of range sequence ids (MarcoFalke)
fa53d3d826 test: Check that bitcoin-tx accepts whitespace around sequence id and multisig numbers (MarcoFalke)
Pull request description:
Seems odd to silently accept arbitrary strings that don't even represent integral values.
Fix that.
ACKs for top commit:
practicalswift:
cr ACK fa6f29de51
laanwj:
Code review ACK fa6f29de51
Empact:
Code review ACK fa6f29de51
promag:
Code review ACK fa6f29de51.
Tree-SHA512: e31f7f21fe55ac069e755557bdbcae8d5d29e20ff82e441ebdfc65153e3a31a4edd46ad3e6dea5190ecbd1b8ea5a8f94daa5d59a3b7558e46e794e30db0e6c79
7e88f61b28 multiprocess: Make interfaces::Chain::isTaprootActive non-const (Russell Yanofsky)
Pull request description:
`interfaces::Chain` is an abstract class, so declaring the method const would be exposing internal implementation details of subclasses to interface callers. And specifically this doesn't work because the multiprocess implementation of the `interfaces::Chain::isTaprootActive` method can't be const because IPC connection state and request state is not constant during the call.
---
This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/projects/10).
ACKs for top commit:
jamesob:
ACK 7e88f61b28
Tree-SHA512: 1c5ed89870aeb7170b9048c41299ab650dfa3d0978088e08c4c866fa0babb292722710b16f25540f26667220cb4747b1c256c4bd42893c552291eccc155346a3
38fd709fa5 build: make --enable-werror just -Werror (fanquake)
Pull request description:
No longer special case a set of warnings, to make up our own -Werror,
just use -Werror outright. This shouldn't really have any effect on
existing builders, who were already using `--enable-werror`, and is more
inline with what they would expect `--enable-werror` to be, which is
erroring on any/all warnings.
We keep `-Wno-error=return-type` because we know that is broken when using
mingw-w64. It should only be applied when cross-compiling for Windows.
Similar to the change in #20544, but with (hopefully) less work-arounds,
and other bundled changes. A step towards some configure "cleanups".
ACKs for top commit:
hebasto:
ACK 38fd709fa5 (also see https://github.com/bitcoin/bitcoin/pull/23149#issuecomment-940420776), tested:
MarcoFalke:
Concept ACK 38fd709fa5
Tree-SHA512: 37f1857d9408442cab63e40f9280427b73e09cdf03146b19c1339d7e44abd78e93df7f270ca1da0e83b79343cd3ea915f7b9e4e347488b5bc5ceaaa7540e5926
fa8d492894 rest: Return error when header count is not integral (MarcoFalke)
Pull request description:
Seems odd to interpret a hash (or any other string) as integer when it contains more than the digits 0 to 9.
ACKs for top commit:
practicalswift:
cr ACK fa8d492894
promag:
Code review ACK fa8d492894.
shaavan:
Code Review ACK fa8d492894
Tree-SHA512: d6335b132ca2010fb8cae311dd936b2dea99a5bd0e6b2556a604f93698b8456df9190c5151345a03344243ede4aad0e2526cedc2aa8b5b1b8e8ce786cb3b6e50
fa4d0aacf2 test: * -> & (MarcoFalke)
Pull request description:
This changes background_cs from being a pointer to a reference to work
around a gcc false warning. Also, this makes the test easier to read.
Fixes bitcoin#23101
Can be reviewed with --ignore-all-space.
ACKs for top commit:
practicalswift:
cr ACK fa4d0aacf2
jamesob:
ACK fa4d0aacf2
hebasto:
ACK fa4d0aacf2, tested on Linux Mint 20.2 (x86_64) by merging this PR on top of the current master.
Tree-SHA512: 93a0d8859201f7074bea52fab8f6701409148bc50cfbb142cacfa6c991fc12c07584df04fead645f11703883df99535423d154f9945202e1c5aff49540d9b607
fa43e7c2d9 bitcoin-tx: Avoid treating overflow as OP_0 (MarcoFalke)
fa053c0019 style: Fix whitespace in Parse* functions (MarcoFalke)
fa03dec7e9 refactor: Use C++11 range based for loop in ParseScript (MarcoFalke)
fad55e79ca doc: Fixup ToIntegral docs (MarcoFalke)
Pull request description:
Seems odd to treat integer overflow as `OP_0`, so fix that.
ACKs for top commit:
theStack:
re-ACK fa43e7c2d9
shaavan:
ACK fa43e7c2d9
Tree-SHA512: 1bbe2de62d853badc18d57d169c6e78ddcdff037e5a85357995dead11c8e67a4fe35087e08a181c60753f8ce91058b7fcc06f5b7901afedc78fbacea8bc3ef4f
These test-*-check scripts should compile "test" binaries in a way that
is as close to what autotools would do, since the goal is to make sure
that if we run the *-check script, they can correctly detect flaws in
binaries which are compiled by our autotools-based system.
Therefore, we should emulate what happens when the binary is linked in
autotools, meaning that for C binaries, we need to supply the CFLAGS,
CPPFLAGS, and LDFLAGS flags in that order.
Note to future developers: perhaps it'd be nice to have these
test-*-check scripts be part of configure.ac to avoid having to manually
replicate autoconf-like behaviour every time we find a discrepancy. Of
course, that would also mean you'd have to write more m4...
c2fbdca549 Add BECH32_INVALID_VERSION test (lsilva01)
b142f79ddb skip test_getaddressinfo() if wallet is disabled (lsilva01)
Pull request description:
Most of `test/functional/rpc_invalid_address_message.py` does not requires wallet.
But if the project is compiled in disable-wallet mode, the entire test will be skipped.
This PR changes the test to run the RPC tests first and then checks if the wallet is compiled.
ACKs for top commit:
stratospher:
tested ACK c2fbdca
Tree-SHA512: 11fa2fedf4a15aa45e3f12490df8e22290a867d5de594247211499533c32289c68c0b60bd42dbf8305e43dbcc042789e7139317ef5c9f8cf386f2d84c91b9ac2
0bc666b053 doc: add info for debugging with relative paths (S3RK)
a8b515c317 configure: keep relative paths in debug info (S3RK)
Pull request description:
This is a follow-up for #20353 that fixes#21885
It also adds a small section to assist debugging without absolute paths in debug info.
ACKs for top commit:
kallewoof:
Tested ACK 0bc666b053
Zero-1729:
Light crACK 0bc666b053
Tree-SHA512: d4b75183c3d3a0f59fe786841fb230581de87f6fe04cf7224e4b89c520d45513ba729d4ad8c0e62dd1dbaaa7a25741f04d036bc047f92842e76c9cc31ea47fb2
7fc487afd1 refactor: use `{Read,Write}BE32` helpers for BIP32 nChild (de)serialization (Sebastian Falbesoner)
Pull request description:
This small refactoring PR replaces manual bit-fiddling (de)serialization of the BIP32 child number (nChild) by the helpers `ReadBE32`/`WriteBE32`. Note that those were first introduced in #4100, almost one year _after_ the BIP32 derivation implementation has been merged (#2829, eb2c9990).
ACKs for top commit:
sipa:
utACK 7fc487afd1
laanwj:
Code review ACK 7fc487afd1
Tree-SHA512: bbe3e411fb0429fa74c8a5705a91f4d6ed704dac9d6623ecb633563f22acf8e21f3189a16f1d0cf1aeedfc56a5b695df54ae51e9577e34eb6d7dc335de2da6de
fa165e9545 Replace stoul with ToIntegral in dbwrapper (MarcoFalke)
Pull request description:
The string is created with `%llu`. See: 7fcf53f7b4/src/leveldb/db/db_impl.cc (L1436-L1437)
So it seems odd to silently accept when parsing: whitespace, a sign character, trailing chars, overflow, ....
Fix that by using the stricter ToIntegral.
ACKs for top commit:
laanwj:
Code review ACK fa165e9545
practicalswift:
cr ACK fa165e9545
theStack:
Code-review ACK fa165e9545
Tree-SHA512: b87f01431ca0b971ff84610022da8679d3c33470b88cfc3f4a337e6e176a0455715588aefd40e8e2bbe7459d902dc89d7bfe34e7fd66755f631cc18dc039fa2f
747cd17404 build: no-longer fail default configure if BDB isn't available (fanquake)
Pull request description:
Inline with moving to descriptor (sqlite) wallets by default for 0.23,
this adapts the build system so that a default `./configure` invocation
no-longer fails if BDB isn't present. Currently, if configure is run
with no options, and no BDB is present, we'll fail with:
```bash
checking for Berkeley DB C++ headers... no
configure: error: libdb_cxx headers missing, Bitcoin Core requires this library for BDB wallet support (--without-bdb to disable BDB wallet support)
```
If descriptor wallets are to be the default, this behaviour no longer
makes sense, as a builder should be able to configure and build, to use
a wallet, without BDB installed, and without passing additional
arguments, i.e `--without-bdb` or `--with-incompatible-bdb`, to
configure.
With this change, running configure will no-longer fail, but will
instead print:
```bash
checking for Berkeley DB C++ headers... no
configure: WARNING: libdb_cxx headers missing
configure: WARNING: Bitcoin Core requires this library for BDB (legacy) wallet support
configure: WARNING: Passing --without-bdb will suppress this warning
checking for sqlite3 >= 3.7.17... yes
checking whether to build wallet with support for sqlite... yes
```
ACKs for top commit:
hebasto:
ACK 747cd17404, tested on Linux Mint 20.2 (x86_64) with the (un)installed system packages `libdb-dev` and `libdb++-dev`.
Tree-SHA512: ae316d71ad0803c9d4b02a5fedcade08242650d987cc047840493ba4a881e71ff48b099075bb7c325307d44744fcdeccb57f7fa8db4135c81a5835841f562afa
d54ec27bac qt: Add helper to load font (João Barbosa)
Pull request description:
Originally submitted as https://github.com/bitcoin-core/gui-qml/pull/49.
ACKs for top commit:
hebasto:
re-ACK d54ec27bac
stratospher:
Tested ACK d54ec27. Refactoring the code and defining `loadFont()` in `src/qt/guiutil.cpp` reduces redundant imports of the `QFontDatabase` and is a better design.
shaavan:
ACK d54ec27bac
Tree-SHA512: b156bb6ffb08dd57476f383a29bbb0a1108b62794d430debb77252f7d09df1409a7532b09d17d8836d1c2ab7c126a6618231164b9d0def1b8f361a81ef22d107
44452110f0 [fuzz] Update comment in FillAddrman() (John Newbery)
640476eb0e [fuzz] Make Fill() a free function in fuzz/addrman.cpp (John Newbery)
90ad8ad61a [fuzz] Make RandAddr() a free function in fuzz/addrman.cpp (John Newbery)
491975c596 [fuzz] Pass FuzzedDataProvider& into Fill() in addrman fuzz tests (John Newbery)
56303e382e [fuzz] Create a FastRandomContext in addrman fuzz tests (John Newbery)
Pull request description:
#22974 improved the performance of `CAddrMan::Good()` significantly so that it could be used directly in the fuzz tests, instead of those tests reaching directly into addrman's internal data structures.
This PR continues the work of making the fuzz tests only use `CAddrMan`'s public methods and pulls the `Fill()` and `RandAddr()` methods from `CAddrManDeterministic` into free functions, since they no longer need access to `CAddrMan` internals.
ACKs for top commit:
theuni:
utACK 44452110f0. Agree the failure seems unrelated, looks like some startup race.
mzumsande:
ACK 44452110f0
vasild:
ACK 44452110f0
Tree-SHA512: fcf994e1dedd0012b77f632720b6423d51ceda4eb85c9efe572f2a1150117f9e511114a5206738dd94409137287577f3b01a9998f5237de845410d3d96e7cb7f
