e6a94d4446 guix: Bump to version-1.3.0 from upstream (Carl Dong)
90fd13b954 guix: Pin kernel header version (Carl Dong)
Pull request description:
```
- Use 4.19 for riscv64 (earliest LTS release w/ riscv64 support)
- Use 4.9 for all others (second-oldest LTS release, released in
combination with glibc glibc 2.24 in Debian stretch)
```
```
The chosen commit is the HEAD of Guix's version-1.3.0 branch as of July
15th, 2021.
Also fix visual indenting.
```
-----
This + the documentation PR should make our Guix system ready for release!
ACKs for top commit:
MarcoFalke:
review ACK e6a94d4446 to change to vanilla guix. Did not review the kernel change.
laanwj:
ACK e6a94d4446
fanquake:
ACK e6a94d4446
Tree-SHA512: a175e4ddb3ee786a39f5e800ce336932ad2f6797a3a28400a6f723875d0f19833fd36cedc41b3580e4604110517211bd9f557be36adf7265fd8e591c434ae032
0a5723beea macdeploy: cleanup .temp.dmg if present (fanquake)
ecffe8689d macdeploy: remove qt4 related code (fanquake)
639f064253 macdeploy: select the plugins we need, rather than excluding those we don't (fanquake)
3d26b6b9e9 macdeploy: fix framework printing when passing -verbose (fanquake)
dca6c90329 macdeploy: remove unused plistlib import (fanquake)
Pull request description:
This includes [one followup](https://github.com/bitcoin/bitcoin/pull/20422#discussion_r534207899) and [one bug fix](3d26b6b9e9) from #20422, as well as some simplifications to the `macdeployqtplus` code.
ACKs for top commit:
hebasto:
ACK 0a5723beea, tested on macOS Big Sur 11.4 (20F71, x86_64) + Homebrew's Qt 5.15.2.
Tree-SHA512: cfad9505eacd32fe3a9d06eb13b2de0b6d2cad7b17778e90b503501cbf922e53d4e7f7f74952d1aed58410bdae9b0bb3248098583ef5b85689cb27d4dc06c029
fac4814106 doc/release-process: Add torrent creation details (Carl Dong)
5d24cc3d82 guix/INSTALL: Guix installs init scripts in libdir (Carl Dong)
5da2ee49d5 guix/INSTALL: Add coreutils/inotify-dir-recreate troubleshooting (Carl Dong)
318c60700b guix: Adapt release-process.md to new Guix process (Carl Dong)
fcab35b229 guix-attest: Produce and sign normalized documents (Carl Dong)
c2541fd0ca guix: Overhaul README (Carl Dong)
46ce6ce378 tree-wide: Rename gitian-keys to builder-keys (Carl Dong)
fc4f8449f3 guix: Update various check_tools lists (Carl Dong)
263220a85c guix: Check for a sane services database (Carl Dong)
Pull request description:
Based on: #21462
Keeping the README in one file so that it's easy to search through. Will add more jumping links later so navigation is easier.
Current TODOs:
- [x] Shell installer option: prompt user to re-login for `/etc/profile.d` entry to be picked up
- [x] Binary tarball option: prompt user to create `/etc/profile.d` entry and re-login
- [x] Fanquake docker option: complete section
- [x] Arch Linux AUR option: prompt to start `guix-daemon-latest` unit after finishing "optional setup" section
- [x] Building from source option: Insert dependency tree diagram that I made
- [x] Building from source option: redo sectioning, kind of a mess right now
- [x] Optional setup: make clear which parts are only needed if building from source
- [x] Workaround 1 for GnuTLS: perhaps mention how to remove Guix build farm's key
- [x] Overall (after everything): Make the links work.
Note to self: wherever possible, tell user how to check that something is true rather than branching by installation option.
ACKs for top commit:
fanquake:
ACK fac4814106 - going to go ahead and merge this now. It's a lot of documentation, and could probably be nit-picked / improved further, however, that can continue over the next few weeks. I'm sure more (backportable) improvements / clarifications will be made while we progress through RCs towards a new release.
Tree-SHA512: dc46c0ecdfc67c7c7743ca26e4a603eb3f54adbf81be2f4c1f4c20577ebb84b5250b9c9ec89c0e9860337ab1c7cff94d7963c603287267deecfe1cd987fa070a
facd56750c scripted-diff: Revert "fuzz: Add Temporary debug assert for oss-fuzz issue" (MarcoFalke)
Pull request description:
No longer needed, as it wouldn't help to debug this issue. See https://github.com/bitcoin/bitcoin/pull/22472#issuecomment-882692900
ACKs for top commit:
fanquake:
ACK facd56750c
Tree-SHA512: 13352b3529c43d6e65ab127134b32158d3072dc2fbbb326fea9adfeada5a8610d0477ea75748b8b68e7abb3b9869a989df3a3169e92bdd458053d64bae6ed379
That way we can easily combine the document and detached signature to
produce cleartext signature files for upload during the release process.
See subsequent commits which modify doc/release-process.md for more
details.
Adds a test for the condition which can trigger a lock order assertion.
Specifically, there must be an unconfirmed transaction in the mempool
which belongs to the wallet being loaded. This will establish the order
of cs_wallet -> cs_main -> cs_KeyStore. Then dumpwallet is called on
that wallet. Previously, this would have used a lock order of cs_wallet
-> cs_KeyStore -> cs_main, but this should be fixed now. The test
ensures that.
DEBUG_LOCKORDER expects cs_wallet, cs_main, and cs_KeyStore to be
acquired in that order. However dumpwallet would take these in the order
cs_wallet, cs_KeyStore, cs_main. So when configured with
`--enable-debug`, it is possible to hit the lock order assertion when
using dumpwallet.
To fix this, cs_wallet and cs_KeyStore are no longer locked at the same
time. Instead cs_wallet will be locked first. Then the functions which
lock cs_main will be run. Lastly cs_KeyStore will be locked afterwards.
This avoids the lock order issue.
Furthermore, since GetKeyBirthTimes (only used by dumpwallet) also uses
a function that locks cs_main, and itself also locks cs_KeyStore, the
same reordering is done here.
816f29eab2 addrman: detect on-disk corrupted nNew and nTried during unserialization (Vasil Dimov)
Pull request description:
Negative `nNew` or `nTried` are not possible during normal operation.
So, if we read such values during unserialize, report addrman
corruption.
Fixes https://github.com/bitcoin/bitcoin/issues/22450
ACKs for top commit:
MarcoFalke:
cr ACK 816f29eab2
jonatack:
ACK 816f29eab2
lsilva01:
Code Review ACK 816f29eab2. This change provides a more accurate description of the error.
Tree-SHA512: 01bdd72d2d86a0ef770a319fee995fd1e147b24a8db84ddb8cd121688e7f94fed73fddc0084758e7183c4f8d08e971f0b1b224f5adb10928a5aa4dbbc8709d74
a4bcd687c9 Improve tests using statistics (John Newbery)
f424d601e1 Add logging and addr rate limiting statistics (Pieter Wuille)
b4ece8a1cd Functional tests for addr rate limiting (Pieter Wuille)
5648138f59 Randomize the order of addr processing (Pieter Wuille)
0d64b8f709 Rate limit the processing of incoming addr messages (Pieter Wuille)
Pull request description:
The rate at which IP addresses are rumoured (through ADDR and ADDRV2 messages) on the network seems to vary from 0 for some non-participating nodes, to 0.005-0.025 addr/s for recent Bitcoin Core nodes. However, the current codebase will happily accept and process an effectively unbounded rate from attackers. There are measures to limit the influence attackers can have on the addrman database (bucket restrictions based on source IPs), but still - there is no need to permit them to feed us addresses at a rate that's orders of magnitude larger than what is common on the network today, especially as it will cause us to spam our peers too.
This PR implements a [token bucket](https://en.wikipedia.org/wiki/Token_bucket) based rate limiter, allowing an average of 0.1 addr/s per connection, with bursts up to 1000 addresses at once. Whitelisted peers as well as responses to GETADDR requests are exempt from the limit. New connections start with 1 token, so as to not interfere with the common practice of peers' self-announcement.
ACKs for top commit:
laanwj:
ACK a4bcd687c9
vasild:
ACK a4bcd687c9
jnewbery:
ACK a4bcd687c9
jonatack:
ACK a4bcd687c9
Tree-SHA512: b757de76ad78a53035b622944c4213b29b3b55d3d98bf23585afa84bfba10808299d858649f92269a16abfa75eb4366ea047eae3216f7e2f6d3c455782a16bea
8465978f23 Make IsSegWitOutput return true for taproot outputs (Pieter Wuille)
Pull request description:
This fixes a bug: currently `utxoupdatepsbt` will not fill in UTXO data for PSBTs spending taproot outputs.
ACKs for top commit:
achow101:
Code Review ACK 8465978f23
jonatack:
ACK 8465978f23
meshcollider:
utACK 8465978f23
Tree-SHA512: 2f8f873450bef4b5a4ce5962a231297b386c6b1445e69ce5f36ab28eca7343be3a11bc09c38534b0f75e6f99ba15d78d3ba5d484f6c63e5a9775e1f3f55a74e0
a2aca207b1 Move implementations of non-template fuzz helpers (Sriram)
Pull request description:
There are 78 cpp files that include `util.h` (`grep -iIr "#include <test/fuzz/util.h>" src/test/fuzz | wc -l`). Modifying the implementation of a fuzz helper in `src/test/fuzz/util.h` will cause all fuzz tests to be recompiled. Keeping the declarations of these non-template fuzz helpers in `util.h` and moving their implementations to `util.cpp` will skip the redundant recompilation of all the fuzz tests, and builds these helpers only once in `util.cpp`.
Functions moved from `util.h` to `util.cpp`:
- `ConsumeTxMemPoolEntry`
- `ContainsSpentInput`
- `ConsumeNetAddr`
- Methods of `FuzzedFileProvider::(open, read, write, seek, close)`
ACKs for top commit:
MarcoFalke:
review ACK a2aca207b1🍂
Tree-SHA512: e7037ebb86d0fc56048e4f3d8733eefc21da11683b09d2b22926bda410719628d89c52ddd9b4c18aa243607a66fdb4d13a63e62ca010e66b3ec9174fd18107f0
5012a7912e Test that descriptor wallet upgrade does nothing (Andrew Chow)
48bd7d3b77 Change ScriptPubKeyMan::Upgrade to default to return true (Andrew Chow)
Pull request description:
When adding a new ScriptPubKeyMan, it's likely that there will be nothing for `Upgrade` to do. If it is called (via `upgradewallet`), then it should do nothing, successfully. This PR changes the default `ScriptPubKeyMan::Upgrade` function so that it returns a success instead of failure when doing nothing.
Fixes#22460
ACKs for top commit:
jonatack:
ACK 5012a7912e
meshcollider:
utACK 5012a7912e
Tree-SHA512: 578c6521e997f7bb5cc44be2cfe9e0a760b6bd4aa301026a6b8b3282e8757473e4cb9f68b2e79dacdc2b42dddae718450072e0a38817df205dfea177a74d7f3d
1edddf5de4 Avoid GCC 7.1 ABI change warning in guix build (Pieter Wuille)
Pull request description:
The arm-linux-gnueabihf guix build output is littered with warnings like:
```
/gnu/store/7a96hdqdb2qi8a39f09n84xjy2hr23rs-gcc-cross-arm-linux-gnueabihf-8.4.0/include/c++/bits/stl_vector.h:1085:4: note:
parameter passing for argument of type '__gnu_cxx::__normal_iterator<CRecipient*, std::vector<CRecipient> >' changed in GCC 7.1
```
These are irrelevant for us. Disable them using `-Wno-psabi`.
ACKs for top commit:
laanwj:
ACK 1edddf5de4
hebasto:
ACK 1edddf5de4, after thorough reading related materials, I agree this change can be merged. As I mentioned above, I have been compiling my arm-32bit binaries with `-Wno-psabi` flag for two years, and no related flaws were observed.
Tree-SHA512: 485c7500547ac5da567ad23847341c18ff832607f5a1002676404cc647e437cf3445b6894ecff5b52929ca52bea946c06bd90eace1997c895e56204e787065e4
1155978d8f build, qt: Do not install *.prl files (Hennadii Stepanov)
763793b60e build, qt: Fix wrong cross-compiling detection on macOS (Hennadii Stepanov)
30982721ab build, qt: Force bootstrap while building linguist tools (Hennadii Stepanov)
689320e307 build, qt: Drop translations.pro hack (Hennadii Stepanov)
6a1f98f253 build, qt: Drop lrelease dependency patch (Hennadii Stepanov)
39e561e087 build, qt: Add linguist_tools list (Hennadii Stepanov)
27d3def1c6 build: Use Qt top-level build facilities (Hennadii Stepanov)
Pull request description:
This PR:
- uses Qt top-level build facilities without the need to download all-in-one archive
- is based on **BlockMechanic**'s [idea](https://github.com/bitcoin/bitcoin/pull/20600), and is an alternative to #20600
- makes it easy to integrate [new modules](https://github.com/bitcoin/bitcoin/pull/16883) into static builds
- has the minimal diff
- makes the qt package build process streamlined by dropping some patches and hacks (an alternative to #21420 and #20642)
Fixes#18536 (a non-intrusive alternative to #21589 and #19785).
Fixes#14648.
Fixes#21588 (a non-intrusive alternative to #21591).
Required for adding [Wayland support](https://github.com/bitcoin/bitcoin/issues/19950) on Linux.
---
**Note for reviewers**: With 9046de8a4cbc3899fed9eae084115f423e7ac5bd from #21995 it is easy to verify that there are no changes in the resulted `qt` package archive on the per commit basis. For example, for `HOST=i686-pc-linux-gnu` no commit in this PR introduces any changes.
ACKs for top commit:
fanquake:
ACK 1155978d8f
Tree-SHA512: 667b06b72cb7ff26d68b9b88e8dddb51084783ca9e3d80b3392710794c1dc7fd77bbcc3ccf4ccece9919d33b9bf8fce13c5059502bd228021dc7c93fdb87ca7a
fb7be92b09 Mark print-% target as phony. (Dmitry Goncharov)
Pull request description:
.PHONY does not take patterns (such as print-%) as prerequisites.
Have print-% depend on force and mark force as phony.
This change ensures print-% rule works even when there is a file that matches the target.
```
$ # on master
$ make print-host
host=x86_64-pc-linux-gnu
$ touch print-host
$ make print-host
make: 'print-host' is up to date.
$
$ git co mark_print_as_phony
Switched to branch 'mark_print_as_phony'
$ make print-host
host=x86_64-pc-linux-gnu
$ touch force
$ make print-host
host=x86_64-pc-linux-gnu
```
ACKs for top commit:
hebasto:
ACK fb7be92b09, tested on Linux Mint 20.2 (x86_64).
Tree-SHA512: b89ae66aa8c7aa6a7ab5f0956f9eb3b3ef9d56994b60dc2a97d498d4c1bba537845c190723e8a10310280b1b35df2cd935cc30aeb76735cac2dc621ad7823772
3c4c8e79ba build: Add -Werror=implicit-fallthrough compile flag (Hennadii Stepanov)
014110c47d Use C++17 [[fallthrough]] attribute, and drop -Wno-implicit-fallthrough (Hennadii Stepanov)
Pull request description:
ACKs for top commit:
fanquake:
ACK 3c4c8e79ba - looks ok to me now. Checked that warnings occur in our code & leveldb by removing a `[[fallthrough]]` or `FALLTHROUGH_INTENDED`.
jarolrod:
ACK 3c4c8e79ba
theStack:
ACK 3c4c8e79ba
Tree-SHA512: 4dce91f0f26b8a3de09bd92bb3d7e1995e078e3a8b3ff861c4fbf6c0b32b2327d063633b07b89c4aa94a1141d7f78d46d9d43ab8df865273e342693ad30645b6
- Use 4.19 for riscv64 (earliest LTS release w/ riscv64 support)
- Use 4.9 for all others (second-oldest LTS release, released in
combination with glibc glibc 2.24 in Debian stretch)
While limitations on the influence of attackers on addrman already
exist (affected buckets are restricted to a subset based on incoming
IP / network group), there is no reason to permit them to let them
feed us addresses at more than a multiple of the normal network
rate.
This commit introduces a "token bucket" rate limiter for the
processing of addresses in incoming ADDR and ADDRV2 messages.
Every connection gets an associated token bucket. Processing an
address in an ADDR or ADDRV2 message from non-whitelisted peers
consumes a token from the bucket. If the bucket is empty, the
address is ignored (it is not forwarded or processed). The token
counter increases at a rate of 0.1 tokens per second, and will
accrue up to a maximum of 1000 tokens (the maximum we accept in a
single ADDR or ADDRV2). When a GETADDR is sent to a peer, it
immediately gets 1000 additional tokens, as we actively desire many
addresses from such peers (this may temporarily cause the token
count to exceed 1000).
The rate limit of 0.1 addr/s was chosen based on observation of
honest nodes on the network. Activity in general from most nodes
is either 0, or up to a maximum around 0.025 addr/s for recent
Bitcoin Core nodes. A few (self-identified, through subver) crawler
nodes occasionally exceed 0.1 addr/s.
If a ScriptPubKeyMan does not implement Upgrade, then using upgraewallet
will fail unexpectedly. By changing the default to return true, then
this error can be avoided. This is still correct because a successful
upgrade can be that nothing happened.
7593b06bd1 test: ensure I2P addresses are relayed (Vasil Dimov)
e7468139a1 test: make CAddress in functional tests comparable (Vasil Dimov)
33e211d2a4 test: implement ser/unser of I2P addresses in functional tests (Vasil Dimov)
86742811ce test: use NODE_* constants instead of magic numbers (Vasil Dimov)
ba45f02708 net: relay I2P addresses even if not reachable (by us) (Vasil Dimov)
Pull request description:
Nodes that can reach the I2P network (have set `-i2psam=`) will relay
I2P addresses even without this patch. However, nodes that can't reach
the I2P network will not. This was done as a precaution in
https://github.com/bitcoin/bitcoin/pull/20119 before anybody could
connect to I2P because then, for sure, it would have been useless.
Now, however, we have I2P support and a bunch of I2P nodes, so get all
nodes on the network to relay I2P addresses to help with propagation,
similarly to what we do with Tor addresses.
ACKs for top commit:
jonatack:
ACK 7593b06bd1
naumenkogs:
ACK 7593b06bd1.
laanwj:
Code review ACK 7593b06bd1
kristapsk:
ACK 7593b06bd1. Code looks correct, tested that functional test suite passes and also that `test/functional/p2p_addrv2_replay.py` fails if I undo changes in `IsRelayable()`.
Tree-SHA512: c9feec4a9546cc06bc2fec6d74f999a3c0abd3d15b7c421c21fcf2d610eb94611489e33d61bdcd5a4f42041a6d84aa892f7ae293b0d4f755309a8560b113b735
fa84caebc7 doc: added info to bitcoin.conf doc (Brian Liotti)
Pull request description:
Should probably be explicitly stated to not make modifications to the conf file while daemon is running. ref #11586
For example, if rpc credentials are modified while bitcoind is running, `bitcoin-cli stop` is unable to stop bitcoind until the original credentials are restored in `bitcoin.conf`
ACKs for top commit:
rajarshimaitra:
ACK fa84caebc7
Zero-1729:
ACK fa84caebc7
theStack:
ACK fa84caebc7🗄️
Tree-SHA512: f6ddffc25563c0b01e661b6abe43a7909938ad8eca38d6d0e2d4a2ce9fb850e51b54d950ef3118b74b6e340c64fe3f37205861720a2de2933db29782234869bb
a16378e501 doc: Remove unnecessary steps from translations update process (Wladimir J. van der Laan)
2584929823 doc: Add steps for transifex to release process (Wladimir J. van der Laan)
Pull request description:
Document how to update settings on and for the transifex website before and after branch-off of a new release.
(This is #21440, updated with the review feedback.)
ACKs for top commit:
laanwj:
ACK a16378e501
hebasto:
ACK a16378e501
Tree-SHA512: 9669cc3bc7ba056f913ee77c2ef9d9ee2313da947fc07f8cd955807c34c5d39e3af73587adfe734274ab2412a7dfb1e2dfe7ee4904ca28cfef9bf8061d26a573
b1d905c225 p2p: earlier continuation when no remaining eviction candidates (Vasil Dimov)
c9e8d8f9b1 p2p: process more candidates per protection iteration (Jon Atack)
02e411ec45 p2p: iterate eviction protection only on networks having candidates (Jon Atack)
5adb064574 bench: add peer eviction protection benchmarks (Jon Atack)
566357f8f7 refactor: move GetRandomNodeEvictionCandidates() to test utilities (Jon Atack)
Pull request description:
This follow-up to #21261 improves `ProtectEvictionCandidatesByRatio()` for better performance.
Benchmarks are added; the performance improvement is between 2x and 5x for the benchmarked cases (CPU 2.50GHz, Turbo off, performance mode, Debian Clang 11 non-debug build).
```
$ ./src/bench/bench_bitcoin -filter="EvictionProtection*.*"
```
The refactored code is well-covered by existing unit tests and also a fuzzer.
- `$ ./src/test/test_bitcoin -t net_peer_eviction_tests`
- `$ FUZZ=node_eviction ./src/test/fuzz/fuzz ../qa-assets/fuzz_seed_corpus/node_eviction`
ACKs for top commit:
klementtan:
Tested and code review ACK b1d905c2.
vasild:
ACK b1d905c225
jarolrod:
ACK b1d905c225
Tree-SHA512: a3a6607b9ea2fec138da9780c03f63e177b6712091c5a3ddc3804b896a7585216446310280791f5e20cc023d02d2f03a4139237e12b5c1d7f2a1fa1011610e96
Negative `nNew` or `nTried` are not possible during normal operation.
So, if we read such values during unserialize, report addrman
corruption.
Fixes https://github.com/bitcoin/bitcoin/issues/22450
ceb7b35a39 refactor: move UpdateTip into CChainState (James O'Beirne)
4abf0779d6 refactor: no mempool arg to GetCoinsCacheSizeState (James O'Beirne)
46e3efd1e4 refactor: move UpdateMempoolForReorg into CChainState (James O'Beirne)
617661703a validation: make CChainState::m_mempool optional (James O'Beirne)
Pull request description:
Make `CChainState::m_mempool` optional by making it a pointer instead of a reference. This will allow a simplification to assumeutxo semantics (see https://github.com/bitcoin/bitcoin/pull/15606#pullrequestreview-692965905) and help facilitate the `-nomempool` option.
ACKs for top commit:
jnewbery:
ACK ceb7b35a39
naumenkogs:
ACK ceb7b35a39
ryanofsky:
Code review ACK ceb7b35a39 (just minor style and test tweaks since last review)
lsilva01:
Code review ACK and tested on Signet ACK ceb7b35a39
MarcoFalke:
review ACK ceb7b35a39😌
Tree-SHA512: cc445ad33439d5918cacf80a6354eea8f3d33bb7719573ed5b970fad1a0dab410bcd70be44c862b8aba1b71263b82d79876688c553e339362653dfb3d8ec81e6
