54faac9689 guix: Remove guix store paths from glibc (Andrew Chow)
1d4d711de2 guix: Map all guix store prefixes to /usr (Andrew Chow)
Pull request description:
This cherry-picks two commits from #24615, with minor edits. When building master, only the `x86_64-apple-darwin` build is reproducible across x86_64 and arm64. With these two changes we get x86_64 -> arm64 reproducibility for:
* `powerpc64-linux-gnu`
* `powerpc64le-linux-gnu`
* `x86_64-apple-darwin`
* `x86_64-linux-gnu`
We can't compare `aarch64-linux-gnu`, because we can't currently build for `aarch64-linux-gnu` on `aarch64-linux-gnu`/`arm64`. See (#22458).
For all the other hosts, the reproducibility issues are known / being worked on (see discussion in #24615). However there's no real reason to wait for those to be fixed before merging these changes, as it'd be great to have cross arch reproducibility just for `x86_64-linux-gnu`. I'm also unsure about the approach taken in that PR in regards to the libtool changes (and think there might even be a Guix bug involved).
As I've added to the patch file, we may be able to drop these patches and use `--with-nonshared-cflags` when we are building newer versions of glibc.
Guix Build (x86_64):
```bash
1eadc59775a209e707539a6bdfe4c96e13a17f5373bfaf6477a65c1a44b2459d guix-build-54faac968971/output/arm-linux-gnueabihf/SHA256SUMS.part
1f424e448223a1e1ee1658efeb3bcc0d8b08a2a2bdc9d2dc779c931b956b527f guix-build-54faac968971/output/arm-linux-gnueabihf/bitcoin-54faac968971-arm-linux-gnueabihf-debug.tar.gz
212f55cf55dac34a3a6471bf0585f5ba1ed0a4801e9c0003961617881edb7ee7 guix-build-54faac968971/output/arm-linux-gnueabihf/bitcoin-54faac968971-arm-linux-gnueabihf.tar.gz
f920fdc9407371be8fc5638a0f5ce2f1202889d820bda4451096d162b5d28d94 guix-build-54faac968971/output/arm64-apple-darwin/SHA256SUMS.part
066098fe095a3dd46fa3c84e459aca1e8e4b3d564ab3f20a2542c8c46dd37b5a guix-build-54faac968971/output/arm64-apple-darwin/bitcoin-54faac968971-arm64-apple-darwin-unsigned.dmg
d2dc320975f8ec67595c274fcb7eda56c18ba72b905160537be97408cfdf1baf guix-build-54faac968971/output/arm64-apple-darwin/bitcoin-54faac968971-arm64-apple-darwin-unsigned.tar.gz
68013ac011ebac25e8ee2d2421266c66d1e20d309133eb6121ba89807e17fda5 guix-build-54faac968971/output/arm64-apple-darwin/bitcoin-54faac968971-arm64-apple-darwin.tar.gz
d992dd9f50fec89e0bdf63e8e9352ebcd2b503424cb441f01c06074aa0b63b0e guix-build-54faac968971/output/dist-archive/bitcoin-54faac968971.tar.gz
e26ea298d15a87ce484afa59ca0c36ecfd173314b440e091cd93de11fe2dd91c guix-build-54faac968971/output/powerpc64-linux-gnu/SHA256SUMS.part
1a17091e7e515cc89a2e0a91a08ea0deb48847d5650be936f3365449002a59a5 guix-build-54faac968971/output/powerpc64-linux-gnu/bitcoin-54faac968971-powerpc64-linux-gnu-debug.tar.gz
63ce21f6ee3a971a7a4533934ec559c727d2e3fc123fb65b2dec622168af76d5 guix-build-54faac968971/output/powerpc64-linux-gnu/bitcoin-54faac968971-powerpc64-linux-gnu.tar.gz
267ff9d9ed3d8108033d676218399c4e56ba83eb30a3b4a6417cac8d1ce5e2f4 guix-build-54faac968971/output/powerpc64le-linux-gnu/SHA256SUMS.part
a0db19b9829bd65067e2075c2d3e55065d03a4456a31ccb3ed8a70f7f3dcf1a0 guix-build-54faac968971/output/powerpc64le-linux-gnu/bitcoin-54faac968971-powerpc64le-linux-gnu-debug.tar.gz
9971ba819854a77306358b31ddc2d21074b8cca57cfd8ce7f2ca83a1cc8f0a46 guix-build-54faac968971/output/powerpc64le-linux-gnu/bitcoin-54faac968971-powerpc64le-linux-gnu.tar.gz
e18fe6d4db3048368db25846555a30ec97f71dcfdfcc3e86d7fe46c33c762a26 guix-build-54faac968971/output/riscv64-linux-gnu/SHA256SUMS.part
4879fd332bee8570e6edfe5d0cbd3eea7df18f058ed25286aedb377858e27793 guix-build-54faac968971/output/riscv64-linux-gnu/bitcoin-54faac968971-riscv64-linux-gnu-debug.tar.gz
470fafbab9d7328e172c658f58fd6f62f7890ce5d3dfe3bbfacd717b6d2c00f3 guix-build-54faac968971/output/riscv64-linux-gnu/bitcoin-54faac968971-riscv64-linux-gnu.tar.gz
6b119a38b8ac1dd50e3cee69e8464ab21d624217cb3f50a1aef216e9ece27946 guix-build-54faac968971/output/x86_64-apple-darwin/SHA256SUMS.part
afff2a8184007d6a3eaf7b6735417e7b9b404a801306c71f7d653907055d442c guix-build-54faac968971/output/x86_64-apple-darwin/bitcoin-54faac968971-x86_64-apple-darwin-unsigned.dmg
775298169fa45197f5b560e5581d6e3c021ed009487065f7e3a042914396b3cd guix-build-54faac968971/output/x86_64-apple-darwin/bitcoin-54faac968971-x86_64-apple-darwin-unsigned.tar.gz
51b0d2adae63aeca021621ec25278c202e82d8bc3ef5e7a9d2bfd93bd7a3fcab guix-build-54faac968971/output/x86_64-apple-darwin/bitcoin-54faac968971-x86_64-apple-darwin.tar.gz
ca9cd0083083ec1632a37396076fef638c00fed99efbb9e39521195867746301 guix-build-54faac968971/output/x86_64-linux-gnu/SHA256SUMS.part
0e5a017bca3c0a2a466d230a470ec9c44b5d0bfd60d98f62a3a54f4cccb8cd23 guix-build-54faac968971/output/x86_64-linux-gnu/bitcoin-54faac968971-x86_64-linux-gnu-debug.tar.gz
3b26566a553f17ccbcbf548675d0e8ce47a58a1e23c6b7c9792565117a223855 guix-build-54faac968971/output/x86_64-linux-gnu/bitcoin-54faac968971-x86_64-linux-gnu.tar.gz
fa2f240f7bd7cfadfeb08188c044c8e34fd9c24785f4e2035c1256c3173c5589 guix-build-54faac968971/output/x86_64-w64-mingw32/SHA256SUMS.part
0212ae95d100c9a4dbe062a14b49952279c91cbf352c786369320c3ed006c23a guix-build-54faac968971/output/x86_64-w64-mingw32/bitcoin-54faac968971-win64-debug.zip
750aa7c31cfa1bd5e0ae3f2ea52e526a73f1d3879b9f1a365967bbc317d4cca4 guix-build-54faac968971/output/x86_64-w64-mingw32/bitcoin-54faac968971-win64-setup-unsigned.exe
f7bdda020d299df778fd68ad556d8124d87f7f9f0c4a77b62e05db20f5bbec2b guix-build-54faac968971/output/x86_64-w64-mingw32/bitcoin-54faac968971-win64-unsigned.tar.gz
d7f8b16634ceb95db3d28a024827a51f689dc0ab34ed40f205861bbc254f6206 guix-build-54faac968971/output/x86_64-w64-mingw32/bitcoin-54faac968971-win64.zip
```
Guix Build (arm64):
```bash
e4492294c284054e8378f8ac0c08b5d2efe5eeeec57ca12c3192da87a4dd4266 guix-build-54faac968971/output/arm-linux-gnueabihf/SHA256SUMS.part
1f424e448223a1e1ee1658efeb3bcc0d8b08a2a2bdc9d2dc779c931b956b527f guix-build-54faac968971/output/arm-linux-gnueabihf/bitcoin-54faac968971-arm-linux-gnueabihf-debug.tar.gz
b2c454f589afea22d5126be14d1b9eb3aed7d99d59e50591db18c2cc3f190b23 guix-build-54faac968971/output/arm-linux-gnueabihf/bitcoin-54faac968971-arm-linux-gnueabihf.tar.gz
575bbf78dd6002a7a5653277c24cd8d98471b6454e0801b558ac2a754788f2fa guix-build-54faac968971/output/arm64-apple-darwin/SHA256SUMS.part
03d8175e0db26b56fee757b307b8b519e4df0c639caa6afdfdea5ce7de63ecc4 guix-build-54faac968971/output/arm64-apple-darwin/bitcoin-54faac968971-arm64-apple-darwin-unsigned.dmg
8ab4e3a65f09168b42c40ec736e507eb3e9b787d27a56db21a3c601e81d81262 guix-build-54faac968971/output/arm64-apple-darwin/bitcoin-54faac968971-arm64-apple-darwin-unsigned.tar.gz
43024487cf0120a0a42aeba228f01f2a5303d2940d838f59106fa246a484662d guix-build-54faac968971/output/arm64-apple-darwin/bitcoin-54faac968971-arm64-apple-darwin.tar.gz
d992dd9f50fec89e0bdf63e8e9352ebcd2b503424cb441f01c06074aa0b63b0e guix-build-54faac968971/output/dist-archive/bitcoin-54faac968971.tar.gz
e26ea298d15a87ce484afa59ca0c36ecfd173314b440e091cd93de11fe2dd91c guix-build-54faac968971/output/powerpc64-linux-gnu/SHA256SUMS.part
1a17091e7e515cc89a2e0a91a08ea0deb48847d5650be936f3365449002a59a5 guix-build-54faac968971/output/powerpc64-linux-gnu/bitcoin-54faac968971-powerpc64-linux-gnu-debug.tar.gz
63ce21f6ee3a971a7a4533934ec559c727d2e3fc123fb65b2dec622168af76d5 guix-build-54faac968971/output/powerpc64-linux-gnu/bitcoin-54faac968971-powerpc64-linux-gnu.tar.gz
267ff9d9ed3d8108033d676218399c4e56ba83eb30a3b4a6417cac8d1ce5e2f4 guix-build-54faac968971/output/powerpc64le-linux-gnu/SHA256SUMS.part
a0db19b9829bd65067e2075c2d3e55065d03a4456a31ccb3ed8a70f7f3dcf1a0 guix-build-54faac968971/output/powerpc64le-linux-gnu/bitcoin-54faac968971-powerpc64le-linux-gnu-debug.tar.gz
9971ba819854a77306358b31ddc2d21074b8cca57cfd8ce7f2ca83a1cc8f0a46 guix-build-54faac968971/output/powerpc64le-linux-gnu/bitcoin-54faac968971-powerpc64le-linux-gnu.tar.gz
603e09db23e20ee834cfdeaa58517fa6795779131f76c8e67c79ee4118043ba6 guix-build-54faac968971/output/riscv64-linux-gnu/SHA256SUMS.part
4879fd332bee8570e6edfe5d0cbd3eea7df18f058ed25286aedb377858e27793 guix-build-54faac968971/output/riscv64-linux-gnu/bitcoin-54faac968971-riscv64-linux-gnu-debug.tar.gz
aad8eb83730dbf079ee2c894e33ebf6df78e302500493b10e72a2aab9fa51b49 guix-build-54faac968971/output/riscv64-linux-gnu/bitcoin-54faac968971-riscv64-linux-gnu.tar.gz
6b119a38b8ac1dd50e3cee69e8464ab21d624217cb3f50a1aef216e9ece27946 guix-build-54faac968971/output/x86_64-apple-darwin/SHA256SUMS.part
afff2a8184007d6a3eaf7b6735417e7b9b404a801306c71f7d653907055d442c guix-build-54faac968971/output/x86_64-apple-darwin/bitcoin-54faac968971-x86_64-apple-darwin-unsigned.dmg
775298169fa45197f5b560e5581d6e3c021ed009487065f7e3a042914396b3cd guix-build-54faac968971/output/x86_64-apple-darwin/bitcoin-54faac968971-x86_64-apple-darwin-unsigned.tar.gz
51b0d2adae63aeca021621ec25278c202e82d8bc3ef5e7a9d2bfd93bd7a3fcab guix-build-54faac968971/output/x86_64-apple-darwin/bitcoin-54faac968971-x86_64-apple-darwin.tar.gz
ca9cd0083083ec1632a37396076fef638c00fed99efbb9e39521195867746301 guix-build-54faac968971/output/x86_64-linux-gnu/SHA256SUMS.part
0e5a017bca3c0a2a466d230a470ec9c44b5d0bfd60d98f62a3a54f4cccb8cd23 guix-build-54faac968971/output/x86_64-linux-gnu/bitcoin-54faac968971-x86_64-linux-gnu-debug.tar.gz
3b26566a553f17ccbcbf548675d0e8ce47a58a1e23c6b7c9792565117a223855 guix-build-54faac968971/output/x86_64-linux-gnu/bitcoin-54faac968971-x86_64-linux-gnu.tar.gz
da10e58c2616b7d96fb00d24f34834b737b190c91bd533fc11130c5faf77d697 guix-build-54faac968971/output/x86_64-w64-mingw32/SHA256SUMS.part
7b69ac09edb7795b61242d8b929808b7615e4011f1d312d495cc9410ded6c574 guix-build-54faac968971/output/x86_64-w64-mingw32/bitcoin-54faac968971-win64-debug.zip
750aa7c31cfa1bd5e0ae3f2ea52e526a73f1d3879b9f1a365967bbc317d4cca4 guix-build-54faac968971/output/x86_64-w64-mingw32/bitcoin-54faac968971-win64-setup-unsigned.exe
f7bdda020d299df778fd68ad556d8124d87f7f9f0c4a77b62e05db20f5bbec2b guix-build-54faac968971/output/x86_64-w64-mingw32/bitcoin-54faac968971-win64-unsigned.tar.gz
e90e9a7e86839cbbde7b17610b7fb6eb9a960703232d3b92040dce8df55861bd guix-build-54faac968971/output/x86_64-w64-mingw32/bitcoin-54faac968971-win64.zip
```
ACKs for top commit:
laanwj:
ACK 54faac9689
Tree-SHA512: 258e7de8e8df00995b6e952ef10d054ad127237265811eaafa537501be84459ae8f8f638618365d81e4eee8b7013db768b61c343d68e46d1d90a194bdc26b852
4e569c8bd8 guix: remove explicit glibc stack protector disabling (fanquake)
Pull request description:
While glibc 2.25 and newer *can* be built with stack-smashing-protection
enabled, it isn't used by default, and still isn't, as of glibc 2.35,
so I can't see a reason to explicitly disable it.
I'd also like to move in the direction of enabling, by default,
hardening options for the toolchains we build, so removing the explicit
disabling is a step in that direction.
Will be following up with some changes based on this change.
Guix Build (x86_64):
```bash
954b393f5c775919e32b725a45aa93af8a5e75ead348f904304c0367583b41ff guix-build-4e569c8bd85e/output/aarch64-linux-gnu/SHA256SUMS.part
0ff27062ba2ac4c11a966de2d9aea070f54ab5c255068dd992b19fcb33661ffd guix-build-4e569c8bd85e/output/aarch64-linux-gnu/bitcoin-4e569c8bd85e-aarch64-linux-gnu-debug.tar.gz
bf48baf97e21467ce439f6e733cf3a20732adee01bb1d98aa9519c2ec54b5f41 guix-build-4e569c8bd85e/output/aarch64-linux-gnu/bitcoin-4e569c8bd85e-aarch64-linux-gnu.tar.gz
041eac2a2e045e2283cc78361adcc2232c5eecc9ad465624499225a4ad44f5fe guix-build-4e569c8bd85e/output/arm-linux-gnueabihf/SHA256SUMS.part
7a3bbca762f6c3c4fd851fbf74a2f00c21d98c211de5baa06d0ba2fc59505694 guix-build-4e569c8bd85e/output/arm-linux-gnueabihf/bitcoin-4e569c8bd85e-arm-linux-gnueabihf-debug.tar.gz
89e0260075472d10d02de7e3bb382d87f9ffb3d548f533aab0ba7e39f4c796df guix-build-4e569c8bd85e/output/arm-linux-gnueabihf/bitcoin-4e569c8bd85e-arm-linux-gnueabihf.tar.gz
96f893eefaa9fb5af41f46eece3831a3956a5a9ab1f825e4c17c5675bd020bbe guix-build-4e569c8bd85e/output/arm64-apple-darwin/SHA256SUMS.part
65865e481d33e1023adef769ca9a38ca8cdf03e8476a61f1724bb1ab0bc54750 guix-build-4e569c8bd85e/output/arm64-apple-darwin/bitcoin-4e569c8bd85e-arm64-apple-darwin-unsigned.dmg
38da0510c34d9c2bff98da60c873593c6a85bc6f73025990daaa8d5b022819c0 guix-build-4e569c8bd85e/output/arm64-apple-darwin/bitcoin-4e569c8bd85e-arm64-apple-darwin-unsigned.tar.gz
a1b5ccda7780df15fda2131d260542e57601fed2c18092edaa3094c23eafd99d guix-build-4e569c8bd85e/output/arm64-apple-darwin/bitcoin-4e569c8bd85e-arm64-apple-darwin.tar.gz
f1e4fb6d96420865ee1cfdc10960d8b0407ae49d367d5df1901510a8a87a69bf guix-build-4e569c8bd85e/output/dist-archive/bitcoin-4e569c8bd85e.tar.gz
5cc5e3193435bdd0aafc1a43e1dfc7582e585a27453e92e3383ceb8ba6c162ad guix-build-4e569c8bd85e/output/powerpc64-linux-gnu/SHA256SUMS.part
56bfeecb15b0c59dcccb31d1d5df978e7bb9c60bc0661638af7b263958cb8d4d guix-build-4e569c8bd85e/output/powerpc64-linux-gnu/bitcoin-4e569c8bd85e-powerpc64-linux-gnu-debug.tar.gz
46e61a752ba3ac1e553c82f4615854c27b38b9c2e5abd318840d3d5383e1384d guix-build-4e569c8bd85e/output/powerpc64-linux-gnu/bitcoin-4e569c8bd85e-powerpc64-linux-gnu.tar.gz
52ea9adf7b3a88fa88e89b53852d1d7917af959bee0b67c218959b1123f67c57 guix-build-4e569c8bd85e/output/powerpc64le-linux-gnu/SHA256SUMS.part
2a1a65bd55cc9c83ccbb296e7fe41d5b313466cf8d70ef8aec81aa47e346e422 guix-build-4e569c8bd85e/output/powerpc64le-linux-gnu/bitcoin-4e569c8bd85e-powerpc64le-linux-gnu-debug.tar.gz
b8dbcf97a83a1dd53d23eeafa714e3a3cb8d0b087c060978137e47fdab2b261f guix-build-4e569c8bd85e/output/powerpc64le-linux-gnu/bitcoin-4e569c8bd85e-powerpc64le-linux-gnu.tar.gz
87e3823e246e139ad14c4d44c8e2ed5e1bebea1a02d3931e66232505a1b35463 guix-build-4e569c8bd85e/output/riscv64-linux-gnu/SHA256SUMS.part
1307b92c608b1001628fda1792fbcb61183286cff707be930bcb1d887f5a4b02 guix-build-4e569c8bd85e/output/riscv64-linux-gnu/bitcoin-4e569c8bd85e-riscv64-linux-gnu-debug.tar.gz
0f660a9165a26f627be913e6bf1bb81cbabebee742031d0a75131a7e380e6c8a guix-build-4e569c8bd85e/output/riscv64-linux-gnu/bitcoin-4e569c8bd85e-riscv64-linux-gnu.tar.gz
d530151878bdf70c0913a12ba1aa49dad9ba62ac9edd70cda3982fd0fe327e93 guix-build-4e569c8bd85e/output/x86_64-apple-darwin/SHA256SUMS.part
dc3a9ee571854066ea03d60c1f2b8012fdff12ea1e74ab4ce02b1effc6689436 guix-build-4e569c8bd85e/output/x86_64-apple-darwin/bitcoin-4e569c8bd85e-x86_64-apple-darwin-unsigned.dmg
94c0522390e5650d91d63c6afa5bce895a60c17c1365e0d87a898c2868093dc9 guix-build-4e569c8bd85e/output/x86_64-apple-darwin/bitcoin-4e569c8bd85e-x86_64-apple-darwin-unsigned.tar.gz
364d72282e8824d5dffe184fc10bdcbc9cdf96e8c0ac379b8392e1e1992e3307 guix-build-4e569c8bd85e/output/x86_64-apple-darwin/bitcoin-4e569c8bd85e-x86_64-apple-darwin.tar.gz
0d3ce4cbf3444fc9a3c488f12ef7b73d07b85bcf3d4d9500b689d44506a79818 guix-build-4e569c8bd85e/output/x86_64-linux-gnu/SHA256SUMS.part
f584d494db5594ae2bc06e09f8e68c11e446bc82cb8a1dfa6afee5d3a079b5af guix-build-4e569c8bd85e/output/x86_64-linux-gnu/bitcoin-4e569c8bd85e-x86_64-linux-gnu-debug.tar.gz
ea0e9316dd25ebee9023f3c65cb99fe846de6fe56152d4926005f9da500a502c guix-build-4e569c8bd85e/output/x86_64-linux-gnu/bitcoin-4e569c8bd85e-x86_64-linux-gnu.tar.gz
a41966b6d13aa1e702cc357bbedfd51bcb431caa39bb904efd9611e3a945bf1c guix-build-4e569c8bd85e/output/x86_64-w64-mingw32/SHA256SUMS.part
5f150613204341d91a4b755c74120e233567187ba4f9151a12e39e5304efb3a1 guix-build-4e569c8bd85e/output/x86_64-w64-mingw32/bitcoin-4e569c8bd85e-win64-debug.zip
6a94dc9c5dcb2a4448a37573baab50f405e08af0d5a4de8a8046cba5445153e4 guix-build-4e569c8bd85e/output/x86_64-w64-mingw32/bitcoin-4e569c8bd85e-win64-setup-unsigned.exe
31fe85ba31ed84ebbbc4cc42f593e1de1811c1ecc9a0a094d05b0914bd151174 guix-build-4e569c8bd85e/output/x86_64-w64-mingw32/bitcoin-4e569c8bd85e-win64-unsigned.tar.gz
0a6d590c26a47c51192e4003ad97ecd6b7ad91c8f8612ea310fb324bce5dc15a guix-build-4e569c8bd85e/output/x86_64-w64-mingw32/bitcoin-4e569c8bd85e-win64.zip
```
Guix Build (arm64):
```bash
2ce621cb469772c318a29b21bc4dd546353130a688a5ecb66373256c7be2c37a guix-build-4e569c8bd85e/output/arm-linux-gnueabihf/SHA256SUMS.part
13abe55069581ca711529d058a8e5de732c6630a94b7e912e9c31f606241c264 guix-build-4e569c8bd85e/output/arm-linux-gnueabihf/bitcoin-4e569c8bd85e-arm-linux-gnueabihf-debug.tar.gz
7a60cd7d9aee30bc8e08cf9d52bd032f82e48214c81130e2f61ca3da71c01477 guix-build-4e569c8bd85e/output/arm-linux-gnueabihf/bitcoin-4e569c8bd85e-arm-linux-gnueabihf.tar.gz
d614a4acfed70f61814a5c26bf51594e0cc666fc3dadc3df805e5cc608835550 guix-build-4e569c8bd85e/output/arm64-apple-darwin/SHA256SUMS.part
c9d5705b947c461ade878d7a0110ae5b34b384991f5bf6e86db0b79f421d4f81 guix-build-4e569c8bd85e/output/arm64-apple-darwin/bitcoin-4e569c8bd85e-arm64-apple-darwin-unsigned.dmg
c1897d204e75b9ef8a58fb3f2c85d9c306b05dbd6c8f74a2b4ccfbd85aed5574 guix-build-4e569c8bd85e/output/arm64-apple-darwin/bitcoin-4e569c8bd85e-arm64-apple-darwin-unsigned.tar.gz
1c9e188d76c18785d4500c1c7ab0e049cf35c878803266580913e8cc4bd01bf6 guix-build-4e569c8bd85e/output/arm64-apple-darwin/bitcoin-4e569c8bd85e-arm64-apple-darwin.tar.gz
f1e4fb6d96420865ee1cfdc10960d8b0407ae49d367d5df1901510a8a87a69bf guix-build-4e569c8bd85e/output/dist-archive/bitcoin-4e569c8bd85e.tar.gz
f3e7d6b6aca3ca4f150e0e91e9532f4eb21c4f60ab1c21b6ecfaa9c862f9f8a8 guix-build-4e569c8bd85e/output/powerpc64-linux-gnu/SHA256SUMS.part
48e630949976ee298bccf01cfbbb7fea29c9bd0bc658cf0564d4a3e1997556e8 guix-build-4e569c8bd85e/output/powerpc64-linux-gnu/bitcoin-4e569c8bd85e-powerpc64-linux-gnu-debug.tar.gz
9e30c4987f3657ba6499a78c5c578e430c55f71991fc9ad6f3ecf4847ed1814e guix-build-4e569c8bd85e/output/powerpc64-linux-gnu/bitcoin-4e569c8bd85e-powerpc64-linux-gnu.tar.gz
128ce9194e377b013baceafc4ddba0f70c239e36057c9dc0a9213caa34c5064f guix-build-4e569c8bd85e/output/powerpc64le-linux-gnu/SHA256SUMS.part
8e4a41c07e9427de4054069c3af668157372cc6cd86d758c0b35b7ed906e5365 guix-build-4e569c8bd85e/output/powerpc64le-linux-gnu/bitcoin-4e569c8bd85e-powerpc64le-linux-gnu-debug.tar.gz
da6924709b35f7fbaf9b7b772e0f14be5b583e9453b0cae58b6a5b1e159580c7 guix-build-4e569c8bd85e/output/powerpc64le-linux-gnu/bitcoin-4e569c8bd85e-powerpc64le-linux-gnu.tar.gz
2415604bf3651ea18dcbb4ec5bf73372bdc19c80aa316b864de20c8a5df4bf34 guix-build-4e569c8bd85e/output/riscv64-linux-gnu/SHA256SUMS.part
8f6ee4b69fb33b40ad505c091684258ded340ab9936b554f8fa4e499d4da1155 guix-build-4e569c8bd85e/output/riscv64-linux-gnu/bitcoin-4e569c8bd85e-riscv64-linux-gnu-debug.tar.gz
0a941d44532287288a9859c35fdaa940c940c1e8f4a17b7994e3796c9a668d57 guix-build-4e569c8bd85e/output/riscv64-linux-gnu/bitcoin-4e569c8bd85e-riscv64-linux-gnu.tar.gz
d530151878bdf70c0913a12ba1aa49dad9ba62ac9edd70cda3982fd0fe327e93 guix-build-4e569c8bd85e/output/x86_64-apple-darwin/SHA256SUMS.part
dc3a9ee571854066ea03d60c1f2b8012fdff12ea1e74ab4ce02b1effc6689436 guix-build-4e569c8bd85e/output/x86_64-apple-darwin/bitcoin-4e569c8bd85e-x86_64-apple-darwin-unsigned.dmg
94c0522390e5650d91d63c6afa5bce895a60c17c1365e0d87a898c2868093dc9 guix-build-4e569c8bd85e/output/x86_64-apple-darwin/bitcoin-4e569c8bd85e-x86_64-apple-darwin-unsigned.tar.gz
364d72282e8824d5dffe184fc10bdcbc9cdf96e8c0ac379b8392e1e1992e3307 guix-build-4e569c8bd85e/output/x86_64-apple-darwin/bitcoin-4e569c8bd85e-x86_64-apple-darwin.tar.gz
047d3eae54136b7f5fb20487fb2c57455dda6bb88594065b71843400fbc41824 guix-build-4e569c8bd85e/output/x86_64-linux-gnu/SHA256SUMS.part
8bfb97bcab8d5e0a86a2a8d20be5215d8bb615a8b6c3ad69e1db5028caf2dd29 guix-build-4e569c8bd85e/output/x86_64-linux-gnu/bitcoin-4e569c8bd85e-x86_64-linux-gnu-debug.tar.gz
4962550d7d113e8544a33e2ffa5a0e77e172984c17bfa461a631bf08dc7cc545 guix-build-4e569c8bd85e/output/x86_64-linux-gnu/bitcoin-4e569c8bd85e-x86_64-linux-gnu.tar.gz
5ad1661c475308c6df102aee51261e36583fe0f5f73713d7f19384b63755a3c5 guix-build-4e569c8bd85e/output/x86_64-w64-mingw32/SHA256SUMS.part
62cf3e15e638f48bd0931a847ba5e5636422fb6bd00da41251c1f636d39c5822 guix-build-4e569c8bd85e/output/x86_64-w64-mingw32/bitcoin-4e569c8bd85e-win64-debug.zip
6a94dc9c5dcb2a4448a37573baab50f405e08af0d5a4de8a8046cba5445153e4 guix-build-4e569c8bd85e/output/x86_64-w64-mingw32/bitcoin-4e569c8bd85e-win64-setup-unsigned.exe
31fe85ba31ed84ebbbc4cc42f593e1de1811c1ecc9a0a094d05b0914bd151174 guix-build-4e569c8bd85e/output/x86_64-w64-mingw32/bitcoin-4e569c8bd85e-win64-unsigned.tar.gz
a3c7db0ca4b557810b5e5f1ec14cecaf47b5bf51631798d8675243bb6ecedf8f guix-build-4e569c8bd85e/output/x86_64-w64-mingw32/bitcoin-4e569c8bd85e-win64.zip
```
ACKs for top commit:
laanwj:
Code review ACK 4e569c8bd8
Tree-SHA512: 7f75c304ec67d824ce17be1acb0d67c3946cc346444abcac0a13762365566d101aa784f92dd28ef15b664f1a5f64ae1f60ca91b2538de7ea08a7684bf33cda0d
Without ffile-prefix-map, the debug symbols will contain paths for the
guix store which will include the hashes of each package. However, the
hash for the same package will differ when on different architectures.
In order to be reproducible regardless of the architecture used to build
the package, map all guix store prefixes to something fixed, e.g. /usr.
We might be able to drop this in favour of using --with-nonshared-cflags
when we being using newer versions of glibc.
Without ffile-prefix-map, the debug symbols will contain paths for the
guix store which will include the hashes of each package. However, the
hash for the same package will differ when on different architectures.
In order to be reproducible regardless of the architecture used to build
the package, map all guix store prefixes to something fixed, e.g. /usr.
While glibc 2.25 and newer *can* be built with stack-smashing-protection
enabled, it isn't used by default, and still isn't, as of glibc 2.35,
so I can't see a reason to explicitly disable it.
I'd also like to move in the direction of enabling, by default,
hardening options for the toolchains we build, so removing the explicit
disabling is a step in that direction.
Will be following up with some changes based on this PR.
667e316bcb contrib: Update makeseeds to asmap-nextgen (laanwj)
ae00b9e02c contrib: add seeds progress indicator and remove asmap one in makeseeds script (Jon Atack)
b54180303d contrib: Use asmap for ASN lookup in makeseeds (laanwj)
Pull request description:
Add an argument `-a` to provide a asmap file to do the IP to ASN lookups.
This speeds up the script greatly, and makes the output deterministic. Also removes the dependency on `dns.lookup`.
I've annotated the output with ASxxxx comments to provide a way to verify the functionality.
For now I've added instructions in README.md to download and use the `demo.map` from the asmap repository. When we have some other mechanism for distributing asmap files we could switch to that.
This continues #24824. I've removed the fallbacks and extra complexity, as everyone will be using the same instructions anyway.
Co-authored-by: Pieter Wuille <pieter.wuille@gmail.com>
Co-authored-by: russeree <reese.russell@ymail.com>
ACKs for top commit:
sipa:
ACK 667e316bcb
dunxen:
re-ACK 667e316
Tree-SHA512: c4cedfbd1dee6be7547aa92dd9e262c46f0ff8099e647559b2a40eab0cc9874e9a813706630dd5c880390d23f432e789fb3e7e8a09f376f567071e68f5904c65
As of version 2.4.7, libtool now respects ARFLAGS, which we use, and has
changed the default ARFLAGS from cru to cr (which we also do, see
configure).
This eliminates spammy `ar` output such as:
```bash
CXXLD libunivalue.la
/root/.guix-profile/bin/x86_64-linux-gnu-ar: `u' modifier ignored since `D' is the default (see `U')
AR libbitcoin_zmq.a
AR libbitcoin_consensus.a
CXXLD crypto/libbitcoin_crypto_base.la
CXXLD crypto/libbitcoin_crypto_sse41.la
/root/.guix-profile/bin/x86_64-linux-gnu-ar: `u' modifier ignored since `D' is the default (see `U')
/root/.guix-profile/bin/x86_64-linux-gnu-ar: `u' modifier ignored since `D' is the default (see `U')
CXXLD crypto/libbitcoin_crypto_avx2.la
CXXLD crypto/libbitcoin_crypto_x86_shani.la
CXXLD leveldb/libleveldb.la
/root/.guix-profile/bin/x86_64-linux-gnu-ar: `u' modifier ignored since `D' is the default (see `U')
CXXLD crc32c/libcrc32c.la
/root/.guix-profile/bin/x86_64-linux-gnu-ar: `u' modifier ignored since `D' is the default (see `U')
CXXLD leveldb/libmemenv.la
/root/.guix-profile/bin/x86_64-linux-gnu-ar: `u' modifier ignored since `D' is the default (see `U')
/root/.guix-profile/bin/x86_64-linux-gnu-ar: `u' modifier ignored since `D' is the default (see `U')
/root/.guix-profile/bin/x86_64-linux-gnu-ar: `u' modifier ignored since `D' is the default (see `U')
AR libbitcoin_cli.a
```
Libtool 2.4.7 release notes:
https://lists.gnu.org/archive/html/autotools-announce/2022-03/msg00000.html
Mostly changes to remove src/univalue exceptions from the various linters,
and the required code changes to make them happy. As well as minor doc
changes.
Now that we use GCC 10 for release builds, we no-longer need to
pass-Wl,-z,noexecstack to get a non-executable stack in RISC-V binaries.
This was originally removed in #21036, but then re-added in #21799, when
we reverted to using GCC 8.
Add an argument `-a` to provide a asmap file to do the IP to ASN
lookups.
This speeds up the script greatly, and makes the output deterministic.
Also removes the dependency on `dns.lookup`.
I've annotated the output with ASxxxx comments to provide a way to
verify the functionality.
For now I've added instructions in README.md to download and use the
`demo.map` from the asmap repository. When we have some other mechanism
for distributing asmap files we could switch to that.
This continues #24824. I've removed all the fallbacks and extra
complexity, as everyone will be using the same instructions anyway.
Co-authored-by: Pieter Wuille <pieter.wuille@gmail.com>
Co-authored-by: James O'Beirne <james.obeirne@pm.me>
Co-authored-by: russeree <reese.russell@ymail.com>
7e9fe6d800 windeploy: Renewed windows code signing certificate (Andrew Chow)
Pull request description:
The current windows code signing certificate expires on May 26 23:59:59 2022 GMT. I have purchased a new code signing certificate which will expire on May 29 23:59:59 2024 GMT.
ACKs for top commit:
laanwj:
ACK 7e9fe6d800
fanquake:
ACK 7e9fe6d800 - tested above with OpenSSL 3 & faketime.
Tree-SHA512: 283eb863d4db0573c7e78fe9d8f1b855533fc45b0995cd2d66e40b5242eb9bc9317b01e1b151fe49d512cd4aa6c48e2390017070f79db46493813fdd0a0f568a
Compiling glibc 2.24 and 2.27 with the new GCC 10 results in a number of new warnings,
i.e:
```bash
libc-tls.c: In function ‘__libc_setup_tls’:
libc-tls.c:208:30: error: array subscript 1 is outside the bounds of an interior zero-length array ‘struct dtv_slotinfo[0]’ [-Werror=zero-length-bounds]
208 | static_slotinfo.si.slotinfo[1].map = main_map;
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~
In file included from ../sysdeps/x86_64/ldsodefs.h:54,
from ../sysdeps/gnu/ldsodefs.h:46,
from ../sysdeps/unix/sysv/linux/ldsodefs.h:25,
from libc-tls.c:20:
../sysdeps/generic/ldsodefs.h:398:7: note: while referencing ‘slotinfo’
398 | } slotinfo[0];
| ^~~~~~~~
```
While we could try and backport all the patches required to fix these up, it would
currently seem easier to disable -Werror, which Guix uses by default when building
glibc.
There are two reasons to perform this bump:
* Fixes#25082 by bumping to a commit that includes a fix for time-dependent unit
tests in libgit2 (f5fe0082abe4547f3fb9f29d8351473cfb3a387b).
* Gives us access to clang-toolchain-14 (14.0.3, 998eda3067c7d21e0d9bb3310d2f5a14b8f1c681),
which is useful for the Guix portion of #21778.
Note that with this bump:
Linux kernels headers update from 5.15.28 to 5.15.37.
eb02713efc doc: add minimum required kernel version to dependencies.md (fanquake)
dcad5f70f1 guix: consolidate kernel headers to 5.15 (fanquake)
Pull request description:
Our minimum supported kernel version is currently defined by Guix, as the version passed to the [`--enable-kernel=`](https://www.gnu.org/software/libc/manual/html_node/Configuring-and-compiling.html) option when configuring glibc. That version is [currently set to 3.2.0](https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/base.scm?id=34e9eae68c9583acce5abc4100add3d88932a5ae#n776):
```scheme
;; This is the default for most architectures as of GNU libc 2.26,
;; but we specify it explicitly for clarity and consistency. See
;; "kernel-features.h" in the GNU libc for details.
"--enable-kernel=3.2.0"
```
and has been that way since we started using Guix (i.e from Guix 1.3.0, with the release v22.0).
Passing `--enable-kernel` defines `__LINUX_KERNEL_VERSION` inside glibc, which is then used to determine supported features & syscall usage. For example, some defines in `unix/sysv/linux/kernel-features.h`, from glibc version 2.24, where glibcs default supported kernel version was still 2.6.32 (it's more modern as of recent releases):
```cpp
#ifndef __LINUX_KERNEL_VERSION
/* We assume the worst; all kernels should be supported. */
# define __LINUX_KERNEL_VERSION0
#endif
/* Support for various CLOEXEC and NONBLOCK flags was added in
2.6.23. */
#define __ASSUME_O_CLOEXEC1
/* prlimit64 is available in 2.6.36. */
#if __LINUX_KERNEL_VERSION >= 0x020624
# define __ASSUME_PRLIMIT641
#endif
```
Note that because we currently specify the `5.15` headers, the exact version being used, i.e 5.15.x, changes when we update our time-machine commit, as Guix updates all it's header packages as new point releases become available. Currently it is [`5.15.28`](https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/linux.scm?id=34e9eae68c9583acce5abc4100add3d88932a5ae#n380). The changelog for the 5.15 headers is available [here](https://cdn.kernel.org/pub/linux/kernel/v5.x/ChangeLog-5.15.36).
Similar to glibc, it may currently be possible to build and run bitcoind against older kernels, however, for the purposes of documenting what we support for our release binaries, I can't see a reason to document anything other than the version that we are targeting when building the glibc used to build the release binaries.
Guix Build (on x86_64):
```bash
22ff2d3a72d337c4eccbfa4c834a67c7c3397f225aedb71a3c636f2708964e93 guix-build-eb02713efc17/output/aarch64-linux-gnu/SHA256SUMS.part
e316b2d0806183e0e51a25722f48af85d145b1581f44f68b925d9f484a5aa0d3 guix-build-eb02713efc17/output/aarch64-linux-gnu/bitcoin-eb02713efc17-aarch64-linux-gnu-debug.tar.gz
da667d935d9331c5cbca2e0e334cd7e56202ee294553459672fa50f13f501c4d guix-build-eb02713efc17/output/aarch64-linux-gnu/bitcoin-eb02713efc17-aarch64-linux-gnu.tar.gz
f44f2921f3dac2c545806dffb579137fb9eeeee15671395f6a7f817ed6213143 guix-build-eb02713efc17/output/arm-linux-gnueabihf/SHA256SUMS.part
101bab25ab2a6c36729cdf840264a85700cc3cfa23d3900b0bee0ac9ae637e8d guix-build-eb02713efc17/output/arm-linux-gnueabihf/bitcoin-eb02713efc17-arm-linux-gnueabihf-debug.tar.gz
9f7338ee42234949ef3104c6cb2b8a723d616a46d0047d833062adc92e3b6b72 guix-build-eb02713efc17/output/arm-linux-gnueabihf/bitcoin-eb02713efc17-arm-linux-gnueabihf.tar.gz
5ca08e7b38c4dd4456145602f25b015c164c0cd0317dd2a26855dc0495605418 guix-build-eb02713efc17/output/arm64-apple-darwin/SHA256SUMS.part
186c9e2df4034472107964e835ee7a05777c7a0fa5e0db12b5740f18e732d7d5 guix-build-eb02713efc17/output/arm64-apple-darwin/bitcoin-eb02713efc17-arm64-apple-darwin-unsigned.dmg
4e252c7775662777ddcb0a1b0efac6b6e71c25479d6b44b821199ae00abd18ca guix-build-eb02713efc17/output/arm64-apple-darwin/bitcoin-eb02713efc17-arm64-apple-darwin-unsigned.tar.gz
4360342994d54839bbf5fb4d86c6c0b0a3cbcef68b0d2c991aab6e81301638e7 guix-build-eb02713efc17/output/arm64-apple-darwin/bitcoin-eb02713efc17-arm64-apple-darwin.tar.gz
f354822050cd625de7445cfa317475cfad90a7e39d135c5b99950ee69969f445 guix-build-eb02713efc17/output/dist-archive/bitcoin-eb02713efc17.tar.gz
aa864574dd692bb40df95e17c08113f5bcd04b7d5997a2f74ca0557cae3edfcd guix-build-eb02713efc17/output/powerpc64-linux-gnu/SHA256SUMS.part
45df2a4ec5592834fd08d36068cd0968de281870cb9df48cc4783078b85985dd guix-build-eb02713efc17/output/powerpc64-linux-gnu/bitcoin-eb02713efc17-powerpc64-linux-gnu-debug.tar.gz
2bc1c481635ac073d61f51db425576a42a54b16218a00e5f80579426dbf9677b guix-build-eb02713efc17/output/powerpc64-linux-gnu/bitcoin-eb02713efc17-powerpc64-linux-gnu.tar.gz
403d7790e611d3e07b25e02549c9f50e51fff2e1a323605db4f9a569712771a7 guix-build-eb02713efc17/output/powerpc64le-linux-gnu/SHA256SUMS.part
db40032c3b25d95012496f1b3fa5df7f207dcbeefa510bd140b96df4dfd84c88 guix-build-eb02713efc17/output/powerpc64le-linux-gnu/bitcoin-eb02713efc17-powerpc64le-linux-gnu-debug.tar.gz
eea8b4ee96dc8a9813b727550bf07202a6f9cba99605247813beb5251c7f2623 guix-build-eb02713efc17/output/powerpc64le-linux-gnu/bitcoin-eb02713efc17-powerpc64le-linux-gnu.tar.gz
7164e539d25c300b993a620caacdcef659bb6a7c4775a873e30ee645c9ceed15 guix-build-eb02713efc17/output/riscv64-linux-gnu/SHA256SUMS.part
3ae271e6fb94e5d4e46a402508a02d659e879d222c6696e57c78530157eb39ae guix-build-eb02713efc17/output/riscv64-linux-gnu/bitcoin-eb02713efc17-riscv64-linux-gnu-debug.tar.gz
8c37d0b790c28b692804b360605baec4371af4f080c0024ba75f06c0096a4356 guix-build-eb02713efc17/output/riscv64-linux-gnu/bitcoin-eb02713efc17-riscv64-linux-gnu.tar.gz
51bbf04cb32b579ba5609fe3ef24e9901f8d49e3311fe9776ee1fdb644f7e0b1 guix-build-eb02713efc17/output/x86_64-apple-darwin/SHA256SUMS.part
8e81f8badb0cff1aa430a899065cf9744b4b2d45addb8e30606a2f8bf08faa26 guix-build-eb02713efc17/output/x86_64-apple-darwin/bitcoin-eb02713efc17-x86_64-apple-darwin-unsigned.dmg
8b4120b6d83c03dae34b0b5a189522d01c523ab005d816339fdfddf9c412ef15 guix-build-eb02713efc17/output/x86_64-apple-darwin/bitcoin-eb02713efc17-x86_64-apple-darwin-unsigned.tar.gz
26e633faba4f05f51f4e0bffaa2bbbf8c2d5d134d6777c9395bf9b65af6a808c guix-build-eb02713efc17/output/x86_64-apple-darwin/bitcoin-eb02713efc17-x86_64-apple-darwin.tar.gz
16c96d1f349ca3fbf5ffb8e00d5defe1af5a14abb6f61abdbd367e9a5e99bf33 guix-build-eb02713efc17/output/x86_64-linux-gnu/SHA256SUMS.part
faa203c9c3943c2f30ca3f4f30c3eee52e38ac9a2f15c6303b0c8ff0be146e07 guix-build-eb02713efc17/output/x86_64-linux-gnu/bitcoin-eb02713efc17-x86_64-linux-gnu-debug.tar.gz
b5cf5154ac0e2138a4ccbc7639026d909e606b9f55c5859ae54d941eb950759b guix-build-eb02713efc17/output/x86_64-linux-gnu/bitcoin-eb02713efc17-x86_64-linux-gnu.tar.gz
bc14a09399ef3a6d9696116ddda6509b6cb1726719dfd462106cb9d2fde32efc guix-build-eb02713efc17/output/x86_64-w64-mingw32/SHA256SUMS.part
7c0a773f0e892e41fe8f7b299be655e53f110a64bd6e77c2e7a6b4c699605498 guix-build-eb02713efc17/output/x86_64-w64-mingw32/bitcoin-eb02713efc17-win64-debug.zip
27f6a50394c61c0efa2f3afe655a265c64a34249041ef0090f9043cd4cdc8c71 guix-build-eb02713efc17/output/x86_64-w64-mingw32/bitcoin-eb02713efc17-win64-setup-unsigned.exe
dd5bb661a9d99bbc2c2c7256996b26bef116b2e61b7497ada26b3322550d53cb guix-build-eb02713efc17/output/x86_64-w64-mingw32/bitcoin-eb02713efc17-win64-unsigned.tar.gz
84d5068dd59180498473263d757192a422859c0704a2e45a762d1635e49efb80 guix-build-eb02713efc17/output/x86_64-w64-mingw32/bitcoin-eb02713efc17-win64.zip
```
Guix build (on arm64):
```bash
2bbcf455381d4be6bb402c705dba5655de3e1b62a0ed1dbbfcb573450a63d148 guix-build-eb02713efc17/output/arm-linux-gnueabihf/SHA256SUMS.part
c9d982eb6c9efc752fffb3a1d2b14e8cc4d9e2cf4c03c2f02eb320d04f52a86d guix-build-eb02713efc17/output/arm-linux-gnueabihf/bitcoin-eb02713efc17-arm-linux-gnueabihf-debug.tar.gz
0c0b3122dcdc051bee1022dc9b0cf7771b7f6b30fa3e7369c97907d8c10d7ea3 guix-build-eb02713efc17/output/arm-linux-gnueabihf/bitcoin-eb02713efc17-arm-linux-gnueabihf.tar.gz
5388f8a7525ed49d11569988e598a0ec68ddcaf9b35cff0c8bcb02187b0fbaad guix-build-eb02713efc17/output/arm64-apple-darwin/SHA256SUMS.part
16b745e2b2e036d65b549be740116d9b6e819730cc76075f01bdbc4beb166724 guix-build-eb02713efc17/output/arm64-apple-darwin/bitcoin-eb02713efc17-arm64-apple-darwin-unsigned.dmg
18eaede02d12dcbb83003272b5b79a08a10067a326542687ab445bfc623ce9e8 guix-build-eb02713efc17/output/arm64-apple-darwin/bitcoin-eb02713efc17-arm64-apple-darwin-unsigned.tar.gz
5fb73968c7ea50c9642d3cddcd745a512be3043ada314b8a1fc94f179744a1d8 guix-build-eb02713efc17/output/arm64-apple-darwin/bitcoin-eb02713efc17-arm64-apple-darwin.tar.gz
f354822050cd625de7445cfa317475cfad90a7e39d135c5b99950ee69969f445 guix-build-eb02713efc17/output/dist-archive/bitcoin-eb02713efc17.tar.gz
fad672b9e5d372ba5511c14ed48ef77bcf303d475f35680bd4a668fee150225a guix-build-eb02713efc17/output/powerpc64-linux-gnu/SHA256SUMS.part
246aa854e87675a0b90cc14f7b6affcfefabfc0f79edd3dc96ae6b98010b8b1c guix-build-eb02713efc17/output/powerpc64-linux-gnu/bitcoin-eb02713efc17-powerpc64-linux-gnu-debug.tar.gz
90e968e0bab84e80a9f2fe9498eea7c59d8908f5a16accd93d7f9318a7098ce0 guix-build-eb02713efc17/output/powerpc64-linux-gnu/bitcoin-eb02713efc17-powerpc64-linux-gnu.tar.gz
65f0c018d882d7fc845a9bb1581824b17e7ecf0df7081ab2538f0e617e120a8b guix-build-eb02713efc17/output/powerpc64le-linux-gnu/SHA256SUMS.part
0bf7fcd127180e5e04112914747496db535226bf05126690f259fa0cf2a96642 guix-build-eb02713efc17/output/powerpc64le-linux-gnu/bitcoin-eb02713efc17-powerpc64le-linux-gnu-debug.tar.gz
14fc3e17dfa903f83e44f970c8b4e4726e7476c59d0fffdec815a1c80ec1b51a guix-build-eb02713efc17/output/powerpc64le-linux-gnu/bitcoin-eb02713efc17-powerpc64le-linux-gnu.tar.gz
9018f95b54d0643d734260b6eb69ee5f086c98e62f25dd579675b467a844793d guix-build-eb02713efc17/output/riscv64-linux-gnu/SHA256SUMS.part
e1f2b3678c22103d7b89cbbeec9b2863c9c6f749ff4cbedd74cb6e62598c0a04 guix-build-eb02713efc17/output/riscv64-linux-gnu/bitcoin-eb02713efc17-riscv64-linux-gnu-debug.tar.gz
bf9fa35119344dfc93048196dd9cd5bb230b0785350ae5150bb4bdb28fd8423d guix-build-eb02713efc17/output/riscv64-linux-gnu/bitcoin-eb02713efc17-riscv64-linux-gnu.tar.gz
51bbf04cb32b579ba5609fe3ef24e9901f8d49e3311fe9776ee1fdb644f7e0b1 guix-build-eb02713efc17/output/x86_64-apple-darwin/SHA256SUMS.part
8e81f8badb0cff1aa430a899065cf9744b4b2d45addb8e30606a2f8bf08faa26 guix-build-eb02713efc17/output/x86_64-apple-darwin/bitcoin-eb02713efc17-x86_64-apple-darwin-unsigned.dmg
8b4120b6d83c03dae34b0b5a189522d01c523ab005d816339fdfddf9c412ef15 guix-build-eb02713efc17/output/x86_64-apple-darwin/bitcoin-eb02713efc17-x86_64-apple-darwin-unsigned.tar.gz
26e633faba4f05f51f4e0bffaa2bbbf8c2d5d134d6777c9395bf9b65af6a808c guix-build-eb02713efc17/output/x86_64-apple-darwin/bitcoin-eb02713efc17-x86_64-apple-darwin.tar.gz
6e5828e2efa4e951b147b8de42f79dee1652933e04c50093bd31ee375c0c4ca9 guix-build-eb02713efc17/output/x86_64-linux-gnu/SHA256SUMS.part
3d3af1d078eee6f66aac2af891fd7d7a77abc3d7164a807d0a7cc44f15e52b9d guix-build-eb02713efc17/output/x86_64-linux-gnu/bitcoin-eb02713efc17-x86_64-linux-gnu-debug.tar.gz
13f452bc65194de16fa91ed87be9790ed0d1a178deefb102fa54d3f9832b8c25 guix-build-eb02713efc17/output/x86_64-linux-gnu/bitcoin-eb02713efc17-x86_64-linux-gnu.tar.gz
1673b5fca6687ea0f196a5f2ce2b79662b3efe01b71f341fc596069a1ade610c guix-build-eb02713efc17/output/x86_64-w64-mingw32/SHA256SUMS.part
d6228bdfb4fbc7b895ed4f0c30e1343c3392bd6e8e5ed33a973887ba0bb749ba guix-build-eb02713efc17/output/x86_64-w64-mingw32/bitcoin-eb02713efc17-win64-debug.zip
27f6a50394c61c0efa2f3afe655a265c64a34249041ef0090f9043cd4cdc8c71 guix-build-eb02713efc17/output/x86_64-w64-mingw32/bitcoin-eb02713efc17-win64-setup-unsigned.exe
dd5bb661a9d99bbc2c2c7256996b26bef116b2e61b7497ada26b3322550d53cb guix-build-eb02713efc17/output/x86_64-w64-mingw32/bitcoin-eb02713efc17-win64-unsigned.tar.gz
50a68d3644dbe5cb5de21993cd0e8992ab9c6aa88c009a75e5d5a55180476ea6 guix-build-eb02713efc17/output/x86_64-w64-mingw32/bitcoin-eb02713efc17-win64.zip
```
ACKs for top commit:
laanwj:
ACK eb02713efc
dongcarl:
Code Review ACK eb02713efc
vincenzopalazzo:
ACK eb02713efc
Tree-SHA512: afee459d881d3231b72711b0beec5410d6b37deb0d94aa0aaca13010f54bf95fadaf7e5081ea8b9c6975a9e2b8be48b761e3b6ce284c06f82be2210db1156e96
b42643c253 doc: update init.cpp -conf help text (josibake)
970b9987ad doc: update devtools, release-process readmes (josibake)
50635d27b4 build: include bitcoin.conf in build outputs (josibake)
6aac946f49 doc: update bitcoin-conf.md (Josiah Baker)
1c7e820ded script: add script to generate example bitcoin.conf (josibake)
b483084d86 doc: replace bitcoin.conf with placeholder file (josibake)
Pull request description:
create a script for parsing the output from `bitcoind --help` to create an example conf file for new users
## problem
per #10746 , `bitcoin.conf` not being put into the data directory during installation causes some confusion for users when running bitcoin. in the discussion on the issue, one proposed solution was to have an example config file and instruct users to `cp` it into their data directory after startup. in addition to #10746 , there have been other requests for a "skeleton config file" (https://github.com/bitcoin/bitcoin/issues/19641) to help users get started with configuring bitcoind.
the main issue with an example config file is that it creates a second source of truth regarding what options are available for configuring bitcoind. this means any changes to the options (including the addition or removal of options) would have to be updated for the command line and also updated in the example file.
this PR addresses this issue by providing a script to generate an example file directly from the `bitcoind --help` on-demand by running `contrib/devtools/gen-bitcoin-conf.sh`. this solution was originally proposed on #10746 and would also solve #19641 . this guarantees any changes made to the command-line options or the command-line options help would also be reflected in the example file after compiling and running the script.
the main purpose of this script is to generate a config file to be included with releases, same as `gen-manpages.sh`. this ensures every release also includes an up-to-date, full example config file for users to edit. the script is also available for users who compile from source for generating an example config for their compiled binary.
## special considerations
this removes the `bitcoin.conf` example file from the repo as it is now generated by this script. the original example file did contain extra text related to how to use certain options but going forward all option help docs should be moved into `init.cpp`
this also edits `init.cpp` to have the option help indicate that `-conf` is not usable from the config file. this is similar to how `-includeconf` 's help indicates it cannot be used from the command line
ACKs for top commit:
laanwj:
Tested and code review ACK b42643c253
Tree-SHA512: 4546e0cef92aa1398da553294ce4712d02e616dd72dcbe0b921af474e54f24750464ec813661f1283802472d1e8774e634dd1cc26fbf1f13286d3e0406c02c09
copy over bitcoin.conf during the build process.
this means `contrib/devtools/gen-bitcoin-conf.sh` will need
to be run and the generated file committed during the release process.
this is the same process used for generating man pages for each release.
027aab663a test, contrib, refactor: use `with` when opening a file (brunoerg)
Pull request description:
When manipulating a file in Python without using `with()`, you have to close the file manually, so this PR does it in `get_block_hashes` (`contrib/linearize/linearize-data.py`).
Edit: this PR does it for all occurances that previously weren't using `with`.
ACKs for top commit:
laanwj:
Code review ACK 027aab663a
Tree-SHA512: 879400968e0013e8678ec16f1fe5d0963a73c1e0d442ca34802d885214f0783d2e9a9b500fc6be7c3b93560a367b6a3d685eee24d2f9ce53fddf064ea6feecf8
this ensures bitcoind option help is the source of truth and also
gives an example conf file for users to customize and copy to their
data directory.
closes#10746
9b0a13a289 tidy: Add include-what-you-use (fanquake)
74cd038e30 refactor: fix includes in src/init (fanquake)
c79ad935f0 refactor: fix includes in src/compat (fanquake)
Pull request description:
We recently added a [`clang-tidy` job](https://github.com/bitcoin/bitcoin/blob/master/ci/test/00_setup_env_native_tidy.sh) to the CI, which generates a compilation database. We can leverage that now existing database to begin running [include-what-you-use](https://include-what-you-use.org/) over the codebase.
This PR demonstrates using a mapping_file to indicate fixups / includes that may differ from IWYU suggestions. In this case, I've added some fixups for glibc includes that I've [upstreamed changes for](https://github.com/include-what-you-use/include-what-you-use/pull/1026):
```bash
# Fixups / upstreamed changes
[
{ include: [ "<bits/termios-c_lflag.h>", private, "<termios.h>", public ] },
{ include: [ "<bits/termios-struct.h>", private, "<termios.h>", public ] },
{ include: [ "<bits/termios-tcflow.h>", private, "<termios.h>", public ] },
]
```
The include "fixing" commits of this PR:
* Adds missing includes.
* Swaps C headers for their C++ counterparts.
* Removes the pointless / unmaintainable `//for abc, xyz` comments. When using IWYU, if anyone wants to see / generate those comments, to see why something is included, it is trivial to do so (IWYU outputs them by default). i.e:
```cpp
// The full include-list for compat/stdin.cpp:
#include <compat/stdin.h>
#include <poll.h> // for poll, pollfd, POLLIN
#include <termios.h> // for tcgetattr, tcsetattr
#include <unistd.h> // for isatty, STDIN_FILENO
```
TODO:
- [ ] Qt mapping_file. There is one in the IWYU repo, but it's for Qt 5.11. Needs testing.
- [ ] Boost mapping_file. There is one in the IWYU repo, but it's for Boost 1.75. Needs testing.
I'm not suggesting we turn this on the for entire codebase, or immediately go-nuts refactoring all includes. However I think our dependency includes are now slim enough, and our CI infrastructure in place such that we can start doing this in some capacity, and just automate away include fixups / refactorings etc.
ACKs for top commit:
MarcoFalke:
review ACK 9b0a13a289
jonatack:
ACK 9b0a13a289 reviewed changes and run CI output in https://cirrus-ci.com/task/4750910332076032
Tree-SHA512: 00beab5a5f2a6fc179abf08321a15391ecccaa91ab56f3c50c511e7b29a0d7c95d8bb43eac2c31489711086f6f77319d43d803cf8ea458e7cd234a780d9ae69e
914076ed5f guix: Improve error message about missed macOS SDK (Hennadii Stepanov)
Pull request description:
The error message now mentions another option for users to specify the path to the macOS SDK.
ACKs for top commit:
fanquake:
ACK 914076ed5f
Tree-SHA512: e8b9a6c8aa984f02cdb4ad1eeb6c22510c0c74c47c831104ac410ef68b3bc40024a52400b5065a3e25603f8a040e72c8c464436cb9a0a018ca73b0d28e0d609d
This change makes naming of `install_name_tool` consistent across
the whole build system.
-BEGIN VERIFY SCRIPT-
sed --in-place --expression='s/INSTALLNAMETOOL/INSTALL_NAME_TOOL/g' $(git grep --files-with-matches 'INSTALLNAMETOOL')
-END VERIFY SCRIPT-
1dd8cbfbc6 build: don't compress macOS DMG (fanquake)
Pull request description:
Skip compressing the macOS DMG, and drop related build steps and dependencies. Uncompressed the DMG increases from ~16mb to ~30mb, which compared to other software a user may download, (Firefox 125mb, VLC 52mb, Open Office 176mb), is still relatively small. When contrasted against the 100's of GB of blockchain data a node will download, an additional 15mb to get the release binary, isn't much additional overhead. Note that if / when we build with LTO enabled for releases, this size will shrink back down significantly again.
`native_libdmg-hfsplus` is not maintained, and I doubt the DMG creation feature will ever be fixed. If at some point `xorrisofs` supports compressing dmgs, we could enable that.
Guix Build on x86_64:
```bash
25b7c8bb7bc8ea014d43cebb844a842d2ac8d5a343039a820d24b649c9e6bc8a guix-build-1dd8cbfbc631/output/arm64-apple-darwin/SHA256SUMS.part
16beb5c52c9bf51b5ce9ef5a0d17c0038238a833383586a1b14acbca78533e4b guix-build-1dd8cbfbc631/output/arm64-apple-darwin/bitcoin-1dd8cbfbc631-arm64-apple-darwin-unsigned.dmg
d8f89a61a7448d6334dbb3639386a7b6340542393933f35421a9e6dfc724e455 guix-build-1dd8cbfbc631/output/arm64-apple-darwin/bitcoin-1dd8cbfbc631-arm64-apple-darwin-unsigned.tar.gz
11617dc261ef602433f5bb29956a40a9085dbc783f519f75fbe06e80970148d0 guix-build-1dd8cbfbc631/output/arm64-apple-darwin/bitcoin-1dd8cbfbc631-arm64-apple-darwin.tar.gz
aa8550d4a394d3161d14ec5e6012ed07354135afb022e905a1946785b4665664 guix-build-1dd8cbfbc631/output/dist-archive/bitcoin-1dd8cbfbc631.tar.gz
2b837f2f971a9738d0b7b8497f7ded740ef5e67c8baa7f30ca33e6b7d826eec8 guix-build-1dd8cbfbc631/output/x86_64-apple-darwin/SHA256SUMS.part
db972b2c06dbde5525a3f9e6ceb9c20a8120bc9a6f15e1d852a4bfac09d88569 guix-build-1dd8cbfbc631/output/x86_64-apple-darwin/bitcoin-1dd8cbfbc631-x86_64-apple-darwin-unsigned.dmg
50fe990c3f9923ee92195125faf6517396e7c1b017a8f4f7d52e991ebce52f0c guix-build-1dd8cbfbc631/output/x86_64-apple-darwin/bitcoin-1dd8cbfbc631-x86_64-apple-darwin-unsigned.tar.gz
1d9022b0ae46ead41046c40f82291ce363760660a3cd6e6ef6a5b1128b90faef guix-build-1dd8cbfbc631/output/x86_64-apple-darwin/bitcoin-1dd8cbfbc631-x86_64-apple-darwin.tar.gz
```
Guix Build on arm64:
```bash
```
ACKs for top commit:
Sjors:
re-tACK 1dd8cbfbc6 on Intel macOS
laanwj:
Build system changes code review ACK 1dd8cbfbc6, I don't know anything about MacOS application formats and their internals so do not have an opinion on the contents of this change.
jarolrod:
ACK 1dd8cbfbc6
Tree-SHA512: 04c5bf78f26a9877777093ec4c50c457107bef59d720839ea5e7d7e4f7961dfee9f86b40cf791524a9e60e9e77403a797e9fcdae3849b60b759f9f66cc31b6ab
ba30a5407e contrib: macdeploy: monkey-patch gen-sdk to be deterministic (Pavol Rusnak)
1868a17e5a contrib: macdeploy: make gen-sdk deterministic (Pavol Rusnak)
Pull request description:
This PR attempts to make `contrib/macdeploy/gen-sdk` deterministic
Can anyone with the `Xcode_12.2.xip` confirm that `gen-sdk` produces the same hash? => `e7ca56bc8804d16624fad68be2e71647747d6629cacaaa3de5fbfa7f444e9eae `
ACKs for top commit:
laanwj:
Tested ACK ba30a5407e
jarolrod:
Tested ACK ba30a5407e
Tree-SHA512: 1638ceaf28e87ef0d21a1a71ef02989f75942b60a12f07236ac709bde96f08f39f816767e35a0fe68c26bf5978e63e74f5385be9d4b8f80a2e89b30f163f4526
107582039a doc: Add gpg key import instructions for Windows (Dave Scotese)
Pull request description:
This is a single commit to replace the three commits from #23916
I propose this change so that Windows users can more easily import signers' keys.
ACKs for top commit:
sipsorcery:
tACK 107582039a.
Tree-SHA512: 7d4ec77ce10f751748c49f1453fa8baf0976b15af4f87dc27f4e2715ad73fbd7dc1f07fcf3e660d63a6b9eb895a5e4105774613d39a2328f73b92d9e6cff4ebd
I have some qualms with maintaining a suspicious hosts list as part as
the repository. But also, it's stale and irrelevant. I've checked the
entire list and none of them is connectable. Only one still appars in
`nodes_main.txt` but with low uptime and an old subversion string so it
wouldn't be picked in the first place.
c457fb144c improve clarity and up max ipv6 ASNs (Baas)
Pull request description:
This PR attempts to address some of the areas of improvement raised in #17020 . Concretely, my proposed change is fairly minor but addresses the following changes to [`makeseeds.py`](https://github.com/bitcoin/bitcoin/blob/master/contrib/seeds/makeseeds.py):
- Increase max seeds per ASN for IPv6 to 10 as recommended [here](https://github.com/bitcoin/bitcoin/pull/16999#issuecomment-536999544), while keeping max seeds per ASN for IPv4 at 2.
- Bump `MIN_BLOCKS` to 730000.
- Improved script clarity: added function types and more docs to functions, added progress indicator when performing ASN lookup, and change string formatting to better align with [bitcoin python style guidelines](https://github.com/bitcoin/bitcoin/blob/master/test/functional/README.md#style-guidelines)
With the different ASN limits for IPv4 and IPv6, and the new minimum block requirement, the current stats look look like:
```
IPv4 IPv6 Onion Pass
470689 73238 0 Initial
470689 73238 0 Skip entries with invalid address
470689 73238 0 After removing duplicates
470688 73238 0 Skip entries from suspicious hosts
6098 1676 0 Enforce minimal number of blocks
5252 1443 0 Require service bit 1
3812 898 0 Require minimum uptime
3738 877 0 Require a known and recent user agent
3715 869 0 Filter out hosts with multiple bitcoin ports
512 512 0 Look up ASNs and limit results per ASN and per net
```
The new ASN max seeds of 10 allows for 512 IPv6 addresses to be included, up from the ~150 that was filtered by the previous version.
While there is more to do for #17020 , these changes I think are fairly isolated from the rest and should make it a bit easier for others to get up to speed with what the functions in the script do.
ACKs for top commit:
laanwj:
Concept and code review ACK c457fb144c
Tree-SHA512: 3ed67868443cc50544e23b27e2341758c3a8866997b0dba47b137032d5e1a13428855daaeed682626ed471542b44435635178d54848a2cd6fe73777679428032
65c49ac750 test: throw `ValueError` for invalid base58 checksum (Sebastian Falbesoner)
219d2c7ee1 contrib: testgen: use base58 methods from test framework (Sebastian Falbesoner)
605fecfb66 scripted-diff: rename `chars` to `b58chars` in test_framework.address (Sebastian Falbesoner)
11c63e374d contrib: testgen: import OP_* constants from test framework (Sebastian Falbesoner)
7d755bb31c contrib: testgen: avoid need for manually setting PYTHONPATH (Sebastian Falbesoner)
Pull request description:
This PR removes the redundant base58 implementation [contrib/testgen/base58.py](https://github.com/bitcoin/bitcoin/blob/master/contrib/testgen/base58.py) for the test generation script `gen_key_io_test_vectors.py` and uses the one from the test framework instead. Additionally, three other cleanups/improvements are done:
- import script operator constants `OP_*` from test framework instead of manually defining them
- add Python path to test framework directly in the script (via `sys.path.append(...)`) instead of needing the caller to specify `PYTHONPATH=...` on the command line (the same approach is done for the signet miner and the message capture scripts)
- rename `chars` to `b58chars` in the test_framework.address module (is more explicit and makes the diff for the base58 replacement smaller)
ACKs for top commit:
laanwj:
Code review ACK 65c49ac750
Tree-SHA512: 92e1534cc320cd56262bf455de7231c6ec821bfcd0ed58aa5718271ecec1a89df7951bf31527a2306db6398e7f2664d2ff8508200c28163c0b164d3f5aaf8b0e
af74e061c0 guix: make it possible to override gpg binary (Pavol Rusnak)
Pull request description:
For example on Qubes OS one might want to use qubes-gpg-client-wrapper instead
Fixes https://github.com/bitcoin/bitcoin/issues/24346
ACKs for top commit:
laanwj:
Concept and code review ACK af74e061c0
Tree-SHA512: 9e56b5fab231f8908fff15c88fe5b356ac4a31a14a27ae2dd3b6e876f32628910a666a4e2da5bf7c5d159de66cf57652c94c81cdc3b1c3d39a23c23e2c77dd03
15893a0781 supp: remove Boost Valgrind suppression (fanquake)
b0740fdcb8 ci: use DWARF-4 for Valgrind CI job (fanquake)
Pull request description:
[clang-14 defaults to using DWARF-5](https://releases.llvm.org/14.0.0/tools/clang/docs/ReleaseNotes.html#dwarf-support-in-clang), which breaks vlagrinds (3.18) ability
to parse debug info. Valgrind [claims to support DWARF-5](https://valgrind.org/docs/manual/dist.news.html) from version
3.18 onwards, but maybe that only works when compiling with GCC.
Explicitly use DWARF-4 for now. Note that from 11.0 [GCC also defaults to
using DWARF-5](https://www.gnu.org/software/gcc/gcc-11/changes.html).
Also remove a Boost related suppression.
Top commit has no ACKs.
Tree-SHA512: e4f476170ac5ccbb43d26e990b24753bda3985b2ac5c8a32e74d2d1d64d1b3a2d80a90fbab345f0a9e404eac7fbd783c20147379208e615d526657e8a57890ca
254a63e097 contrib: refactor: replace `hex_switchEndian` in linearize scripts (Sebastian Falbesoner)
3f863cfff1 contrib: refactor: simplify block header string routine in linearize-data.py (Sebastian Falbesoner)
Pull request description:
This PR simplifies the linearization scripts `linearize-data.py` and `linearize-hashes.py` by replacing overly complicated cruft (block header hash string calculation, hex string reverse) with means of the Python3 standard library.
ACKs for top commit:
laanwj:
Code review ACK 254a63e097
Tree-SHA512: 4a0e20e63bd11f23a190480b22dbc2f2a3070e2a4f3a01b8797f99bb5fc830185e91e6712c8ae97562f9a24a98aa4f19d52f02a3f5fcbe4c578ee88a41382d1d
97af652788 guix: Drop code for the unsupported `i686-linux-gnu` host (Hennadii Stepanov)
Pull request description:
Now GUIX build for the `i686-linux-gnu` host is broken, and [there are no plans to re-add it](https://github.com/bitcoin/bitcoin/pull/24448).
ACKs for top commit:
fanquake:
ACK 97af652788
Tree-SHA512: 968181aff65e607a7c1a1b06ac7dfd79f6e2ce49b3c4c3828def020e925769fdbab1859d37ea924ded7632405b30539ac3ec81ac714cb9a01a2f7d5c93301dd9
12cc0201c2 contrib: fix signet miner (sighash mismatch) (Sebastian Falbesoner)
Pull request description:
gruve-p reported that the signet miner doesn't work anymore (see https://github.com/bitcoin/bitcoin/issues/24501#issuecomment-1062088351), failing with the following error of the `walletprocesspsbt` RPC:
```
error code: -22
error message:
Specified sighash value does not match value stored in PSBT
.....
subprocess.CalledProcessError: Command '['bitcoin-cli', '-signet', '-stdin', 'walletprocesspsbt']' returned non-zero exit status 22
```
PSBT signing was changed to use SIGHASH_DEFAULT by default in #22514. The signet miner script sets the sighash type of the created PSBT to SIGHASH_ALL (3 is the per-input type PSBT_IN_SIGHASH_TYPE, following a little-endian 32 unsigned integer of the sighash type):
e04720ec33/contrib/signet/miner (L169-L170)
hence this leads to a sighash mismatch when the `walletprocesspsbt` RPC is called. Fix this by explicitly passing the correct sighash type. The same change was needed in one of our functional tests, see commit d3992669df.
Note that instead of feeding the PSBT via `-stdin` it is directly passed as parameter, as I couldn't figure out a way to pass multiple parameters otherwise (separating by newline also didn't work).
ACKs for top commit:
kallewoof:
ACK 12cc0201c2
ajtowns:
ACK 12cc0201c2 ; code review only
Tree-SHA512: 8509e768e96f85e28c0ca0dc2d35874aa29623febddc46bf90472ec38f38cb3a1b5407c563fd9101d07088775d0fdb18e9137cc38955e847885b83c16591c736
PSBT signing was changed to use SIGHASH_DEFAULT by default in #22514.
The signet miner script sets the sighash type of the created PSBT to
SIGHASH_ALL, hence this leads to a sighash mismatch when the
`walletprocesspsbt` RPC is called. Fix this by explicitly passing the
correct sighash type.
Note that the same change was needed in one of our functional tests,
see commit d3992669df.
Reported by gruve-p.
The macOS and Windows builds do not require a GCC 7 toolchain, and this
is actually causing build issues, i.e #24211. So switch to using a GCC
10 native toolchain for both.
```bash
test3.c: In function 'main':
test3.c:6:21: warning: implicit declaration of function 'CoFreeUnusedLibrariesEx' [-Wimplicit-function-declaration]
6 | CoFreeUnusedLibrariesEx(0,0);
```
d80dc12097 net: Update hardcoded seeds for 23.x (laanwj)
9f27157894 contrib: make-seeds updates for 23.x (laanwj)
Pull request description:
Update hardcoded P2P network seeds for 23.x, and update the generation script and documentation as necessary
Tool output:
```
IPv4 IPv6 Onion Pass
469910 72944 0 Initial
469910 72944 0 Skip entries with invalid address
469910 72944 0 After removing duplicates
469909 72944 0 Skip entries from suspicious hosts
165760 65113 0 Enforce minimal number of blocks
160668 63183 0 Require service bit 1
4951 1376 0 Require minimum uptime
4406 1051 0 Require a known and recent user agent
4307 1031 0 Filter out hosts with multiple bitcoin ports
ERR: Could not resolve ASN for "2001:678:7dc:8::2": The DNS query name does not exist: 8.0.0.0.c.d.7.0.8.7.6.0.1.0.0.2.origin6.asn.cymru.com.
512 134 0 Look up ASNs and limit results per ASN and per net
```.
ACKs for top commit:
achow101:
ACK d80dc12097
jonatack:
ACK d80dc12097 reviewed the changes and ran the README steps
Tree-SHA512: c651b0501cc28d397cc0778eff6aed4273669082d6ef207ce58ce198b443be66532bf1e8d618ccae3ba671ae4cccfd9b4dd2dfebacc97f3c3bd4e9fa58a3d7a3
1513727e2b build, qt: (Re-)sign package (Hennadii Stepanov)
c26a0a5af7 build, qt: Align frameworks with macOS codesign tool requirements (Hennadii Stepanov)
Pull request description:
Fixes#22403
This PR follows Apple [docs](https://developer.apple.com/documentation/macos-release-notes/macos-big-sur-11_0_1-universal-apps-release-notes):
> - New in macOS 11 on Macs with Apple silicon, and starting in macOS Big Sur 11 beta 6, the operating system enforces that any executable must be signed before it’s allowed to run. There isn’t a specific identity requirement for this signature: a simple ad-hoc signature is sufficient...
> - ... If you use a custom workflow involving tools that modify a binary after linking (e.g. `strip` or `install_name_tool`) you might need to manually call `codesign` as an additional build phase to properly ad-hoc sign your binary. These new signatures are not bound to the specific machine that was used to build the executable, they can be verified on any other system and will be sufficient to comply with the new default code signing requirement on Macs with Apple silicon...
When building with system Qt frameworks (i.e., without depends), a new string has been added to the `make deploy` log on M1-based macOS:
```
% make deploy
...
+ Generating .DS_Store +
dist/Bitcoin-Qt.app: replacing existing signature
+ Preparing .dmg disk image +
...
```
This PR does not change build system behavior:
- when building with depends
- on Intel-based macOS
ACKs for top commit:
jarolrod:
ACK 1513727e2b
fanquake:
ACK 1513727e2b - although didn't test on M1 hardware. Given the forced signing is scoped to only occur when running the deploy script on macOS, this doesn't interfere with our release signing.
Tree-SHA512: 3aa778fdd6ddb54f029f632f2fe52c2ae3bb197ba564cb776493aa5c3a655bd51d10ccbe6c007372d717e9b01fc4193dd5c29ea0bc7e069dcae7e991ae259f0c
Documentation:
- Use https URL for bitcoin.sipa.be (http sends a redirect, fooling
curl).
- Add explicit step to add manual seeds.
Code:
- Change PATTERN_ONION to v3 (effectively means that the no onion hosts
are delivered).
- Add versions to PATTERN_AGENT filter.
- Print specific message on resolve exception.
48742693ac Replace "can not" with "cannot" in docs, user messages, and tests (Jon Atack)
e670edd434 User-facing content fixups from transifex translator feedback (Jon Atack)
Pull request description:
Closes#24366.
ACKs for top commit:
laanwj:
Code review re-ACK 48742693ac
hebasto:
re-ACK 48742693ac, only suggested change since my previous [review](https://github.com/bitcoin/bitcoin/pull/24367#pullrequestreview-885938219).
Tree-SHA512: 4dcdcb417251a413e65fab6070515e13a1267c8e0dbcf521386b842511391f24c84a0c2168fe13458c977682034466509bf2a3453719d4d94d3c568fd9f4adb4
87f54060ff doc: Swap gen-manpages and update RC steps in release process (laanwj)
42c202893b doc: Fix gen-manpages, rewrite in Python (laanwj)
Pull request description:
Rewrite the manual page generation script in Python.
This:
- solves '-' stripping issue (fixes#22681)
- makes that a copyright footer is generated correctly again
Also change the release process to swap gen-manpages and update RC steps, so that the pages will have the correct rc and/or final version.
ACKs for top commit:
dongcarl:
Code Review ACK 87f54060ff
fanquake:
ACK 87f54060ff - tested generating and opening the man pages locally, but didn't run through the release process. Will propose some changes to address consolidating the help / version output.
Tree-SHA512: 39254721ca84e4f223a321c554f2e08c36428b15019a0f9fa3eff408b4c6f1e1d74941143f4d2927427afa3ad7a7e6f999d6ec660132d817809b640a87ae9f7d
799968e8b3 tracing: misc follow-ups to 22902 (0xb10c)
36a6584703 tracing: correctly scope utxocache:flush tracepoint (Arnab Sen)
Pull request description:
This PR is a follow-up to the [#22902](https://github.com/bitcoin/bitcoin/pull/22902).
Previously, the tracepoint `utxocache:flush` was called, even when it was not flushing. So, the tracepoint is now scoped to write only when coins cache to disk.
ACKs for top commit:
0xB10C:
ACK 799968e8b3
Tree-SHA512: ebb096cbf991c551c81e4339821f10d9768c14cf3d8cb14d0ad851acff5980962228a1c746914c6aba3bdb27e8be53b33349c41efe8bab5542f639916e437b5f
- mention 'Lost X events' workaround
- clarify flush tracepoint docs
- fix typo in tracepoint context
- clarify flush for prune
The documentation and examples for the `fFlushForPrune` argument
of the utxocache flush tracepoint weren't clear without looking
at the code.
See these comments: https://github.com/bitcoin/bitcoin/pull/22902#issuecomment-987094612
- doc: note that there can be temporary UTXO caches
Bitcoin Core uses temporary clones of it's _main_ UTXO cache in some
places. The utxocache:add and :spent tracepoints are triggered when
temporary caches are changed too. This is documented.
Switching the endianness of a hex string `str` can simply be
achieved by `bytes.fromhex(str)[::-1].hex()`, i.e. we can use
that and remove those helper methods.
ddcac22f09 doc: cleanup doc on need of Developer Account to obtain macOS SDK (jarolrod)
Pull request description:
The explicit statement that an Apple Developer Account is required in order to obtain the SDK is buried within the `Deterministic macOS DMG Notes` section. It should be the first thing mentioned under the `SDK Extraction` section.
The reason to do this is to set expectations of what is required before starting any steps or clicking on links.
This fixes the issue by doing just that; moving this information to the `SDK Extraction` section. Now that the information is moved, this also deletes unnecessary SDK related notes from the `Deterministic macOS DMG Notes` section. It is not necessary to explain under what sub-directory 'most' of the important files are inside of the `Xcode.app`.
Note that this also fixes a missed Xcode version link bump by deleting it :)
ACKs for top commit:
fanquake:
ACK ddcac22f09
Tree-SHA512: 9fe7fddd66b68a0475be8b0c78cb58932bf5b68d962ece36a86f3b743a88d8561c0ec3e8d88bcaf338da7ab9d3dfcfb9399f6e1a884d83c8f3117c186d049469
The explicit statement that an Apple Developer Account is required in
order to obtain the SDK is buried within the "Deterministic macOS DMG
Notes" section. It should be the first thing mentioned under the "SDK
Extraction" section. The reason to do this is to set expectations of
what is required before starting any steps or clicking on links.
This fixes the issue by doing just that; moving this information to the
"SDK Extraction" section. Now that the information is moved, this also
deletes unnecessary SDK related notes from the "Deterministic macOS DMG
Notes" section. It is not necessary to explain under what sub-directory
'most' of the important files are inside of the 'Xcode.app'.
c73415bc10 build: Fix xargs warnings for Guix builds (Hennadii Stepanov)
Pull request description:
On master (e3ce019667) there are warnings in `./contrib/guix/guix-build` logs:
```
xargs: warning: options --max-args and --replace/-I/-i are mutually exclusive, ignoring previous --max-args value
```
This PR fixes such warnings.
ACKs for top commit:
prusnak:
utACK c73415b
Tree-SHA512: a7b55f59afbb19b78f795cea64acacf29903cfcd5fd7c37a771b073c1f2ff54555a26f3d00c1c73a8ef588396217ddf598e32b2ae961559042cc051b0aad162a
======================================================================
ERROR: test_revocation_mode_soft (tests.test_validate.ValidateTests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/tmp/guix-build-python-certvalidator-0.1-1.e5bdb4b.drv-0/source/tests/test_validate.py", line 85, in test_revocation_mode_soft
validate_path(context, path)
File "/tmp/guix-build-python-certvalidator-0.1-1.e5bdb4b.drv-0/source/tests/../certvalidator/validate.py", line 50, in validate_path
return _validate_path(validation_context, path)
File "/tmp/guix-build-python-certvalidator-0.1-1.e5bdb4b.drv-0/source/tests/../certvalidator/validate.py", line 358, in _validate_path
raise PathValidationError(pretty_message(
certvalidator.errors.PathValidationError: The path could not be validated because the end-entity certificate expired 2022-01-14 12:00:00Z
e09773d20a build: use a static .tiff for macOS .dmg over generating (fanquake)
Pull request description:
For demonstration, after [discussion in #23778](https://github.com/bitcoin/bitcoin/pull/23778#issuecomment-1003005503), and the question as to why we can't just have a `background.tiff` that we copy into the macOS DMG, and do away with the somewhat convoluted image generation steps.
From my understanding, the only reason we have this image generation as part of our build system is so that forks of Core can adapt the imagery for their own branding via `PACKAGE_NAME`. It don't think it provides much value to us, and could just have a static .tiff that we copy into the dmg (replacing the .svg that currently lives in macdeploy/).
Doing this would eliminate the following build dependencies:
For native macOS:
* `sed` (usage in Makefile.am)
* `librsvg` (rsvg-convert)
* `tiffutil`
Linux macOS cross-compile:
* `sed` (usage in Makefille.am)
* `librsvg`
* `tiffcp`
* `convert` (imagemagick)
* `font-tuffy`
Guix Build:
```bash
bash-5.1# find guix-build-$(git rev-parse --short=12 HEAD)/output/ -type f -print0 | env LC_ALL=C sort -z | xargs -r0 sha256sum
c98d67796863f4b1bab0ad600d46bd74e744d94072cbd4bc856a6aeaba3bb329 guix-build-e09773d20a92/output/dist-archive/bitcoin-e09773d20a92.tar.gz
3336f90bab312798cb7665e2b4ae24d1a270fb240647d5fed8dbfcd83e3ed37e guix-build-e09773d20a92/output/x86_64-apple-darwin/SHA256SUMS.part
8fd680c7ee158c64bad212385df7b0b302c6c2143d4e672b4b0eb5da41f9256d guix-build-e09773d20a92/output/x86_64-apple-darwin/bitcoin-e09773d20a92-osx-unsigned.dmg
34f54177c2f0700e8cfaf5d85d91e404807cd9d411e22006cdff82653e5f4af2 guix-build-e09773d20a92/output/x86_64-apple-darwin/bitcoin-e09773d20a92-osx-unsigned.tar.gz
da6b8f54ef755d40330c8eac4f5bd0329637e827be9ee61318600d5d0bdcc3dc guix-build-e09773d20a92/output/x86_64-apple-darwin/bitcoin-e09773d20a92-osx64.tar.gz
```
ACKs for top commit:
hebasto:
ACK e09773d20a
jarolrod:
ACK e09773d20a
Zero-1729:
ACK e09773d20a
Tree-SHA512: 0ad06699a5451daa8cfaaa46759eb7bd85254a72e23f857f70d433a2ffb1a4bf6dd464d9c4ac9f8c20aab045f4e2b61c6dcdcbcceef96ce515b1a0c501665b1f
Previously, the `utxocache:flush` tracepoint was in the wrong scope and
reached every time `CChainState::FlushStateToDisk` was called, even when
there was no flushing of the cache. The tracepoint is now properly scoped
and will be reached during a full flush.
Inside the scope, the `fDoFullFlush` value will always be `true`, so it
doesn't need to be logged separately. Hence, it's dropped from the
tracepoint arguments.
b062da0090 contrib: add check for wget command in install_db4.sh (Florian Baumgartl)
Pull request description:
This PR is motivated by 7bb8eb0bc3 commit (see also https://github.com/bitcoin/bitcoin/pull/23579) and ensures that `install_db4.sh` will check for `curl` and `wget` utilities. Currently, the conditional statement in the `http_get()` function assumes that `wget` is always available but we actually do not know it since there is no check or validation for the `wget` command. So let's make sure that we check for both commands and print an error message if they are missing.
ACKs for top commit:
jamesob:
ACK b062da0090
laanwj:
Tested ACK b062da0090
shaavan:
ACK b062da0090
Tree-SHA512: bfc1ccad9a5b99764b759e02dde1976616c2af4747b7d5af8e71d33624c2cb21d93a09a60d244756e86bbd5fd7541331c62d7eb84d3458b6a059f1d9cb2a5f42
2f356a0ca8 scripted-diff: Drop Darwin version for better maintainability (Hennadii Stepanov)
Pull request description:
After this PR, any macOS tools version bumping in the future will touch fewer files in the repo.
Pointing a Darwin version for the `--host` system does not matter for the following reasons:
- in terms of the resulted binaries, we should only care about the minimum supported macOS version which is a separated parameter in our build system.
- in terms of the build system itself, the usage of the `$(host)` variable is self-consistent enough. Btw `$(host_os)` value already has the version dropped:
```
$ make -C depends --no-print-directory print-host_os HOST=x86_64-apple-darwin19
host_os=darwin
```
ACKs for top commit:
gruve-p:
ACK 2f356a0ca8
promag:
ACK 2f356a0ca8.
fanquake:
ACK 2f356a0ca8
Tree-SHA512: 374896ab0ba02b0d8b4b21431fe963bd213b0d09586e0898c13a4c5fa294c1b693f1b2c92880c245c4157c14217b4825b36522f461930477f4d2a727086ebb2a
a3f61676e8 test: Make more shell scripts verifiable by the `shellcheck` tool (Hennadii Stepanov)
Pull request description:
Some shell scripts from `contrib/guix` and `contrib/shell` are not verifiable by the `shellcheck` tool for the following reasons:
- they have no extension (see 4eccf063b2 from bitcoin/bitcoin#21375)
- they have the `.bash` extension while `.sh` is expected
This PR adds these scripts to the input for the `shellcheck` tool, and it fixes discovered `shellcheck` warnings.
ACKs for top commit:
dongcarl:
Code Review ACK a3f61676e8, this is a good robustness improvement for our shell scripts.
jamesob:
crACK a3f61676e8
Tree-SHA512: 6703f5369d9c04c1a174491f381afa5ec2cc4d37321c1b93615abcdde4dfd3caae82868b699c25b72132d8c8c6f2e9cf24d38eb180ed4d0f0584d8c282e58935
Fix the warning:
```
./contrib/macdeploy/gen-sdk:84: FutureWarning: GzipFile was opened for writing, but this will change in future Python releases. Specify the mode argument for opening it for writing.
```
From what I can see the only platform this drops support for is CentOS
7. CentOS 7 reached the end of it's "full update" support at the end of
2020. It does receive maintenance updates until 2024, however I don't
think supporting glibc 2.17 until 2024 is realistic. Note that anyone
wanting to self-compile and target a glibc 2.17 runtime could build with
--disable-threadlocal.
glibc 2.18 was released in August 2013.
https://sourceware.org/legacy-ml/libc-alpha/2013-08/msg00160.html
365f35481d script: Add commits signed with sipas expired key to allow-revsig-commits (nthumann)
Pull request description:
Fixes https://github.com/bitcoin/bitcoin/issues/22737.
While investigating the issue above, I noticed that there are 141 commits that are signed with sipas expired key.
To allow `./contrib/verify-commits/verify-commits.py` to succeed, this PR adds them to `allow-revsig-commits`.
Feel free to confirm that they're indeed signed with an expired key using e.g. `git show --show-signature d8cd7b137fb075616f31d2b43b85fa2e27ea7477` :)
ACKs for top commit:
laanwj:
Code review ACK 365f35481d
Tree-SHA512: 860e372c5314714c6c205cd234ebec89756c9ade43a2ed65ed25575ae0a0d4d8dd7cf43692a5b267abe742f87e5cba0a3f1fb76a5fed7b1615ea2859902dfcdf
29173d6c6c ubsan: add minisketch exceptions (Cory Fields)
54b5e1aeab Add thin Minisketch wrapper to pick best implementation (Pieter Wuille)
ee9dc71c1b Add basic minisketch tests (Pieter Wuille)
0659f12b13 Add minisketch dependency (Gleb Naumenko)
0eb7928ab8 Add MSVC build configuration for libminisketch (Pieter Wuille)
8bc166d5b1 build: add minisketch build file and include it (Cory Fields)
b2904ceb85 build: add configure checks for minisketch (Cory Fields)
b6487dc4ef Squashed 'src/minisketch/' content from commit 89629eb2c7 (fanquake)
Pull request description:
This takes over #21859, which has [recently switched](https://github.com/bitcoin/bitcoin/pull/21859#issuecomment-921899200) to my integration branch. A few more build issues came up (and have been fixed) since, and after discussing with sipa it was decided I would open a PR to shepherd any final changes through.
> This adds a `src/minisketch` subtree, taken from the master branch of https://github.com/sipa/minisketch, to prepare for Erlay implementation (see #21515). It gets configured for just supporting 32-bit fields (the only ones we're interested in in the context of Erlay), and some code on top is added:
> * A very basic unit test (just to make sure compilation & running works; actual correctness checking is done through minisketch's own tests).
> * A wrapper in `minisketchwrapper.{cpp,h}` that runs a benchmark to determine which field implementation to use.
Only changes since my last update to the branch in the previous PR have been rebasing on master and fixing an issue with a header in an introduced file.
ACKs for top commit:
naumenkogs:
ACK 29173d6c6c
Tree-SHA512: 1217d3228db1dd0de12c2919314e1c3626c18a416cf6291fec99d37e34fb6eec8e28d9e9fb935f8590273b8836cbadac313a15f05b4fd9f9d3024c8ce2c80d02
An IPv6 address from fc00::/8 could be either from the CJDNS network or
from a private-unroutable-reserved segment of IPv6. A seed node with
such an address must be from the CJDNS network, otherwise other peers
will not be able to connect to it.
0f95247246 Integrate univalue into our buildsystem (Cory Fields)
9b49ed656f Squashed 'src/univalue/' changes from 98fadc0909..a44caf65fe (fanquake)
Pull request description:
This PR more tightly integrates building Univalue into our build system. This follows the same approach we use for [LevelDB](https://github.com/bitcoin-core/leveldb/), ([`Makefile.leveldb.include`](https://github.com/bitcoin/bitcoin/blob/master/src/Makefile.leveldb.include)), and [CRC32C](https://github.com/bitcoin-core/crc32c) ([`Makefile.crc32c.include`](https://github.com/bitcoin/bitcoin/blob/master/src/Makefile.crc32c.include)), and will be the same approach we use for [minisketch](https://github.com/sipa/minisketch); see #23114.
This approach yields a number of benefits, including:
* Faster configuration due to one less subconfigure being run during `./configure` i.e 22s with this PR vs 26s
* Faster autoconf i.e 13s with this PR vs 17s
* Improved caching
* No more issues with compiler flags i.e https://github.com/bitcoin/bitcoin/pull/12467
* More direct control means we can build exactly the objects we want
There might be one argument against making this change, which is that builders should have the option to use "proper shared/system libraries". However, I think that falls down for a few reasons. The first being that we already don't support building with a number of system libraries (secp256k1, leveldb, crc32c); some for good reason. Univalue is really the odd one out at the moment.
Note that the only fork of Core I'm aware of, that actively patches in support for using system libs, also explicitly marks them as ["DANGEROUS"](a886811721/configure.ac (L1430)) and ["NOT SUPPORTED"](a886811721/configure.ac (L1312)). So it would seem they exist more to satisfy a distro requirement, as opposed to something that anyone should, or would actually use in practice.
PRs like #22412 highlight the "issue" with us operating with our own Univalue fork, where we actively fix bugs, and make improvements, when upstream (https://github.com/jgarzik/univalue) may not be taking those improvements, and by all accounts, is not currently actively maintained. Bitcoin Core should not be hamstrung into not being able to fix bugs in a library, and/or have to litter our source with "workarounds", i.e #22412, for bugs we've already fixed, based on the fact that an upstream project is not actively being maintained. Allowing builders to use system libs is really only exacerbating this problem, with little benefit to our project. Bitcoin Core is not quite like your average piece of distro packaged software.
There is the potential for us to give the same treatment to libsecp256k1, however it seems doing that is currently less straightforward.
ACKs for top commit:
dongcarl:
ACK 0f95247246 less my comment above, always nice to have an include-able `sources.mk` which makes integration easier.
theuni:
ACK 0f95247246. Thanks fanquake for keeping this going.
Tree-SHA512: a7f2e41ee7cba06ae72388638e86b264eca1b9a8b81c15d1d7b45df960c88c3b91578b4ade020f8cc61d75cf8d16914575f9a78fa4cef9c12be63504ed804b99
The tracepoint `validation:block_connected` was introduced in #22006.
The first argument was the hash of the connected block as a pointer
to a C-like String. The last argument passed the hash of the
connected block as a pointer to 32 bytes. The hash was only passed as
string to allow `bpftrace` scripts to print the hash. It was
(incorrectly) assumed that `bpftrace` cannot hex-format and print the
block hash given only the hash as bytes.
The block hash can be printed in `bpftrace` by calling
`printf("%02x")` for each byte of the hash in an `unroll () {...}`.
By starting from the last byte of the hash, it can be printed in
big-endian (the block-explorer format).
```C
$p = $hash + 31;
unroll(32) {
$b = *(uint8*)$p;
printf("%02x", $b);
$p -= 1;
}
```
See also: https://github.com/bitcoin/bitcoin/pull/22902#discussion_r705176691
This is a breaking change to the block_connected tracepoint API, however
this tracepoint has not yet been included in a release.
It is important that binaries request a standard interpreter location
where most distros would place the linker-loader. Otherwise, the user
would be met with a very confusing message:
bash: <path>/<to>/bitcoind: No such file or directory
When really it's the interpreter that's not found.
I used Guix's values for the powerpc64(le) dynamic linkers, and the
/lib-prefix seems to be a Guix-ism rather than standard. The standard
path for the linker-loaders start with /lib64.
I've taken the new loader values from SYSDEP_KNOWN_INTERPRETER_NAMES in
glibc's sysdeps/unix/sysv/linux/powerpc/ldconfig.h file.
For future reference, loader path values can also be found on glibc's
website: https://sourceware.org/glibc/wiki/ABIList?action=recall&rev=16
These test-*-check scripts should compile "test" binaries in a way that
is as close to what autotools would do, since the goal is to make sure
that if we run the *-check script, they can correctly detect flaws in
binaries which are compiled by our autotools-based system.
Therefore, we should emulate what happens when the binary is linked in
autotools, meaning that for C binaries, we need to supply the CFLAGS,
CPPFLAGS, and LDFLAGS flags in that order.
Note to future developers: perhaps it'd be nice to have these
test-*-check scripts be part of configure.ac to avoid having to manually
replicate autoconf-like behaviour every time we find a discrepancy. Of
course, that would also mean you'd have to write more m4...
This addresses issues like the one in #12467, where some of our compiler flags
end up being dropped during the subconfigure of Univalue. Specifically, we're
still using the compiler-default c++ version rather than forcing c++17.
We can drop the need subconfigure completely in favor of a tighter build
integration, where the sources are listed separately from the build recipes,
so that they may be included directly by upstream projects. This is
similar to the way leveldb build integration works in Core.
Core benefits of this approach include:
- Better caching (for ex. ccache and autoconf)
- No need for a slow subconfigure
- Faster autoconf
- No more missing compile flags
- Compile only the objects needed
There are no benefits to Univalue itself that I can think of. These changes
should be a no-op there, and to downstreams as well until they take advantage
of the new sources.mk.
This also removes the option to use an external univalue to avoid similar ABI
issues with mystery binaries.
Co-authored-by: fanquake <fanquake@gmail.com>
As the faucet will always ask for a captcha now, the current script is
no longer usable.
Change the script to print the captcha in dot-matrix to the terminal,
using unicode Braille characters.
a43b8e9555 build: set OSX_MIN_VERSION to 10.15 (fanquake)
Pull request description:
Taken out of #20744, as splitting up some of the build changes was mentioned [here](https://github.com/bitcoin/bitcoin/pull/22937#discussion_r707303172).
This is required to use `std::filesystem` on macOS, as support for it only landed in the libc++.dylib shipped with 10.15. So if we want to move to using `std::filesystem` for `23.0`, this bump is required.
See also: https://developer.apple.com/documentation/xcode-release-notes/xcode-11-release-notes
> Clang now supports the C++17 \<filesystem\> library for iOS 13, macOS 10.15, watchOS 6, and tvOS 13.
macOS 10.15 was released in October 2019. macOS OS's seem to have a life of about 3 years, so it's possible that 10.14 will become officially unsupported by the end of 2021 and prior to the release of 23.0.
Guix builds:
```bash
bash-5.1# find guix-build-$(git rev-parse --short=12 HEAD)/output/ -type f -print0 | env LC_ALL=C sort -z | xargs -r0 sha256sum
abc8b749be65f1339dcdf44bd1ed6ade2533b8e3b5030ad1dde0ae0cede78136 guix-build-a43b8e955558/output/dist-archive/bitcoin-a43b8e955558.tar.gz
1edcc301eb4c02f3baa379beb8d4c78e661abc24a293813bc9d900cf7255b790 guix-build-a43b8e955558/output/x86_64-apple-darwin19/SHA256SUMS.part
e9dbb5594a664519da778dde9ed861c3f0f631525672e17a67eeda599f16ff44 guix-build-a43b8e955558/output/x86_64-apple-darwin19/bitcoin-a43b8e955558-osx-unsigned.dmg
11b23a17c630dddc7594c25625eea3de42db50f355733b9ce9ade2d8eba3a8f3 guix-build-a43b8e955558/output/x86_64-apple-darwin19/bitcoin-a43b8e955558-osx-unsigned.tar.gz
257ba64a327927f94d9aa0a68da3a2695cf880b3ed1a0113c5a966dcc426eb5e guix-build-a43b8e955558/output/x86_64-apple-darwin19/bitcoin-a43b8e955558-osx64.tar.gz
```
ACKs for top commit:
hebasto:
ACK a43b8e9555
jarolrod:
ACK a43b8e9
Tree-SHA512: 9ac77be7cb56c068578860a3b2b8b7487c9e18b71b14aedd77a9c663f5d4bb19756d551770c02ddd12f1797beea5757b261588e7b67fb53509bb998ee8022369
ab9c34237a release: remove gitian (fanquake)
Pull request description:
Note that this doesn't yet touch any glibc back compat related code.
ACKs for top commit:
laanwj:
Code review ACK ab9c34237a
Tree-SHA512: 8e2fe3ec1097f54bb11ab9136b43818d90eab5dbb0a663ad6a552966ada4bdb49cc12ff4e66f0ec0ec5400bda5c81f3a3ce70a9ebb6fe1e0db612da9f00a51a7
96cc6bb04f guix/prelude: Override VERSION with FORCE_VERSION (Carl Dong)
Pull request description:
```
Previously, if the builder exported $VERSION in their environment (as
past Gitian-building docs told them to), but their HEAD does not
actually point to v$VERSION, their build outputs will differ from those
of other builders.
This is because the contrib/guix/guix-* scripts only ever act on the
current git worktree, and does not try to check out $VERSION if $VERSION
is set in the environment.
Setting $VERSION only makes the scripts pretend like the current
worktree is $VERSION.
This problem was seen in jonatack's attestation for all.SHA256SUMS,
where only his bitcoin-22.0rc3-osx-signed.dmg differed from everyone
else's.
Here is my deduced sequence of events:
1. Aug 27th: He guix-builds 22.0rc3 and uploads his attestations up to
guix.sigs
2. Aug 30th, sometime after POSIX time 1630310848: he pulls the latest
changes from master in the same worktree where he guix-built 22.0rc3
and ends up at 7be143a960
3. Aug 30th, sometime before POSIX time 1630315907: With his worktree
still on 7be143a960, he guix-codesigns. Normally, this would result
in outputs going in guix-build-7be143a960e2, but he had
VERSION=22.0rc3 in his environment, so the guix-* scripts pretended
like he was building 22.0rc3, and used 22.0rc3's guix-build directory
to locate un-codesigned outputs and dump codesigned ones.
However, our SOURCE_DATE_EPOCH defaults to the POSIX time of HEAD
(7be143a960), which made all timestamps in the resulting codesigned
DMG 1630310848, 7be143a960e2's POSIX timestamp. This differs from the
POSIX timestamp of 22.0rc3, which is 1630348517. Note that the
windows codesigning procedure does not consider SOURCE_DATE_EPOCH.
We resolve this by only allowing VERSION overrides via the FORCE_VERSION
environment variable.
```
Please ignore the branch name, it's not relevant to the change.
ACKs for top commit:
fanquake:
ACK 96cc6bb04f - Also makes sense given there are Guix build guides recommending to set `VERSION` as part of the process. i.e https://gist.github.com/hebasto/7293726cbfcd0b58e1cfd5418316cee3.
Tree-SHA512: 9dca3fc637ce11049286a3ebee3cd61cce2125fc51d31cf472fbed7f659e1846fc44062753e0e71bfaec9e7fbab6f040bb88d9d4bc4f8acb28c6890563584acf
Previously, if the builder exported $VERSION in their environment (as
past Gitian-building docs told them to), but their HEAD does not
actually point to v$VERSION, their build outputs will differ from those
of other builders.
This is because the contrib/guix/guix-* scripts only ever act on the
current git worktree, and does not try to check out $VERSION if $VERSION
is set in the environment.
Setting $VERSION only makes the scripts pretend like the current
worktree is $VERSION.
This problem was seen in jonatack's attestation for all.SHA256SUMS,
where only his bitcoin-22.0rc3-osx-signed.dmg differed from everyone
else's.
Here is my deduced sequence of events:
1. Aug 27th: He guix-builds 22.0rc3 and uploads his attestations up to
guix.sigs
2. Aug 30th, sometime after POSIX time 1630310848: he pulls the latest
changes from master in the same worktree where he guix-built 22.0rc3
and ends up at 7be143a960
3. Aug 30th, sometime before POSIX time 1630315907: With his worktree
still on 7be143a960, he guix-codesigns. Normally, this would result
in outputs going in guix-build-7be143a960e2, but he had
VERSION=22.0rc3 in his environment, so the guix-* scripts pretended
like he was building 22.0rc3, and used 22.0rc3's guix-build directory
to locate un-codesigned outputs and dump codesigned ones.
However, our SOURCE_DATE_EPOCH defaults to the POSIX time of HEAD
(7be143a960), which made all timestamps in the resulting codesigned
DMG 1630310848, 7be143a960e2's POSIX timestamp. This differs from the
POSIX timestamp of 22.0rc3, which is 1630348517. Note that the
windows codesigning procedure does not consider SOURCE_DATE_EPOCH.
We resolve this by only allowing VERSION overrides via the FORCE_VERSION
environment variable.
42dbd9025a contrib: return non-zero status if getcoins.py errors (Sebastian Falbesoner)
8c203cf0e1 contrib: catch bitcoin-cli RPC call errors in getcoins.py (Sebastian Falbesoner)
0eca5ebace contrib: refactor: introduce bitcoin-cli RPC call helper in getcoins.py (Sebastian Falbesoner)
Pull request description:
This PR is based on #22565 ("[script] signet's getcoins.py improvements"), which should be reviewed first.
The signet faucet script `contrib/signet/getcoins.py` currently issues bitcoin-cli RPC calls without catching errors -- the only case tackled is if there is no `bitcoin-cli` file found. Instead of crashing with a stack-trace on a failed RPC call, the changes in this PR aim to produce a more user-friendly output (see also https://github.com/bitcoin/bitcoin/pull/22565#discussion_r683754875). Additionally, in case of any error, a non-zero status is now returned (instead of 0, indicating success), which could be useful for other scripts taking use of signet faucet script.
The most straight-forward way to test this is invoking the script without a `bitcoind` running on signet:
PR22565 branch:
```
$ ./contrib/signet/getcoins.py
error: Could not connect to the server 127.0.0.1:8332
Make sure the bitcoind server is running and that you are connecting to the correct RPC port.
Traceback (most recent call last):
File "./contrib/signet/getcoins.py", line 26, in <module>
curr_signet_hash = subprocess.check_output([args.cmd] + args.bitcoin_cli_args + ['getblockhash', '1']).strip().decode()
File "/usr/local/lib/python3.8/subprocess.py", line 415, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "/usr/local/lib/python3.8/subprocess.py", line 516, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['bitcoin-cli', 'getblockhash', '1']' returned non-zero exit status 1.
```
this PR branch:
```
$ ./contrib/signet/getcoins.py
error: Could not connect to the server 127.0.0.1:38332
Make sure the bitcoind server is running and that you are connecting to the correct RPC port.
-----
Error while calling "bitcoin-cli -signet getblockhash 1" (see output above).
```
ACKs for top commit:
kallewoof:
Code ACK 42dbd9025a
Zero-1729:
tACK 42dbd90🧪
Tree-SHA512: 912240a4ed03c87035e370602f4095c7ffe26806421bbbd6cf86588126f2310a01a6a61606e9e2918fb2c1a0debdd0ce768c69ba2e4b8e7750fa3474a56d01a0
b0c8246cac Add cleaner errors for unsuccessful faucet transactions (NikhilBartwal)
1c612b274b [script] Update signet getcoins.py for custom network (NikhilBartwal)
Pull request description:
Currently, using the getcoins.py with a custom signet executes successfully and shows the transfer of 0.001 testBTC as complete, however for obvious reasons, it should not. In fact, upon verification it does not actually execute the transaction, but rather gives the output that it did, as shown below which can be misleading:
```
[nikhilb@nikhil-PC bitcoin]$ echo $datadir
/home/nikhilb/signet-custom
[nikhilb@nikhil-PC bitcoin]$ contrib/signet/getcoins.py -- -datadir=$datadir
Payment of 0.00100000 BTC sent with txid dd22c7d996e95f3e5baf20f73140d517ff48f1b26d0e4fefd61e3c37991b8f86
[nikhilb@nikhil-PC bitcoin]$ bitcoin-cli -datadir=$datadir getrawtransaction dd22c7d996e95f3e5baf20f73140d517ff48f1b26d0e4fefd61e3c37991b8f86
error code: -5
error message:
No such mempool or blockchain transaction. Use gettransaction for wallet transactions.
[nikhilb@nikhil-PC bitcoin]$ bitcoin-cli -datadir=$datadir gettransaction dd22c7d996e95f3e5baf20f73140d517ff48f1b26d0e4fefd61e3c37991b8f86
error code: -5
error message:
Invalid or non-wallet transaction id
```
This PR adds a sanity check for custom signet by comparing the current network's first block hash (the block after the genesis block) with global signet's respective block hash (since all signet networks share the same genesis block) and if a custom network is detected, the user is prompted to either work on the global signet or setup their own faucet.
The PR was checked to be working successfully, giving the output as below:
```
[nikhilb@nikhil-PC bitcoin]$ git checkout update_signet_getcoins
Switched to branch 'update_signet_getcoins'
Your branch is ahead of 'upstream/master' by 1 commit.
(use "git push" to publish your local commits)
[nikhilb@nikhil-PC bitcoin]$ contrib/signet/getcoins.py -- -datadir=$datadir
The global faucet cannot be used with a custom Signet network. Please use the global signet or setup your custom faucet for the same.
You can have a look here for setting up your own faucet: https://en.bitcoin.it/wiki/Signet
```
ACKs for top commit:
prayank23:
utACK b0c8246cac
kallewoof:
ACK b0c8246cac
arnabsen1729:
utACK b0c8246
prakash1512:
utACK b0c8246
0xB10C:
Tested ACK b0c8246cac
theStack:
Tested ACK b0c8246cac
Zero-1729:
crACK b0c8246🧉
Tree-SHA512: 144b47a83008521a5cda13f4c1b12809a125a744f865a8e0f792132d52fdb88926d4f4f4d7230452c2e129b5879892cdbeda981b8af10b789e9fc0cda2905a5d
132cae44f2 doc: Mention the flat directory structure for uploads (Andrew Chow)
fb17c99e35 guix: Don't include directory name in SHA256SUMS (Andrew Chow)
Pull request description:
The SHA256SUMS file can be used in a sha256sum -c command to verify downloaded binaries. However users are likely to download just a single file and not place this file in the correct directory relative to the SHA256SUMS file for the simple verification command to work. By not including the directory name in the SHA256SUMS file, it will be easier for users to verify downloaded binaries.
ACKs for top commit:
Zero-1729:
re-ACK 132cae44f2
fanquake:
ACK 132cae44f2
Tree-SHA512: c9ff416b8dfb2f3ceaf4d63afb84aac9fcaefbbf9092f9e095061b472884ec92c7a809e6530c7132a82cfe3ab115a7328e47994a412072e1d4feb26fc502c8c5
The SHA256SUMS file can be used in a sha256sum -c command to verify
downloaded binaries. However users are likely to download just a single
file and not place this file in the correct directory relative to the
SHA256SUMS file for the simple verification command to work. By not
including the directory name in the SHA256SUMS file, it will be easier
for users to verify downloaded binaries.
Co-authored-by: Carl Dong <contact@carldong.me>
021daedfa1 refactor: replace remaining binascii method calls (Zero-1729)
Pull request description:
This PR removes the remaining `binascii` method calls outside `test/functional` and `test_framework`, as pointed out here https://github.com/bitcoin/bitcoin/pull/22619#pullrequestreview-722153458.
Follow-up to #22593 and #22619Closes#22605
ACKs for top commit:
josibake:
re-ACK 021daedfa1
theStack:
re-ACK 021daedfa1
Tree-SHA512: 2ae9fee8917112c91a5406f219ca70f24cd8902b903db5a61fc2de85ad640d669a772f5c05970be0fcee6ef1cdd32fae2ca5d1ec6dc9798b43352c8160ddde6f
5449d44e37 scripts: prevent GCC optimising test symbols in test-symbol-check (fanquake)
Pull request description:
I noticed in #22381 that when the test-symbol-check target was being built with Clang and run in the CI it would fail due to using a too-new version of `pow` (used [here](d67330d112/contrib/devtools/test-symbol-check.py (L85))). Our CIs use Focal (glibc 2.31) and the version of `pow` was the optimized version introduced in [glibc 2.29](https://lwn.net/Articles/778286/):
```bash
* Optimized generic exp, exp2, log, log2, pow, sinf, cosf, sincosf and tanf.
```
This made sense, except for that if it was failing when built using Clang, why hadn't it also been failing when being built with GCC?
Turns out GCC is optimizing away that call to `pow` at all optimization levels, including `-O0`, see: https://godbolt.org/z/53MhzMxT7, and this has been the case forever, or at least since GCC 5.x. Clang on the other hand, will only optimize away the `pow` call at `-O1` and `-O2`, not `-O0`: https://godbolt.org/z/Wbnqj3q6c. Thus when this test was built with Clang (we don't pass `-O` so we default to `-O0`) it was failing in the CI environment, because it would actually have a call to the "new" `pow`.
Avoid this issue by using a symbol that won't be optimized away, or that we are unlikely to ever have versioning issues with.
ACKs for top commit:
laanwj:
ACK 5449d44e37
Tree-SHA512: 3a26c5c3a5f2905fd0dd90892470e241ba625c0af3be2629d06d5da3a97534c1d6a55b796bbdd41e2e6a26a8fab7d981b98c45d4238565b0eb7edf3c5da02007
7d95777417 builder-keys: Add dongcarl (Carl Dong)
Pull request description:
https://keys.openpgp.org/search?q=04017A2A6D9A0CCDC81D8EC296AB007F1A7ED999
This is my master key, will be bumping the expiration of subkeys or rotating when necessary.
ACKs for top commit:
fanquake:
ACK 7d95777417 - matches what I've got.
Tree-SHA512: 3a76b8eda81821b3221402501cf8191bce73118624b932aa80a7fc1a32a91e3825aeb2b03ed261bbf284b088e927c384f92e08eadddf7f94ed4de579d9f6d2b7
90b3e482e9 release: Release with separate SHA256SUMS and sig files (Carl Dong)
Pull request description:
This allows us to:
- remove the rfc4880 EOL hacks, and
- release with a SHA256SUMS.asc file that's a combination of all signer signatures
ACKs for top commit:
achow101:
ACK 90b3e482e9
laanwj:
Concept and code review ACK 90b3e482e9
Tree-SHA512: 5d5086063d303aa0cbd590e5fdf2ae8f555e25f4e43bf67545e33384449b990e94834c711622530ad0eb3dcc83f52746884a5081dadb0acff8dd799cfadafac7
Currently, using the getcoins.py with a custom signet executes successfully and shows the transaction as complete, however for obvious reasons, it should not.
This PR adds a sanity check for custom signet by comparing the current network's first block hash with global signet's respective hash.
f8f772dc49 macdeploy: alternative info to download the macOS SDK (Antoine Poinsot)
Pull request description:
The previous link wasn't accessible for me, this adds some instructions
given to me by Hebasto on #bitcoin-core-builds as well as a shasum for
the archive to quickly check the downloaded one is the right one before
processing with the entire Guix build.
ACKs for top commit:
fanquake:
ACK f8f772dc49
Tree-SHA512: 620160b593ed8fa4ae4a748b8e72d67b93ff0ec9e6b8ef3c3ac5402c1c48ec0ac325a527b6278cdf84aaf51ba8194d4c366c412ffad141d0412add2710efcff5
The previous link wasn't accessible for me, this adds some instructions
given to me by Hebasto on #bitcoin-core-builds as well as a shasum for
the archive to quickly check the downloaded one is the right one before
processing with the entire Guix build.
This also corrects a link to an older version of the SDK currently in
use.
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
9b313dfef1 guix: Ensure EPOCH_SOURCE_DATE does not include GPG information (Andrew Chow)
43225f0a2a guix: Remove extra \r from all.SHA256SUMS line ending (Andrew Chow)
d080c27066 guix, doc: Add a note that codesigners need to rebuild after tagging (Andrew Chow)
4a466388a0 guix: Allow changing the base manifest in guix-verify (Andrew Chow)
33455c7696 guix: Make all.SHA256SUMS rather than codesigned.SHA256SUMS (Andrew Chow)
Pull request description:
`guix-verify` expects `all.SHA256SUMS` but `guix-attest` produces `codesigned.SHA256SUMS`. Since `all.SHA256SUMS` makes more sense (as the file contains all the sha256sums, not just the codesigned ones), `guix-attest` has been changed to output a file of that name.
As a quality of life improvement, `guix-verify` can take `SIGNER` and use the signer's manifest as the base to compare against. This makes it easier to compare a single person's attestations with everyone else's and can make it more obvious when one builder is clearly mismatching with everyone else.
Lastly `release-process.md` is updated with a note about a gotcha that can cause a mismatch in the codesigned attestation.
ACKs for top commit:
fanquake:
ACK 9b313dfef1
Tree-SHA512: 0d60627def38288dbd3059ad1e72cad224f9205da11b1a561c082ef28250a074df5cc5f2797c91a7be027bc486a3fda3319c2e496a8724e5b539337236c6f990
If the user has set log.showSignature=true in their git config, then the
git log will always output GPG signature information. Since git log is
used to set EPOCH_SOURCE_DATE, this will mistakenly have GPG signature
information in it which causes issues for the build. To avoid this
issue, we override the config and force log.showSignature=false.
guix-attest mistakenly added an extra \r to the line endings in
all.SHA256SUMS, causing guix-verify to erroneously fail.
Co-Authored-By: Carl Dong <contact@carldong.me>
Both added files are extended in the following commits.
doc/usdt.md is based on earlier work by laanwj.
Co-authored-by: W. J. van der Laan <laanwj@protonmail.com>
9f01feda0a guix/build: Remove vestigial SKIPATTEST.TAG (Carl Dong)
Pull request description:
No longer needed or referenced by anything. A relic from prior to the great hierarchy overhaul of #22182
ACKs for top commit:
achow101:
ACK 9f01feda0a
fanquake:
ACK 9f01feda0a
Tree-SHA512: a94cf63f0c5cb8dbacf1025b6c0e81b219c2a3c93b3cbcefc239ccde29e602ecd4b717b1d93dbe53cb791a5017236fb09823c034aec42b0c31894fc9e0ab8b21
a884a1edcd guix/INSTALL: Misc fixups (Carl Dong)
3c4d2c418e guix: Silence getent(1) invocation (Carl Dong)
Pull request description:
Otherwise the `getent(1)` checks will print out the default http, https, and ftp ports, making it seem like something is being spawned that is listening on those ports, which is not the case.
ACKs for top commit:
fanquake:
ACK a884a1edcd
Tree-SHA512: 7706a98fe5f2bcd766fd3a16bfffab899ec45e80d72c485b7bed2a83d2024eddbb44ae4a77e2352e308740ca203c163421a11a5a2327fa94d2032ecceef4d63f
e6a94d4446 guix: Bump to version-1.3.0 from upstream (Carl Dong)
90fd13b954 guix: Pin kernel header version (Carl Dong)
Pull request description:
```
- Use 4.19 for riscv64 (earliest LTS release w/ riscv64 support)
- Use 4.9 for all others (second-oldest LTS release, released in
combination with glibc glibc 2.24 in Debian stretch)
```
```
The chosen commit is the HEAD of Guix's version-1.3.0 branch as of July
15th, 2021.
Also fix visual indenting.
```
-----
This + the documentation PR should make our Guix system ready for release!
ACKs for top commit:
MarcoFalke:
review ACK e6a94d4446 to change to vanilla guix. Did not review the kernel change.
laanwj:
ACK e6a94d4446
fanquake:
ACK e6a94d4446
Tree-SHA512: a175e4ddb3ee786a39f5e800ce336932ad2f6797a3a28400a6f723875d0f19833fd36cedc41b3580e4604110517211bd9f557be36adf7265fd8e591c434ae032
0a5723beea macdeploy: cleanup .temp.dmg if present (fanquake)
ecffe8689d macdeploy: remove qt4 related code (fanquake)
639f064253 macdeploy: select the plugins we need, rather than excluding those we don't (fanquake)
3d26b6b9e9 macdeploy: fix framework printing when passing -verbose (fanquake)
dca6c90329 macdeploy: remove unused plistlib import (fanquake)
Pull request description:
This includes [one followup](https://github.com/bitcoin/bitcoin/pull/20422#discussion_r534207899) and [one bug fix](3d26b6b9e9) from #20422, as well as some simplifications to the `macdeployqtplus` code.
ACKs for top commit:
hebasto:
ACK 0a5723beea, tested on macOS Big Sur 11.4 (20F71, x86_64) + Homebrew's Qt 5.15.2.
Tree-SHA512: cfad9505eacd32fe3a9d06eb13b2de0b6d2cad7b17778e90b503501cbf922e53d4e7f7f74952d1aed58410bdae9b0bb3248098583ef5b85689cb27d4dc06c029
fac4814106 doc/release-process: Add torrent creation details (Carl Dong)
5d24cc3d82 guix/INSTALL: Guix installs init scripts in libdir (Carl Dong)
5da2ee49d5 guix/INSTALL: Add coreutils/inotify-dir-recreate troubleshooting (Carl Dong)
318c60700b guix: Adapt release-process.md to new Guix process (Carl Dong)
fcab35b229 guix-attest: Produce and sign normalized documents (Carl Dong)
c2541fd0ca guix: Overhaul README (Carl Dong)
46ce6ce378 tree-wide: Rename gitian-keys to builder-keys (Carl Dong)
fc4f8449f3 guix: Update various check_tools lists (Carl Dong)
263220a85c guix: Check for a sane services database (Carl Dong)
Pull request description:
Based on: #21462
Keeping the README in one file so that it's easy to search through. Will add more jumping links later so navigation is easier.
Current TODOs:
- [x] Shell installer option: prompt user to re-login for `/etc/profile.d` entry to be picked up
- [x] Binary tarball option: prompt user to create `/etc/profile.d` entry and re-login
- [x] Fanquake docker option: complete section
- [x] Arch Linux AUR option: prompt to start `guix-daemon-latest` unit after finishing "optional setup" section
- [x] Building from source option: Insert dependency tree diagram that I made
- [x] Building from source option: redo sectioning, kind of a mess right now
- [x] Optional setup: make clear which parts are only needed if building from source
- [x] Workaround 1 for GnuTLS: perhaps mention how to remove Guix build farm's key
- [x] Overall (after everything): Make the links work.
Note to self: wherever possible, tell user how to check that something is true rather than branching by installation option.
ACKs for top commit:
fanquake:
ACK fac4814106 - going to go ahead and merge this now. It's a lot of documentation, and could probably be nit-picked / improved further, however, that can continue over the next few weeks. I'm sure more (backportable) improvements / clarifications will be made while we progress through RCs towards a new release.
Tree-SHA512: dc46c0ecdfc67c7c7743ca26e4a603eb3f54adbf81be2f4c1f4c20577ebb84b5250b9c9ec89c0e9860337ab1c7cff94d7963c603287267deecfe1cd987fa070a
That way we can easily combine the document and detached signature to
produce cleartext signature files for upload during the release process.
See subsequent commits which modify doc/release-process.md for more
details.
1edddf5de4 Avoid GCC 7.1 ABI change warning in guix build (Pieter Wuille)
Pull request description:
The arm-linux-gnueabihf guix build output is littered with warnings like:
```
/gnu/store/7a96hdqdb2qi8a39f09n84xjy2hr23rs-gcc-cross-arm-linux-gnueabihf-8.4.0/include/c++/bits/stl_vector.h:1085:4: note:
parameter passing for argument of type '__gnu_cxx::__normal_iterator<CRecipient*, std::vector<CRecipient> >' changed in GCC 7.1
```
These are irrelevant for us. Disable them using `-Wno-psabi`.
ACKs for top commit:
laanwj:
ACK 1edddf5de4
hebasto:
ACK 1edddf5de4, after thorough reading related materials, I agree this change can be merged. As I mentioned above, I have been compiling my arm-32bit binaries with `-Wno-psabi` flag for two years, and no related flaws were observed.
Tree-SHA512: 485c7500547ac5da567ad23847341c18ff832607f5a1002676404cc647e437cf3445b6894ecff5b52929ca52bea946c06bd90eace1997c895e56204e787065e4
- Use 4.19 for riscv64 (earliest LTS release w/ riscv64 support)
- Use 4.9 for all others (second-oldest LTS release, released in
combination with glibc glibc 2.24 in Debian stretch)
On bare systems, it is possible to be lacking a services database. Check
for basic entries before attempting a build.
See the error message in the diff for more context.
Now that our release binaries are build in a glibc 2.24 and 2.27
environment, we can't use a symbol from glibc 2.28 to test our checks.
Replace renameat2() with nextup(), which was introduced in 2.24.
Note that this also means re-disabling the test for RISC-V, however
RISC-V is built in a glibc 2.27 environment, and our minimum required
glibc for that binary is 2.27.
We use these flags in our test-security-check make target, but they are
only available because debian patches them in.
We can patch them in for our Guix builds so that we can check the sanity
of our security/symbol checking suite before running them.
This is important to make sure that we're not testing tools different
from the one we're building with.
Introduce determine_wellknown_cmd, which encapsulates how we
should handle well-known tools specification (IFS splitting, env
override, etc.).
Now that our Guix builds are performed on glibc 2.24 and 2.27 (RISCV),
we no-longer need to pass the --enable-glibc-back-compat option.
Replace it with --disable-threadlocal, to prevent the usage of symbols
from glibc 2.18.
None of the binaries produced required symbols later than 2.17, and 2.27
(RISCV).
Our 'bitcoin-linux-g++' definition better integrates with our depends
system than the stock linux-g++-64 definition.
This fixes a bug whereby Guix builds on x86_64 for x86_64 did not
produce a QMinimalIntegrationPlugin and led to bitcoin-qt not being
built.
Support for riscv64 in glibc landed in 2.27 so it's unavoidable that we
use 2.27.
Running a Bitcoin build with toolchains based on 2.24 for platforms
other than riscv64 seem to produce binaries which do not have 2.17
symbols. So use 2.24 since it's more recent and maintained by Debian
Stretch.
bdb8b9a347 test: doc: improve doc for `from_hex` helper (mention `to_hex` alternative) (Sebastian Falbesoner)
1914054208 scripted-diff: test: rename `FromHex` to `from_hex` (Sebastian Falbesoner)
a79396fe5f test: remove `ToHex` helper, use .serialize().hex() instead (Sebastian Falbesoner)
2ce7b47958 test: introduce `tx_from_hex` helper for tx deserialization (Sebastian Falbesoner)
Pull request description:
There are still many functional tests that perform conversions from a hex-string to a message object (deserialization) manually. This PR identifies all those instances and replaces them with a newly introduced helper `tx_from_hex`.
Instances were found via
* `git grep "deserialize.*BytesIO"`
and some of them manually, when it were not one-liners.
Further, the helper `ToHex` was removed and simply replaced by `.serialize().hex()`, since now both variants are in use (sometimes even within the same test) and using the helper doesn't really have an advantage in readability. (see discussion https://github.com/bitcoin/bitcoin/pull/22257#discussion_r652404782)
ACKs for top commit:
MarcoFalke:
review re-ACK bdb8b9a347😁
Tree-SHA512: e25d7dc85918de1d6755a5cea65471b07a743204c20ad1c2f71ff07ef48cc1b9ad3fe5f515c1efaba2b2e3d89384e7980380c5d81895f9826e2046808cd3266e
e8cd3700ee devtools: Integrate ARCH_MIN_GLIBC_VER table into MAX_VERSIONS in symbol-check.py (W. J. van der Laan)
a33381acf5 devtools: Add xkb version to symbol-check (W. J. van der Laan)
19e598bab0 devtools: Fix verneed section parsing in pixie (W. J. van der Laan)
Pull request description:
I misunderstood the ELF specification for version symbols (verneed): The `vn_aux` pointer is relative to the main verneed record, not the start of the section.
This caused many symbols to not be versioned properly in the return value of `elf.dyn_symbols`. This was discovered in #21454.
Fix it by correcting the offset computation.
- xkb versions symbols (using the prefix `V`), as this library is used by bitcoin-qt, add it to the valid versions in `symbol-check.py`
This unfortunately brings to light some symbols that have been introduced since and weren't caught (from a gitian compile of master):
```
bitcoin-cli: symbol getrandom from unsupported version GLIBC_2.25
bitcoin-cli: failed IMPORTED_SYMBOLS
bitcoind: symbol getrandom from unsupported version GLIBC_2.25
bitcoind: symbol log from unsupported version GLIBC_2.29
bitcoind: symbol fcntl64 from unsupported version GLIBC_2.28
bitcoind: symbol pow from unsupported version GLIBC_2.29
bitcoind: symbol exp from unsupported version GLIBC_2.29
bitcoind: failed IMPORTED_SYMBOLS
bitcoin-qt: symbol exp from unsupported version GLIBC_2.29
bitcoin-qt: symbol fcntl64 from unsupported version GLIBC_2.28
bitcoin-qt: symbol log from unsupported version GLIBC_2.29
bitcoin-qt: symbol pow from unsupported version GLIBC_2.29
bitcoin-qt: symbol statx from unsupported version GLIBC_2.28
bitcoin-qt: symbol getrandom from unsupported version GLIBC_2.25
bitcoin-qt: symbol renameat2 from unsupported version GLIBC_2.28
bitcoin-qt: symbol getentropy from unsupported version GLIBC_2.25
bitcoin-qt: failed IMPORTED_SYMBOLS
bitcoin-wallet: symbol exp from unsupported version GLIBC_2.29
bitcoin-wallet: symbol log from unsupported version GLIBC_2.29
bitcoin-wallet: symbol fcntl64 from unsupported version GLIBC_2.28
bitcoin-wallet: failed IMPORTED_SYMBOLS
test_bitcoin: symbol getrandom from unsupported version GLIBC_2.25
test_bitcoin: symbol log from unsupported version GLIBC_2.29
test_bitcoin: symbol fcntl64 from unsupported version GLIBC_2.28
test_bitcoin: symbol pow from unsupported version GLIBC_2.29
test_bitcoin: symbol exp from unsupported version GLIBC_2.29
test_bitcoin: failed IMPORTED_SYMBOLS
```
ACKs for top commit:
hebasto:
ACK e8cd3700ee
Tree-SHA512: 8c15e3478eb642f01a1ddaadef03f80583f088f9fa8e3bf171ce16b0ec05ffb4675ec147d7ffc6a4360637ed47fca517c6ca2bac7bb30d794c03783cfb964b79
The (ancient) versions specified here were deceptive. Entries older than
MAX_VERSIONS['GLIBC'], which is 2.17, are ignored here. So reorganize
the code to avoid confusion for other people reading this code.
aa80b5759d scripts: check macOS SDK version is set (fanquake)
c972345bac scripts: check minimum required Windows version is set (fanquake)
29615aef52 scripts: check minimum required macOS vesion is set (fanquake)
8732f7b6c9 scripts: LIEF 0.11.5 (fanquake)
Pull request description:
macOS:
We use a compile flag ([-mmacosx-version-min=10.14](https://github.com/bitcoin/bitcoin/blob/master/depends/hosts/darwin.mk#L96)) to set the minimum required version of macOS needed to run our binaries. This adds a sanity check that the version is being set as expected.
Clangs Darwin driver should infer the SDK version used during compilation, and forward that through to the linker. Add a check that this has been done, and the expected SDK version is set. Should help prevent issues like #21771 in future.
Windows:
We use linker flags ([-Wl,--major/minor-subsystem-version](https://github.com/bitcoin/bitcoin/blob/master/configure.ac#L683)) to set the minimum required version of Windows needed to run our binaries. This adds a sanity check that the version is being set as expected.
Gitian builds:
```bash
# macOS:
8b6fcd61d75001c37b2af3fceb5ae09f5d2fe85e97d361f684214bd91c27954a bitcoin-f015e1c2cac9-osx-unsigned.dmg
3c1e412bc7f5a7a5d0f78e2cd84b7096831414e1304c1307211aa3e135d89bbf bitcoin-f015e1c2cac9-osx-unsigned.tar.gz
50b7b2804e8481f63c69c78e3e8a71c0d811bf2db8895dd6d3edae9c46a738ae bitcoin-f015e1c2cac9-osx64.tar.gz
fe6b5c0a550096b76b6727efee30e85b60163a41c83f21868c849fdd9876b675 src/bitcoin-f015e1c2cac9.tar.gz
8a20f21b20673dfc8c23e22b20ae0839bcaf65bf0e02f62381cdf5e7922936f0 bitcoin-core-osx-22-res.yml
# Windows:
b01fcdc2a5673387050d6c6c4f96f1d350976a121155fde3f76c2af309111f9d bitcoin-f015e1c2cac9-win-unsigned.tar.gz
b95bdcbef638804030671d2332d58011f8c4ed4c1db87d6ffd211515c32c9d02 bitcoin-f015e1c2cac9-win64-debug.zip
350bf180252d24a3d40f05e22398fec7bb00e06d812204eb5a421100a8e10638 bitcoin-f015e1c2cac9-win64-setup-unsigned.exe
2730ddabe246d99913c9a779e97edcadb2d55309933d46f1dffd0d23ecf9aae5 bitcoin-f015e1c2cac9-win64.zip
fe6b5c0a550096b76b6727efee30e85b60163a41c83f21868c849fdd9876b675 src/bitcoin-f015e1c2cac9.tar.gz
aa60d7a753e8cb2d4323cfbbf4d964ad3645e74c918cccd66862888f8646d80f bitcoin-core-win-22-res.yml
```
ACKs for top commit:
hebasto:
ACK aa80b5759d, tested by breaking tests:
Tree-SHA512: 10150219910e8131715fbfe20edaa15778387616ef3bfe1a5152c7acd3958fe8f88c74961c3d3641074eb72824680c22764bb1dc01a19e92e946c2d4962a8d2c
e2c40a4ed5 guix-attest: Error out if SHA256SUMS is unexpected (Carl Dong)
4cc35daed5 Rewrite guix-{attest,verify} for new hier (Carl Dong)
28a9c9b839 Make SHA256SUMS fragment right after build (Carl Dong)
Pull request description:
Based on: #22075
Code reviewers: I recommend reading the new `guix-{attest,verify}` files instead of trying to read the diff
The following changes resolve many usability improvements which were pointed out to me:
1. Some maintainers like to extract their "uncodesigned tarball" inside the `output/` directory, resulting in the older `guix-attest` mistakenly attesting to the extracted contents
2. Maintainers whose GPG keys reside on an external smartcard often need to physically interact with the smartcard as a way to approve the signing operation, having one signature per platform means a lot of fidgeting
3. Maintainers wishing to sign on a separate machine now has the option of transferring only a subtree of `output/`, namely `output/*/SHA256SUMS.part`, in order to perform a signature (you may need to specify an `$OUTDIR_BASE` env var)
4. An `all.SHA256SUMS` file should be usable as the base `SHA256SUMS` in bitcoin core torrents and on the release server.
For those who sign on an separate machine than the one you do builds on, the following steps will work:
1. `env GUIX_SIGS_REPO=/home/achow101/guix.sigs SIGNER=achow101 NO_SIGN=1 ./contrib/guix/guix-attest`
2. Copy `/home/achow101/guix.sigs/<tag>/achow101` (which does not yet have signatures) to signing machine
3. Sign the `SHA256SUMS` files:
```bash
for i in "<path-to-achow101>/*.SHA256SUMS"; do
gpg --detach-sign --local-user "<your-key-here>" --armor --output "$i"{.asc,}
done
```
5. Upload `<path-to-achow101>` (now with signatures) to `guix.sigs`
-----
After this change, output directories will now include a `SHA256SUMS.part` fragment, created immediately after a successful build:
```
output
└── x86_64-w64-mingw32
├── bitcoin-4e069f7589da-win64-debug.zip
├── bitcoin-4e069f7589da-win64-setup-unsigned.exe
├── bitcoin-4e069f7589da-win64.zip
├── bitcoin-4e069f7589da-win-unsigned.tar.gz
└── SHA256SUMS.part
```
These `SHA256SUMS.part` fragments look something like:
```
3ebd7262b1a0a5bb757fef1f70e7e14033c70f98c059bc4dbfee5d1992b25825 dist-archive/bitcoin-4e069f7589da.tar.gz
def2e7d3de5ab3e3f955344e75151df4f33713f9101f5295bd13c9375bdf633b x86_64-w64-mingw32/bitcoin-4e069f7589da-win64-debug.zip
643049fe3ee4a4e83a1739607e67b11b7c9b1a66208a6f35a9ff634ba795500e x86_64-w64-mingw32/bitcoin-4e069f7589da-win64-setup-unsigned.exe
a247a1ccec0ccc2e138c648284bd01f6a761f2d8d6d07d91b5b4a6670ec3f288 x86_64-w64-mingw32/bitcoin-4e069f7589da-win-unsigned.tar.gz
fab76a836dcc592e39c04fd2396696633fb6eb56e39ecbf6c909bd173ed4280c x86_64-w64-mingw32/bitcoin-4e069f7589da-win64.zip
```
Meaning that they are valid `SHA256SUMS` files when `sha256sum --check`'d at the `guix-build-*/output` directory level
When `guix-attest` is invoked, these `SHA256SUMS.part` files are combined and sorted (by `-k2`, `LC_ALL=C`) to create:
1. `noncodesigned.SHA256SUMS` for a manifest of all non-codesigned outputs, and
3. `all.SHA256SUMS` for a manifest of all outputs including non-codesigned outputs
Then both files are signed, resulting in the following `guix.sigs` hierarchy:
```
4e069f7589da/
└── dongcarl
├── all.SHA256SUMS
├── all.SHA256SUMS.asc
├── noncodesigned.SHA256SUMS
└── noncodesigned.SHA256SUMS.asc
```
ACKs for top commit:
achow101:
ACK e2c40a4ed5
hebasto:
ACK e2c40a4ed5, tested on Linux Mint 20.1 (x86_64) with and w/o `NO_SIGN=1`. Changes in `contrib/guix/libexec/codesign.sh` and `contrib/guix/guix-verify` are reviewed only.
Tree-SHA512: 618aacefb0eb6595735a9ab6a98ea6598fce65f9ccf33fa1e7ef93bf140c0f6cfc16e34870c6aa3e4777dd3f004b92a82a994141879870141742df948ec59c1f
I misunderstood the ELF specification for version symbols (verneed):
The `vn_aux` pointer is relative to the main verneed record, not the
start of the section.
This caused many symbols to not be versioned properly in the return
value of `elf.dyn_symbols`. This was discovered in #21454.
Fix it by correcting the offset computation.
683d197970 Use latest signapple commit (Andrew Chow)
Pull request description:
Update gitian and guix to use the same latest signapple commit.
Also changed guix to use the actual repo. The changes from the fork were incorporated upstream.
ACKs for top commit:
fanquake:
ACK 683d197970 - sanity checked that the updated package is built:
Tree-SHA512: a4981f8bbe33e6c5654632bc9b9f6f2f1e675741a19ac7296205e370f1e64a747101ecb632e0cc82a0134e4c2e9ce47b3f7b4d8c8f75f0f06dd069c078303759
Clangs Darwin driver should infer the SDK version used during compilation, and
forward that through to the linker. Add a check that this has been done, and the
expected SDK version is set.
Should help prevent issues like #21771 in future.
We use linker flags (-Wl,--major/minor-subsystem-version) to set the
minimum required version of Windows needed to run our binaries. This
adds a sanity check that the version is being set as expected.
We use a compile flag (-mmacosx-version-min) to set the minimum required
version of macOS needed to run our binaries. This adds a sanity check
that the version is being set as expected.
108a6be92a guix: Check for disk space availability before building (Carl Dong)
d7dec89091 guix: Remove dest if OUTDIR mv fails (Carl Dong)
Pull request description:
There seems to be some corner cases that can be hit when guix scripts unexpectedly fail in the middle of operation, see: https://gnusha.org/bitcoin-builds/2021-05-24.log
- Perform an early disk space check for `guix-build`
- Overwrite existing output directory after a successful build (the existing one might be malformed), and cleanup output directory if the `mv` somehow fails
ACKs for top commit:
laanwj:
Tested ACK 108a6be92a
achow101:
ACK 108a6be92a
Tree-SHA512: cf6438317da40bf55714cd2d8cce859b3d435cc66cabefe8d4a53552d7880966acfe84ffe8fadf1c80e368ae6b037992258a6d409df85ffc6ce8bf780e98e2e5
5d82a57db4 contrib: remove torv2 seed nodes (Jon Atack)
5f7e086dac contrib: update generate-seeds.py to ignore torv2 addresses (Jon Atack)
8be56f0f8e p2p, refactor: extract OnionToString() from CNetAddr::ToStringIp() (Jon Atack)
5f9d3c09b4 p2p: remove torv2 from CNetAddr::ToStringIP() (Jon Atack)
3d39042144 p2p: remove torv2 in SetIP() and ADDR_TORV2_SIZE constant (Jon Atack)
cff5ec477a p2p: remove pre-addrv2 onions from SerializeV1Array() (Jon Atack)
4192a74413 p2p: ignore torv2-in-ipv6 addresses in SetLegacyIPv6() (Jon Atack)
1d631e956f p2p: remove BIP155Network::TORV2 from GetBIP155Network() (Jon Atack)
7d1769bc45 p2p: remove torv2 from SetNetFromBIP155Network() (Jon Atack)
eba9a94b9f fuzz: rename CNetAddr/CService deserialize targets (Jon Atack)
c56a1c9b18 p2p: drop onions from IsAddrV1Compatible(), no longer relay torv2 (Jon Atack)
f8e94002fc p2p: remove torv2/ADDR_TORV2_SIZE from SetTor() (Jon Atack)
0f1c58ae87 test: update feature_proxy to torv3 (Jon Atack)
Pull request description:
This patch removes support in Bitcoin Core for Tor v2 onions, which are already removed from the release of Tor 0.4.6.
- no longer serialize/deserialize and relay Tor v2 addresses
- ignore incoming Tor v2 addresses
- remove Tor v2 addresses from the addrman and peers.dat on node launch
- update generate-seeds.py to ignore Tor v2 addresses
- remove Tor v2 hard-coded seeds
Tested with tor-0.4.6.1-alpha (no v2 support) and 0.4.5.7 (v2 support). With the latest Tor (no v2 support), this removes all the warnings like those reported with current master in https://github.com/bitcoin/bitcoin/issues/21351
```
<bitcoind debug log>
Socks5() connect to […].onion:8333 failed: general failure
<tor log>
Invalid hostname [scrubbed]; rejecting
```
and the addrman no longer has Tor v2 addresses on launching bitcoind.
```rake
$ ./src/bitcoin-cli -addrinfo
{
"addresses_known": {
"ipv4": 44483,
"ipv6": 8467,
"torv2": 0,
"torv3": 2296,
"i2p": 6,
"total": 55252
}
}
```
After recompiling back to current master and restarting with either of the two Tor versions (0.4.5.7 or 0.4.6.1), -addrinfo initially returns 0 Tor v2 addresses and then begins finding them again.
Ran nodes on this patch over the past week on mainnet/testnet/signet/regtest after building with DEBUG_ADDRMAN.
Verified that this patch bootstraps an onlynet=onion node from the Tor v3 hardcoded fixed seeds on mainnet and testnet and connects to blocks and v3 onion peers: `rm ~/.bitcoin/testnet3/peers.dat ; ./src/bitcoind -testnet -dnsseed=0 -onlynet=onion`
Tested using `addnode`, `getaddednodeinfo`,`addpeeraddress`, `disconnectnode` and `-addrinfo` that a currently valid, connectable Tor v2 peer can no longer be added:
Thanks to Vasil Dimov, Carl Dong, and Wladimir J. van der Laan for their work on BIP155 and Tor v3 that got us here.
ACKs for top commit:
laanwj:
Code review ACK 5d82a57db4
Tree-SHA512: 590ff3d2f6ef682608596facb4b01f44fef69716d2ab3552ae1655aa225f4bf104f9ee08d6769abb9982a8031de93340df553279ce1f5023771f9f2b651178bb
a58868d201 build: Makes rcc output always deterministic (Hennadii Stepanov)
Pull request description:
The Qt Resource Compiler ([rcc](https://doc.qt.io/qt-5/rcc.html)) has a command-line option `--format-version` which has the [default value](https://code.qt.io/cgit/qt/qtbase.git/tree/src/tools/rcc/main.cpp?h=5.12.10#n172) 2.
The only difference from `--format-version 1` is adding a [last modified timestamp](https://code.qt.io/cgit/qt/qtbase.git/tree/src/tools/rcc/rcc.cpp?h=5.12.10#n207) to the output file ([credits](https://github.com/bitcoin/bitcoin/pull/21654#issuecomment-819198228) to **fanquake**). That, in turn, forces us to use `QT_RCC_SOURCE_DATE_OVERRIDE=1` to get deterministic builds (#13732).
This change makes rcc output always deterministic by using `--format-version 1` option that makes usage of the
`QT_RCC_SOURCE_DATE_OVERRIDE` needless.
---
Also it improves interaction with ccache:
On master (f6c44e999b):
```
$ make && make clean && ccache --zero-stats && make && ccache --show-stats
...
cache directory /home/hebasto/.ccache
primary config /home/hebasto/.ccache/ccache.conf
secondary config (readonly) /etc/ccache.conf
stats updated Sun Apr 11 15:45:43 2021
stats zeroed Sun Apr 11 15:45:05 2021
cache hit (direct) 638
cache hit (preprocessed) 0
cache miss 1
cache hit rate 99.84 %
called for link 10
cleanups performed 0
files in cache 20023
cache size 13.2 GB
max cache size 15.0 GB
```
The missed file is always `qt/libbitcoinqt_a-qrc_bitcoin_locale.o`.
With this PR:
```
$ make && make clean && ccache --zero-stats && make && ccache --show-stats
...
cache directory /home/hebasto/.ccache
primary config /home/hebasto/.ccache/ccache.conf
secondary config (readonly) /etc/ccache.conf
stats updated Sun Apr 11 15:28:46 2021
stats zeroed Sun Apr 11 15:28:21 2021
cache hit (direct) 639
cache hit (preprocessed) 0
cache miss 0
cache hit rate 100.00 %
called for link 10
cleanups performed 0
files in cache 20012
cache size 13.2 GB
max cache size 15.0 GB
```
ACKs for top commit:
fanquake:
ACK a58868d201
Tree-SHA512: 52f4a3267f41883d13025c0de79b6da22e92d60c729e01b986935c6812bbfe7fadc40b742bd715bfdf09df94af6838d4fbbe8208c6123f366108e38c8e1121c5
167fb1fc72 Update Windows code signing certificate (Andrew Chow)
Pull request description:
Updates the Windows code signing certificate to a new one issued by Digicert. This certificate has been issued to Bitcoin Core Code Signing LLC registered in Delaware, US. Note that this is different from the previous Bitcoin Core Code Signing Association registered in Zurich, Switzerland as it was unable to meet the validation requirements in time.
ACKs for top commit:
Sjors:
utACK 167fb1f
laanwj:
ACK 167fb1fc72
Tree-SHA512: 8d5308c710ef94330417955b9bc82c5894d282798cebece82b84b425e3354e566aa6a68693ec359391ea40ddd7e2032d35ce28d104683d75ec3010ddf00be209
6fe0516858 contrib: add torv3 seed nodes for testnet, drop v2 ones (Jon Atack)
Pull request description:
Replace the ancient (2015) Tor V2 hardcoded seeds with new Tor V3 ones. This needs to be done before 0.22 to make sure onion-only testnet nodes can still connect to the network. Continues #21560.
Ways to test:
- Re-generate ` src/chainparamsseeds.h` with `cd contrib/seeds && python3 generate-seeds.py . > ../../src/chainparamsseeds.h`, check if git tree stays the same.
- Create a new testnet node with `bitcoind -testnet -onlynet=onion -proxy=127.0.0.1:9050` (or delete `~/.bitcoin/testnet3/peers.dat`), check if it is able to connect to the network and get blocks.
- Check if the addresses are connectable for ex.:
```python3
#!/usr/bin/env python3
import subprocess
with open('contrib/seeds/nodes_test.txt') as f:
for line in (line for line in (line.rstrip().split('#', 1)[0] for line in f) if line):
subprocess.call(["nc", "-v", "-x", "127.0.0.1:9050", "-z"] + line.split(':'))
```
Thanks to jonatack for providing the list.
ACKs for top commit:
jonatack:
ACK 6fe0516858
Tree-SHA512: 61bfdb44dfab9d02b75e5cb06c089a3b1a1fe7134875e1d09166c4116e961d809aa25422fe03f068876e9423b571ecc4a0c7a7eeacba4aac3b2768717f3ee6d6
ee883201cf guix: repro: Sort find output in libtool for gcc-8 (Carl Dong)
ee0a67c32a codesigning: Use SHA256 as digest for osslsigncode (Windows) (Carl Dong)
38eb91eb06 guix: Add codesigning functionality (Carl Dong)
bac2690e6f guix: Package codesigning tools (Carl Dong)
0a2176d477 guix: Reindent existing manifest.scm (Carl Dong)
c090a3e923 Makefile.am: use APP_DIST_DIR instead of hard-coding dist (Carl Dong)
Pull request description:
This is the last PR before we reach feature-parity with the Gitian process!
Note: I tried using the `Makefile` inside the distsrc to make the dmg instead of manually listing out the commands, but `make` seems to want to re-make a lot of other files which broke the dmg.
The workflow looks something like this:
1. `env [ FOO=bar... ] ./contrib/guix/guix-build` (add additional env vars as necessary)
2. Codesigners only:
1. Copy `guix-build-<short-id>/output/x86_64-apple-darwin18/bitcoin-<short-id>-osx-unsigned.tar.gz` and `guix-build-<short-id>/output/x86_64-w64-mingw32/bitcoin-<short-id>-win-unsigned.tar.gz` to signing computer
2. Codesign with `./detached-sig-create.sh` inside the tarball
3. Upload contents of `signature-{osx,win}.tar.gz` to https://github.com/bitcoin-core/bitcoin-detached-sigs (as a new tag)
3. Checkout new tag for `bitcoin-core/bitcoin-detached-sigs` with the detached signatures
4. `env [ FOO=bar... ] DETACHED_SIGS_REPO=<path/to/bitcoin-detached-sigs> ./contrib/guix/guix-codesign` (modify env vars as necessary)
5. Make sure `guix.sigs` is cloned and updated
6. `env GUIX_SIGS_REPO=<path/to/guix.sigs> SIGNER=0x96AB007F1A7ED999=dongcarl ./contrib/guix/guix-attest` (modify env vars as necessary)
7. Commit your new signatures and SHA256SUMS in `guix.sigs`
8. Optionally, after there are multiple signatures in `guix.sigs`: `env GUIX_SIGS_REPO=<path/to/guix.sigs> ./contrib/guix/guix-verify`
ACKs for top commit:
laanwj:
Tested ACK ee883201cf
achow101:
ACK ee883201cf
Tree-SHA512: e812a07a5f19f900600c70cb9c717769ef544a6c0c12760b5558b76b6b37df863257f3dbf38b0757e6e06e334470267e94c9f2bdbc27409d6837b1a0bfc6acbc
42b589d18f scripts: test for MACHO control flow instrumentation (fanquake)
469a5bc4fa build: build Boost with -fcf-protection when targeting Darwin (fanquake)
Pull request description:
Addresses the macOS portion of #21888.
Build Boost with `-fcf-protection` when targeting Darwin. This should be ok, because our cross-compiler (Clang 10) supports the option, and I'd expect all versions of Apple Clang being used to compile Core would also support it. Building Boost with this option is required so that the `main` provided to `test_bitcoin` has instrumentation.
Note that the presence of instrumentation does not mean it will be used, as that is determined at runtime by the CPU.
From the Intel control flow enforcement documentation:
> The ENDBR32 and ENDBR64 instructions will have the same effect as the NOP instruction on Intel 64 processors that do not support CET. On processors supporting CET, these instructions do not change register or flag state. This allows CET instrumented programs to execute on processors that do not support CET. Even when CET is supported and enabled, these NOP–like instructions do not affect the execution state of the program, do not cause any additional register pressure, and are minimally intrusive from power and performance perspectives.
Follow up from #21135.
Guix builds:
```bash
663df8471400f06d4da739e39a886aa17f56a36d66e0ff7cc290686294ef39c9 guix-build-42b589d18fed/output/dist-archive/bitcoin-42b589d18fed.tar.gz
45e841661e1659a634468b6f8c9fb0a7956c31ba296f1fd0c02cd880736d6127 guix-build-42b589d18fed/output/x86_64-apple-darwin18/bitcoin-42b589d18fed-osx-unsigned.dmg
0ea85c99fef35429a5048fa14850bce6b900eaa887aeea419b019852f8d2be78 guix-build-42b589d18fed/output/x86_64-apple-darwin18/bitcoin-42b589d18fed-osx-unsigned.tar.gz
85857a5a4a5d4d3a172d6c361c12c4a94f6505fc12b527ea63b75bfe54ee1001 guix-build-42b589d18fed/output/x86_64-apple-darwin18/bitcoin-42b589d18fed-osx64.tar.gz
```
Gitian builds:
```bash
# macOS:
bdfd677a6b88273a741b433e1e7f554af50cc76b3342d44ab0c441e2b40efc96 bitcoin-42b589d18fed-osx-unsigned.dmg
f3b2d09f3bea7a5cc489b02e8e53dd76a9922338500fae79cad0506655af56f9 bitcoin-42b589d18fed-osx-unsigned.tar.gz
29d5ad5e46bc9fb0056922a8b47c026e5e9f71e6cf447203b74644587d6fb6f7 bitcoin-42b589d18fed-osx64.tar.gz
663df8471400f06d4da739e39a886aa17f56a36d66e0ff7cc290686294ef39c9 src/bitcoin-42b589d18fed.tar.gz
366f8d7a2fc1f3e22cb1018043099126a71ce65380cc27b1c3280cce42d06c98 bitcoin-core-osx-22-res.yml
```
ACKs for top commit:
laanwj:
Code review ACK 42b589d18f
Tree-SHA512: 12cb8d462d64d845b9fe48c5c6978892adff8bf5b5572bb29f35df1f6176e47b32a68bcb6e4883c7d9454e76e8868851005a7325916852a2d0d32659ac7dae3f
d420e5c1c0 guix-attest: Avoid incomplete sigdirs with ERR traps (Carl Dong)
feda2c8e31 guix: Skip attesting to dist-archive (Carl Dong)
d522d8006b guix: Attest to inputs in inputs.SHA256SUMS (Carl Dong)
f9e2960c01 guix: Construct $OUTDIR in ${DISTSRC}/output (Carl Dong)
022abc85fc guix: Minor quoting fix in libexec/build.sh (Carl Dong)
c83c4fa5b7 guix-attest: Allow skipping GPG signing with NO_SIGN (Carl Dong)
0e1c2e448c guix-attest: Use ascii-armor signatures (Carl Dong)
b5fd89c4c8 guix-attest: Only use cross-platform flags for find+xargs (Carl Dong)
5926432ba6 guix: Add guix-verify script (Carl Dong)
30daf76a97 guix: Add guix-attest script (Carl Dong)
Pull request description:
Adds replacements for `gsign` and `gverify`.
Personally I'm not a big fan of using the word "sign" as it's been used to refer to both codesigning and GPG signing.
ACKs for top commit:
laanwj:
Code review and tested ACK d420e5c1c0
Tree-SHA512: 93d82d201f4596eaea0e3825aa55b013dfb91790e6ccee79893833d37921513d7b4e735f0641103e1e2ea8308abe4cb6218b73160924708802f2e0e3f7f6caf1
The Qt Resource Compiler (rcc) has a command-line option
`--format-version` which has the default value 2.
The only difference from `--format-version 1` is adding a last modified
timestamp to the output file. That, in turn, forces us to use
`QT_RCC_SOURCE_DATE_OVERRIDE=1` to get deterministic builds.
This change makes rcc output always deterministic by using
`--format-version 1` option that makes usage of the
`QT_RCC_SOURCE_DATE_OVERRIDE` needless. Also it improves interaction
with ccache.
Co-authored-by: fanquake <fanquake@gmail.com>
46b025e00d test: add new python linter to check file names and permissions (windsok)
6f6bb3ebc7 test: fix file permissions on various scripts (windsok)
Pull request description:
Adds a new python linter test which tests for correct filenames and file permissions in the repository.
Replaces the existing tests in the `test/lint/lint-filenames.sh` and `test/lint/lint-shebang.sh` linter tests, as well as adding some new and increased testing. This increased coverage is intended to catch issues such as in #21728 and https://github.com/bitcoin/bitcoin/pull/16807/files#r345547050
Summary of tests:
* Checks every file in the repository against an allowed regexp to make sure only lowercase or uppercase alphanumerics (a-zA-Z0-9), underscores (_), hyphens (-), at (@) and dots (.) are used in repository filenames.
* Checks only source files (*.cpp, *.h, *.py, *.sh) against a stricter allowed regexp to make sure only lowercase alphanumerics (a-z0-9), underscores (_), hyphens (-) and dots (.) are used in source code filenames. Additionally there is an exception regexp for directories or files which are excepted from matching this regexp (This should replicate the existing `test/lint/lint-filenames.sh` test)
* Checks all files in the repository match an allowed executable or non-executable file permission octal. Additionally checks that for executable files, the file contains a shebang line.
* Checks that for executable `.py` and `.sh` files, the shebang line used matches an allowable list of shebangs (This should replicate the existing `test/lint/lint-shebang.sh` test)
* Checks every file that contains a shebang line to ensure it has an executable permission
Additionally updates the permissions on various files to comply with the new tests.
Fixes#21729
ACKs for top commit:
practicalswift:
cr re-ACK 46b025e00d: patch still looks correct
kiminuo:
code review ACK 46b025e00d if `contrib/gitian-descriptors/assign_DISTNAME` permission change is deemed OK.
laanwj:
Code review ACK 46b025e00d
Tree-SHA512: 1c8201a2cee0d9cbce15652b68cec9a6458a8b493fcd5392f98560aca0b1a12e668baab65a47100f116f626dadc3f591deb47f7368468c6a46c6c712c2533455
7fc5e865b9 test: install lief in CI (fanquake)
955140b326 contrib: consolidate PIE and NX security checks (fanquake)
2aa1631822 contrib: use LIEF in PE symbol checks (fanquake)
e93ac26b85 contrib: use LIEF in macOS symbol checks (fanquake)
a632cbcee5 contrib: use f strings in symbol-check.py (fanquake)
0f5d77c8e4 contrib: add PE PIE check to security checks (fanquake)
8e1f40dd9a contrib: use LIEF for PE security checks (fanquake)
a25b2e965c contrib: use LIEF for macOS security checks (fanquake)
7e7eae7aa8 contrib: use f strings in security-check.py (fanquake)
2e7a9f7ade guix: install LIEF in Guix container (fanquake)
465967b5ef gitian: install LIEF in gitian container (fanquake)
Pull request description:
This PR is a proof of concept for using [LIEF](https://github.com/lief-project/LIEF) for the PE and MACHO symbol and security checks. It replaces our current approach of manually parsing the output of `objdump` & `otool`. If the consensus is that using LIEF is ok, then I also plan on replacing [pixie.py](https://github.com/bitcoin/bitcoin/blob/master/contrib/devtools/pixie.py), and using LIEF for all checks. LIEF for Linux is also currently blocked (on the next release, unless we want to build master) on one change for RISC-V that I [sent upstream](https://github.com/lief-project/LIEF/pull/562).
LIEF is seemingly well maintained, and is the basis for a number of other tools. It also has some very nice documentation; i.e the [Python API for ELF](https://lief.quarkslab.com/doc/latest/api/python/elf.html). It also has many builtins we can take advantage of. i.e [`is_pie`](https://lief.quarkslab.com/doc/latest/api/python/macho.html#lief.MachO.Binary.is_pie), [`has_nx`](https://lief.quarkslab.com/doc/latest/api/python/macho.html#lief.MachO.Binary.has_nx) etc. This means we can [consolidate some of our checks](9c5eeb5484). If/when end up using LIEF for lightning then we can consolidate further, and cleanup these scripts. i.e to not parse the binary inside the checks, but once at the start of the script.
Guix builds:
```bash
# find guix-build-$(git rev-parse --short=12 HEAD)/output/ -type f -print0 | env LC_ALL=C sort -z | xargs -r0 sha256sum
963a08638c46f9a3d75cd4b0c155d1ca091bbeba27167291adcd3dca03fd4c3d guix-build-f51237d94d98/output/aarch64-linux-gnu/bitcoin-f51237d94d98-aarch64-linux-gnu-debug.tar.gz
a3ce927c46b103789a010c41a6ebfafe4548d90ee7d88f2a735c9183b775da5c guix-build-f51237d94d98/output/aarch64-linux-gnu/bitcoin-f51237d94d98-aarch64-linux-gnu.tar.gz
2503ac8901068805d5e7251fd5cfeb7c1f8ba3528bdfcf3aa1e0c40bfd5c1cbc guix-build-f51237d94d98/output/arm-linux-gnueabihf/bitcoin-f51237d94d98-arm-linux-gnueabihf-debug.tar.gz
5798697e58e1788df85aa9e2e4d33fef0456169fcbd2521f13b3b5806ac0d84d guix-build-f51237d94d98/output/arm-linux-gnueabihf/bitcoin-f51237d94d98-arm-linux-gnueabihf.tar.gz
4185adebc6a0abe7241a3cd409a6ab7be031c26f1c4245e30bb5f87eef0925d2 guix-build-f51237d94d98/output/dist-archive/bitcoin-f51237d94d98.tar.gz
9b4b8756c5c84295eb6b61b6b32a07a8d07723fb38aaa8f519b6133935061bda guix-build-f51237d94d98/output/powerpc64-linux-gnu/bitcoin-f51237d94d98-powerpc64-linux-gnu-debug.tar.gz
cbd821aa464a9c16f7979dbec1a5e66939e777a567f55f7081499a8d528d42c5 guix-build-f51237d94d98/output/powerpc64-linux-gnu/bitcoin-f51237d94d98-powerpc64-linux-gnu.tar.gz
abed530a82e97e3cf621c90a13c0881b0e39ccce2a6f42a3ff80de76e2abc5f7 guix-build-f51237d94d98/output/powerpc64le-linux-gnu/bitcoin-f51237d94d98-powerpc64le-linux-gnu-debug.tar.gz
8b6d2bdd8b58ff1f6072bf8693abe3ce773ff3a7d8d2b7218207e69945b9d31b guix-build-f51237d94d98/output/powerpc64le-linux-gnu/bitcoin-f51237d94d98-powerpc64le-linux-gnu.tar.gz
d99cc705032d22ae819975992216899ed960ba25871a05c8789d00b80418511f guix-build-f51237d94d98/output/riscv64-linux-gnu/bitcoin-f51237d94d98-riscv64-linux-gnu-debug.tar.gz
5240ca4f4ef7c62088185224ac319ad9a4a9b40075df10af18d8a6355bca32fb guix-build-f51237d94d98/output/riscv64-linux-gnu/bitcoin-f51237d94d98-riscv64-linux-gnu.tar.gz
adc16eaee4b51e8615ce8b3be9f6c018698237df4ad6e0886cf0d4ab6bc9e5c4 guix-build-f51237d94d98/output/x86_64-apple-darwin18/bitcoin-f51237d94d98-osx-unsigned.dmg
b188af0572ee682d74cc82c7e6e464115205fc130a457cfe19d42ac9ddd267f8 guix-build-f51237d94d98/output/x86_64-apple-darwin18/bitcoin-f51237d94d98-osx-unsigned.tar.gz
e764062fde144e6fb5d6dd776c10fc2daa8d775831f7e43247d17a6c6e060c97 guix-build-f51237d94d98/output/x86_64-apple-darwin18/bitcoin-f51237d94d98-osx64.tar.gz
dab3d26ac94c669140f7329d14e57ef02b0fe92b8a8f9d96c32a416adea0da0f guix-build-f51237d94d98/output/x86_64-linux-gnu/bitcoin-f51237d94d98-x86_64-linux-gnu-debug.tar.gz
ca59d4379fbe2b9a52deebeaf88508e0eda4215f28d319aff0781289dd159712 guix-build-f51237d94d98/output/x86_64-linux-gnu/bitcoin-f51237d94d98-x86_64-linux-gnu.tar.gz
52b7c35321a85c4f6c95bf0e687574454b71ede9bec1c9cf17f37c578c888a94 guix-build-f51237d94d98/output/x86_64-w64-mingw32/bitcoin-f51237d94d98-win-unsigned.tar.gz
a543895a00f8ffb3ba50ca68396d52ad5a18dd8efe38730e0049dd70d283a092 guix-build-f51237d94d98/output/x86_64-w64-mingw32/bitcoin-f51237d94d98-win64-debug.zip
aec050d03c65268a986148500f7341cceb8c5f85287e0e3cde8933ce4b4dee32 guix-build-f51237d94d98/output/x86_64-w64-mingw32/bitcoin-f51237d94d98-win64-setup-unsigned.exe
57ba33ed6ee8d3a885e342471359301473e83037d5442895beb686921a4c50e9 guix-build-f51237d94d98/output/x86_64-w64-mingw32/bitcoin-f51237d94d98-win64.zip
```
Gitian builds:
```bash
# macOS:
2f066e852bdd30ac46e5ecdf7619d19d408035c318a3edf0f1893ec2e25efb69 bitcoin-41a1b3d1b130-osx-unsigned.dmg
8cf8ac4d21740f490262453c330b5f4a5c5b8139dfc1b322efefce3f3b93d1b2 bitcoin-41a1b3d1b130-osx-unsigned.tar.gz
cf1b84efdd9d2588a1ce9513580fb56b38bfafe60e18f8adbeedf03521c6c2b2 bitcoin-41a1b3d1b130-osx64.tar.gz
14995244b0bb3e80e7b79975c9c70fdfb3ee3c04fda3efd5358ce1c4efa3a312 src/bitcoin-41a1b3d1b130.tar.gz
93881069d5e1dc385c08895a7b035a94eb010325afc2776c99b6aafa21096eb8 bitcoin-core-osx-22-res.yml
# Windows:
4d56dd7713121684b7eaa448679c65df2fd0aa5319bf8d12fb6cfa9f0b005cf7 bitcoin-f51237d94d98-win-unsigned.tar.gz
4558f4173152b084bcba25aa1a53c605208a70fe20392141b63cefb476528c85 bitcoin-f51237d94d98-win64-debug.zip
b63feaca010e86d514cfe38d716e3c8a8b8058e4f969b868aaaeb8a8a3d3dc81 bitcoin-f51237d94d98-win64-setup-unsigned.exe
de7d8586cc91ba391fe911853a99d9fd15fc6f9a60f9b91a0447940173aac67a bitcoin-f51237d94d98-win64.zip
4185adebc6a0abe7241a3cd409a6ab7be031c26f1c4245e30bb5f87eef0925d2 src/bitcoin-f51237d94d98.tar.gz
45efaca35b5fad0a04dfd06e44f7c00b990aa91c7bf2faea57e020d3491a6cf0 bitcoin-core-win-22-res.yml
# Linux:
055d646c5f8cf4708008374546176012ff758566a2645a3a01e1a33eab1002fe bitcoin-f51237d94d98-aarch64-linux-gnu-debug.tar.gz
bfc8b0efc36b0474c88546b12d2723c04b4dc629ae311082025c7e0b8f0d1aa9 bitcoin-f51237d94d98-aarch64-linux-gnu.tar.gz
9dfaa5acfffadad8942b32996458013a155d12ed07be76601f232233627b5cb9 bitcoin-f51237d94d98-arm-linux-gnueabihf-debug.tar.gz
54eb57905ff8513b9f628707b61aa4659c362fb2f6d17e0ee240b4da3674907d bitcoin-f51237d94d98-arm-linux-gnueabihf.tar.gz
ad98d876616eff578ad8cfd17dfbabe48ed14200823579687d66694bae3d2fe3 bitcoin-f51237d94d98-powerpc64-linux-gnu-debug.tar.gz
fe1b421dd1cb6e04d5dc5d341459dc15fa6e15b80906e5d8e0405cf43495e0f7 bitcoin-f51237d94d98-powerpc64-linux-gnu.tar.gz
9001d95cc7d2722d9d7dd83d9da8e5adf575fddf91b615b76b9bcfece30ecf6f bitcoin-f51237d94d98-powerpc64le-linux-gnu-debug.tar.gz
9e0650ad2aba70c0fd1608a077e95f335dc1bb4a79eab9b0b56ac87427a4fd4f bitcoin-f51237d94d98-powerpc64le-linux-gnu.tar.gz
fbfde0134944d3dbd32991455b0a8abdd334853ab8a4c1a1a4c060d9de071c50 bitcoin-f51237d94d98-riscv64-linux-gnu-debug.tar.gz
2fa2cfddce98c44c65305326fc623a7f065129208337503d813a08d51580cb8a bitcoin-f51237d94d98-riscv64-linux-gnu.tar.gz
b2d6caeee0e3c350a43165c39876ebed8e588958007af0d06996e341c7060683 bitcoin-f51237d94d98-x86_64-linux-gnu-debug.tar.gz
bfdb827e75d43d61462513c9a843620b93c9160d9d246cad13278baaa07f64ea bitcoin-f51237d94d98-x86_64-linux-gnu.tar.gz
4185adebc6a0abe7241a3cd409a6ab7be031c26f1c4245e30bb5f87eef0925d2 src/bitcoin-f51237d94d98.tar.gz
34820a093916fa35b0fd98806a50092f46b20271af7422f43e2a4223ef6f9bb7 bitcoin-core-linux-22-res.yml
```
ACKs for top commit:
laanwj:
re-ACK 7fc5e865b9
Tree-SHA512: 0c30838413448ecfcf55e6273f607fdb01cb1acafa1d2762afad59360fca7d8efa78ec55064f50cba56cb2c9e98741e13665cba8e9b4b8e5b62b8a53f9bf8990
c90f6e5109 guix: Consistently use gcc-8 for $HOST (Carl Dong)
Pull request description:
Only non-base commit is the last commit: b5abb07d0d
Right now, here's what we use in Gitian:
- Linux: Focal's [`g++-8-<arch>-linux-gnu`](https://packages.ubuntu.com/focal/g++-8-aarch64-linux-gnu) (`8.4.0-3ubuntu1cross1`)
- MinGW-w64: Focal's [`g++-mingw-w64`](https://packages.ubuntu.com/focal/g++-mingw-w64) (`9.3.0-7ubuntu1+22~exp1ubuntu4`)
In Guix right now we use `gcc-9` across the board.
I think it makes more sense to use `gcc-8` across the board, as it doesn't suffer from the `memcmp` bug, and is what debian buster (stable) does, meaning it will be well tested ([`g++-mingw-w64`](https://packages.debian.org/buster/g++-mingw-w64), [`g++-aarch64-linux-gnu`](https://packages.debian.org/buster/g++-aarch64-linux-gnu)).
We can accomplish this somewhat easily using Guix as we have tighter control over the toolchain (see: b5abb07d0d).
Let me know your thoughts!
ACKs for top commit:
MarcoFalke:
Approach ACK c90f6e5109, haven't reviewed
laanwj:
Code review ACK c90f6e5109
hebasto:
ACK c90f6e5109, I have reviewed the code and it looks OK, I agree it can be merged.
Tree-SHA512: 3e5b9297305232273323aa745ec417ed1be2418ead0e432db7742f5d5f45efe6e4a2ed44328731512cff4bfde80e5f2dc350a131b8b8fb9207a2ef66bce27ed2
142e2da440 net: add I2P seeds to chainparamsseeds (Jon Atack)
e01f173fb9 contrib: add a few I2P seed nodes (Jon Atack)
ea269c7ef1 contrib: parse I2P addresses in generate-seeds.py (Jon Atack)
Pull request description:
Follow-up to #21560 that updated the fixed seeds infra for BIP155 addresses and then added Tor v3 ones:
- Update contrib/generate-seeds.py to parse I2P addresses
- Add a few I2P nodes to contrib/seeds/nodes_main.txt
- Run generate-seeds.py and add the I2P seeds to chainparamsseeds.h
Reviewers, see contrib/seeds/README.md for more info and feel free to use the following CLI one-liner to check for and propose additional seeds for contrib/seeds/nodes_main.txt. You can also see how many I2P peers your node knows with cli -addrinfo.
```rake
bitcoin-cli getnodeaddresses 0 | jq '.[] | (select(.address | contains(".b32.i2p"))) | .address' | sort
```
I verified the I2P addresses are correctly BIP155-serialized/deserialized by building with all seeds removed from chainparamsseeds.h except those added here, restarting with `-datadir=newdir -dnsseed=0` and running rpc ` getnodeaddresses 0` that initially returns only the new I2P addresses.
ACKs for top commit:
laanwj:
ACK 142e2da440
vasild:
ACK 142e2da440
Tree-SHA512: 040576012d5f1f034e2bd566ad654a6fdfd8ff7f6b12fa40c9fda1e948ebf8417fcea64cfc14938a41439370aa4669bab3e97274f9d4f9a6906fa9520afa9cf8
We already attest to the relevant dist-archive in inputs.SHA256SUMS,
which is recorded at build-time.
We use a SKIPATTEST.TAG file to indicate output directories which do not
require attestation (much like the CACHEDIR.TAG specification).
Generally, it's better to have build scripts declare properties of
directories instead of introducing name-based special cases in attest
scripts since build scripts have a more detailed context of what is
going on.
At build/codesigning-time, hash build inputs and output the digest to
${OUTDIR}/inputs.SHA256SUMS, which gets included in the final SHA256SUMS
constructed by guix-attest.
Example final SHA256SUMS:
ee832d2a35b7701bff581dea05a536118b118e3ad0a587a2855b6ee8cd6fba20 inputs/bitcoin-78199266af7b.tar.gz
ca765e70a0c12866dd63c0be228b675278a26329e5f8f5b5c52fd09200fedf21 bitcoin-78199266af7b-powerpc64le-linux-gnu-debug.tar.gz
dae95327d7f2c324e2728c4b73627be6cb2c0d2f2e5bea940d1d5e6463939327 bitcoin-78199266af7b-powerpc64le-linux-gnu.tar.gz
While files are being output to $OUTDIR, it will be under
${DISTSRC}/output, and only when everything is done, will
${DISTSRC}/output be moved to the actual $OUTDIR.
This makes it so that a Ctrl-C in the middle of a build is less likely
to result in a partially-constructed $OUTDIR. In fact, if I understand
correctly, if $OUTDIR and $DISTSRC reside on the same filesystem, the
move (rename) is likely atomic.
Also, since the "working $OUTDIR" is under ${DISTSRC}/output, it will be
cleaned properly by the guix-clean script.
c799a19b4b build, qt: No longer need to set QT_RCC_TEST=1 for determinism (Hennadii Stepanov)
Pull request description:
The Qt Resource Compiler (rcc) output order relies on [`QHash`](https://doc.qt.io/qt-5/qhash.html):
> This randomization of `QHash` is enabled by default. Even though programs should never depend on a particular `QHash` ordering, there may be situations where you temporarily need deterministic behavior, for example for debugging or regression testing. To disable the randomization, define the environment variable `QT_HASH_SEED` to have the value 0.
Since #3620 we use `QT_RCC_TEST=1` to achieve a deterministic output.
Since Qt 5.3.1 hash seeding is disabled for all of the bootstrapped tools, including rcc. Therefore, `QT_RCC_TEST=1` is no longer needed.
See commit [5283a6c87beac5a43f612786fefd6e43f2c70bf6](5283a6c87b).
ACKs for top commit:
fanquake:
ACK c799a19b4b
Tree-SHA512: 9d116ac1e8c605ee3e8ed7f618586f0de85d8b06bbbb70fe8c298939ce203d2a7e97264a9afac037179993ab54c5f69a65ebb9ab27ca7f45acb963011bd45743
55d85834cc script: Add trusted key for hebasto (Hennadii Stepanov)
Pull request description:
It is assumed that my responsibility will be limited to the [GUI repo](https://github.com/bitcoin-core/gui).
ACKs for top commit:
laanwj:
ACK 55d85834cc
MarcoFalke:
matches the key I have locally ACK 55d85834cc🍪
jarolrod:
ACK 55d85834cc🥃
Tree-SHA512: 256d03e108c9a14e251340ac6e91234d076778cb6bd551439182176207051f4efc55d396754867e5a7191c8c698610f92016668e163037c67dde56f4136026b8
Passing ADDITIONAL_GUIX_COMMON_FLAGS="--no-substitutes --bootstrap" as
suggested doesn't work:
```bash
...outputting in: '/bitcoin/guix-build-a1f0b8b62eb8/output/x86_64-linux-gnu'
...bind-mounted in container to: '/outdir-base/x86_64-linux-gnu'
guix time-machine: error: bootstrap: unrecognized option
```
and I think bootstrapping is more than covered in the preceding "Choose
your security model" section.
867a5e172a guix: Register garbage collector root for containers (Carl Dong)
8f8b96fb54 guix: Update hint messages to mention guix-clean (Carl Dong)
44f6d4f56b guix: Record precious directories and add guix-clean (Carl Dong)
84912d4b24 build: Remove spaces from variable-printing rules (Carl Dong)
Pull request description:
```
guix: Record precious directories and add guix-clean
Many users have reported problems that stem from having an unclean
working tree. To that end, I've written a guix-clean script which should
help reset the working tree while respecting user-specified precious
directories.
Precious directories, such as:
- SOURCES_PATH
- BASE_CACHE
- SDK_PATH
- OUTDIR
Should be preserved when cleaning the working tree, and are thus
recorded in ./contrib/guix/var/precious_dirs.
The ./contrib/guix/guix-clean script is able to parse that file and make
sure to avoid them when cleaning out the working tree.
```
ACKs for top commit:
laanwj:
ACK 867a5e172a
Tree-SHA512: c498fad781ff5e6406639df2b91b687fc528273fdf266bcdba8f6eec3b3b37ecce544b6da0252f0b9c6717f9d88e844e4c7b72d1877bdbabfc6871ddd0172af5
By registering the container profiles as garbage collector roots, it
will prevent `guix gc` from garbage collecting derivations which our
container needs and inconvieniencing the user with a rebuild.
b2ee8b207d net: Deserialize hardcoded seeds from BIP155 blob (W. J. van der Laan)
9b29d5df7f contrib: Add explicit port numbers for testnet seeds (W. J. van der Laan)
2a257de113 contrib: Add a few TorV3 seed nodes (W. J. van der Laan)
06030f7a42 contrib: generate-seeds.py generates output in BIP155 format (W. J. van der Laan)
Pull request description:
Closes#20239 and mitigates my node's problem in #21351.
- Add a few hardcoded seeds for TorV3
- As the [bitcoin-seeder](https://github.com/sipa/bitcoin-seeder) doesn't collect TorV3 addresses yet, I have extracted these from my own node using [a script](https://gist.github.com/laanwj/b3d7b01ef61ce07c2eff0a72a6b90183) and added them manually. This is intended to be a temporary stop gap until 22.0's seeds update.
- Change hardcoded seeds to variable length BIP155 binary format.
- It is stored as a single serialized blob in a byte array, instead of pseudo-IPv6 address slots. This is more flexible and, assuming most of the list is IPv4, more compact.
- Only the (networkID, addr, port) subset (CService). Services and time are construed on the fly as before.
- Change input format for `nodes_*.txt`.
- Drop legacy `0xAABBCCDD` format for IPv4. It is never generated by `makeseeds.py`.
- Stop interpreting lack of port as default port, interpret it as 'no port', to accomodate I2P and other port-less protocols (not handled in this PR). An explicit port is always generated by `makeseeds.py` so in practice this makes no difference right now.
A follow-up to this PR could do the same for I2P.
ACKs for top commit:
jonatack:
ACK b2ee8b207d
Tree-SHA512: 11a6b54f9fb0192560f2bd7b218f798f86c1abe01d1bf37f734cb88b91848124beb2de801ca4e6f856e9946aea5dc3ee16b0dbb9863799e42eec1b239d40d59d
Many users have reported problems that stem from having an unclean
working tree. To that end, I've written a guix-clean script which should
help reset the working tree while respecting user-specified precious
directories.
Precious directories, such as:
- SOURCES_PATH
- BASE_CACHE
- SDK_PATH
- OUTDIR
Should be preserved when cleaning the working tree, and are thus
recorded in ./contrib/guix/var/precious_dirs.
The ./contrib/guix/guix-clean script is able to parse that file and make
sure to avoid them when cleaning out the working tree.
7476b46f18 guix: Build dmg as a static binary (Carl Dong)
06d6cf6784 depends: libdmg-hfsplus: Skip CMake RPATH patching (Carl Dong)
65176ab573 guix: Remove codesign_allocate+pagestuff from unsigned tarball (Carl Dong)
ca85679eb4 guix: Use clang-toolchain instead of clang (Carl Dong)
1aec0eda8f guix: Fallback to local build for substitute-enabled Guix users (Carl Dong)
1742f8e12d guix: Add early health check for guix-daemon (Carl Dong)
c1ae726a13 guix: More thoroughly control native toolchain (Carl Dong)
39741128d3 guix: Supply --link-profile (Carl Dong)
d55a1056ee guix: Add troubleshooting documentation entries (Carl Dong)
7f401c953f guix: Adapt guix-build to prelude, restructure hier (Carl Dong)
4eccf063b2 guix: Remove guix-build.sh filename extension (Carl Dong)
7753357a7b guix: Add source-able bash prelude and utils (Carl Dong)
e5b49a01f5 guix: Create windeploy inside distsrc-* (Carl Dong)
3e9982ab38 contrib: Silence git-describe when looking for tag (Carl Dong)
d5a71e9785 guix: Use --cores instead of --max-jobs (Carl Dong)
Pull request description:
This PR addresses a few hiccups encountered by the brave souls who've been experimenting with the Guix scripts:
- Resolves confusion between `--cores=` and `--max-jobs=`
- `guix`'s `--cores=` actually corresponds to make's `--jobs=`, so let's just control `--cores=` with our overridable env var
- `git-describe` will scream `fatal: no tag exactly matches '<hash>'` when looking for a tag, but we don't care, so silence that
- `windeploy/unsigned` should be inside `distsrc-*` and created idempotently (sorry I know this one annoyed people)
- Add troubleshooting documentation to `README.md`
- Add early health check for `guix-daemon` in case user forgot to start a `guix-daemon`
- Depending on configuration, a `--fallback` flag may be needed to tell Guix to not fail if substitutes fail but fallback to building locally
- `codesign_allocate` and `pagestuff` are now unnecessary for codesigning as we're now using `signapple`
A few robustness changes are also included:
- We supply the `--link-profile` flag, as some Guix packages may expect the profile to be available under `$HOME/.guix-profile`
- We now clear and manually set all toolchain-related env vars (e.g. `C*_INCLUDE_PATH`) ourselves, after patching a Qt::moc bug
- We use the native `clang-toolchain` package for darwin builds instead of `clang`, lining up with all our other toolchain packages.
Finally, we restructure the guix building hierarchy such that it looks something like:
```
guix-build-<short-hash-or-version-tag>
├── distsrc-<short-hash-or-version-tag>-${HOST}
│ ├── contrib
│ ├── depends
│ ├── src
│ └── ...
├── distsrc-<short-hash-or-version-tag>-...
└── output
├── dist-archive
│ └── bitcoin-<short-hash-or-version-tag>.tar.gz
├── *-linux-*
│ ├── bitcoin-<short-hash-or-version-tag>-*-linux-*-debug.tar.gz
│ └── bitcoin-<short-hash-or-version-tag>-*-linux-*.tar.gz
├── x86_64-apple-darwin18
│ ├── bitcoin-<short-hash-or-version-tag>-osx64.tar.gz
│ ├── bitcoin-<short-hash-or-version-tag>-osx-unsigned.dmg
│ └── bitcoin-<short-hash-or-version-tag>-osx-unsigned.tar.gz
└── x86_64-w64-mingw32
├── bitcoin-<short-hash-or-version-tag>-win64-debug.zip
├── bitcoin-<short-hash-or-version-tag>-win64-setup-unsigned.exe
├── bitcoin-<short-hash-or-version-tag>-win64.zip
└── bitcoin-<short-hash-or-version-tag>-win-unsigned.tar.gz
```
Separating guix builds by their version identifier (basically namespacing them) allows us to change the layout in the future without worry about potential naming conflicts.
ACKs for top commit:
sipa:
ACK 7476b46f18
laanwj:
ACK 7476b46f18
Tree-SHA512: 0e899aa941aafdf552b2a7e8a08131ee9283180bbef7334439e2461a02aa7235ab7b9ca9c149b80fc5d0a9f4bbd35bc80fcee26197c0836ba8eaf2d86ffa0386
This relatively easy change eliminates all runtime dependencies (except
for the kernel) for dmg, which is the only native build tool that gets
put in our output tarballs.
This allows much more flexibility when constructing the codesigning
environment, and is much more robust.
./windeploy is a "working directory", and therefore belongs inside
distsrc-*. Many people have noticed their Guix builds failing after
hours simply because they did not remove windeploy (but did remove the
distsrc-* directories).
In Guix, there are two flags for controlling parallelism:
Note: When I say "derivation," think "package"
--cores=n
- controls the number of CPU cores to build each derivation. This is
the value passed to `make`'s `--jobs=` flag.
- defaults to 0: as many cores as is available
--max-jobs=n
- controls how many derivations can be built in parallel
- defaults to 1
Therefore, if set --max-jobs=$MAX_JOBS and don't set --cores, Guix could
theoretically spin up $MAX_JOBS * $(nproc) number of threads, and that's
no good.
So we could either default to --cores=1, --max-jobs=$MAX_JOBS
- Pro: --cores=1 means that `make` will be invoked with `-j1`,
avoiding problems with package whose build systems and test
suites break when running multi-threaded.
- Con: There will be times when only 1 or 2 derivations can be built
at a time, because the rest of the dependency graph all depend
on those 1 or 2 derivations. During these times, the machine
will be severely under-utilized.
or --cores=$MAX_JOBS, --max-jobs=1
- Pro: We don't encounter prolonged periods of
severe under-utilization mentioned above.
- Con: Many packages' build systems and test suites break when running
multi-threaded.
or --cores=1, --max-jobs=1 and let the user override with
$ADDITIONAL_GUIX_COMMON_FLAGS
3a0446fad4 script: Add explanatory comment to tc.sh (dscotese)
Pull request description:
This is a replacement for #21289
tc.sh is used to limit bandwidth. I ran it and it is limiting my bandwidth. When I ran it, I got one error. I have not found an explanation anywhere of what the error means, but my best guess is consistent with the result, so I propose the explanatory comment to save others time when they use it and also get the error.
ACKs for top commit:
laanwj:
that said, LGTM ACK 3a0446fad4
Tree-SHA512: 5403a2a0fec3724625c20402a96334c3c7a620324a930c5fd828017da8911d2867aecb7a2ad94a23d1f189009d3eb197a67eb59c8e4531fd215d9b1edb600440
663f6cd9dd contrib: Use -daemonwait in systemd init script (Wladimir J. van der Laan)
Pull request description:
Make systemd invoke dependencies only when ready by using `-daemonwait` in the service file instead of `-daemon`.
Closes#21322 by making bitcoind conform to behavior specified for `type=forking`.
This may need some tuning of timeouts.
ACKs for top commit:
darosior:
ACK 663f6cd
hebasto:
re-ACK 663f6cd9dd
Tree-SHA512: 890005852b632a202caa578e6c796ebdc9da0b2379a9157a4f56f7db9d193c0ffbb78d120bbf112ab2f273855f2a08c3da000b1f7a9fb5222a3b94dcdb16b878
Because only macOS wasy mentioned, I was unsure if this would be a macOS specific tool. I guess Linux is more used than Mac, so Linux guide should be there, too.
0fc0c00f7a test: Drop unused get_machine function (Hennadii Stepanov)
61a0f8f9cc test: Cleanup test files in test-{security,symbol}-check.py (Hennadii Stepanov)
Pull request description:
1) Test source and executable files are neither ignored by `.gitignore` nor removed by `make clean` and `make distclean`.
2) The `get_machine` function is no longer used since #21255.
ACKs for top commit:
fanquake:
ACK 0fc0c00f7a
Tree-SHA512: ef3fcf22d4a04b6e4f37f748bd4be57e09696d2a77982e26292843cb2a1297789c8325f5c4bdad37d8094fce7765c4cc9ab19809e07471487943361b2b1a252c
e4c0cada79 ci, gitian: Drop unneeded python3-dev package for macOS builds (Hennadii Stepanov)
Pull request description:
ACKs for top commit:
fanquake:
ACK e4c0cada79 - gitian builds match and I checked that this doesn't end up installed as a side-effect of another package.
Tree-SHA512: 520a3909b106a0e005b195c5395691edf62b76ee2df43b6971b7aa193648d68e6dac69cb4f1dc474f594b015a2fc2074061865e571d89365174beb5c1780356f
95f97111dd contrib/init: (OpenRC) quote some unquoted variables. (parazyd)
737feadff7 contrib/init: (OpenRC) Do not fail if both rpcuser and rpcpassword are unset. (parazyd)
Pull request description:
This pull request improves the available OpenRC initscripts in
`contrib/init`.
The first commit (737feadff7) reworks
`checkconfig()` to not fail if **both** `rpcuser` and `rpcpassword`
are unset, because this implies that bitcoind shall use the `.cookie`
file for RPC authentication. Currently, the initscript does not allow
starting bitcoind without a set `rpcuser` and `rpcpassword`.
The second commit (95f97111dd) simply
quotes some unquoted variables.
ACKs for top commit:
kristapsk:
ACK 95f97111dd
Tree-SHA512: 62bebcd07143c147e349c0cfc17b54ef21bd4684377b444f58c6bd1f509a4d3e1af58746fa7215f18e33021f691bbbc5e42f4df497458322b055e545b7f30d46
remove fix_configure_mac.patch
Fixed upstream: https://bugreports.qt.io/browse/QTBUG-67286
remove fix_riscv64_arch.patch
Was fixed upstream in 6a39e49a6cdeb28a04a3657bb6a22f848d5dfa9d
remove fix_rcc_determinism.patch
Fixed upstream in https://bugreports.qt.io/browse/QTBUG-62511
remove freetype_back_compat.patch
By the time we ship a release with Qt 5.12, we'll certainly no-longer be
supporting Ubuntu 14.04 and Ubuntu 16.04 ships with FreeType 2.6.1,
which is new enough that using the symbol is no-longer an issue.
The renaming of FT_Get_X11_Font_Format() happened in FreeType 2.6
remove xkb-default.patch
This was removed upstream in d5abf545971da717014d316127045fc19edbcd65
Co-authored-by: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
