While glibc 2.25 and newer *can* be built with stack-smashing-protection
enabled, it isn't used by default, and still isn't, as of glibc 2.35,
so I can't see a reason to explicitly disable it.
I'd also like to move in the direction of enabling, by default,
hardening options for the toolchains we build, so removing the explicit
disabling is a step in that direction.
Will be following up with some changes based on this PR.
667e316bcb contrib: Update makeseeds to asmap-nextgen (laanwj)
ae00b9e02c contrib: add seeds progress indicator and remove asmap one in makeseeds script (Jon Atack)
b54180303d contrib: Use asmap for ASN lookup in makeseeds (laanwj)
Pull request description:
Add an argument `-a` to provide a asmap file to do the IP to ASN lookups.
This speeds up the script greatly, and makes the output deterministic. Also removes the dependency on `dns.lookup`.
I've annotated the output with ASxxxx comments to provide a way to verify the functionality.
For now I've added instructions in README.md to download and use the `demo.map` from the asmap repository. When we have some other mechanism for distributing asmap files we could switch to that.
This continues #24824. I've removed the fallbacks and extra complexity, as everyone will be using the same instructions anyway.
Co-authored-by: Pieter Wuille <pieter.wuille@gmail.com>
Co-authored-by: russeree <reese.russell@ymail.com>
ACKs for top commit:
sipa:
ACK 667e316bcb
dunxen:
re-ACK 667e316
Tree-SHA512: c4cedfbd1dee6be7547aa92dd9e262c46f0ff8099e647559b2a40eab0cc9874e9a813706630dd5c880390d23f432e789fb3e7e8a09f376f567071e68f5904c65
As of version 2.4.7, libtool now respects ARFLAGS, which we use, and has
changed the default ARFLAGS from cru to cr (which we also do, see
configure).
This eliminates spammy `ar` output such as:
```bash
CXXLD libunivalue.la
/root/.guix-profile/bin/x86_64-linux-gnu-ar: `u' modifier ignored since `D' is the default (see `U')
AR libbitcoin_zmq.a
AR libbitcoin_consensus.a
CXXLD crypto/libbitcoin_crypto_base.la
CXXLD crypto/libbitcoin_crypto_sse41.la
/root/.guix-profile/bin/x86_64-linux-gnu-ar: `u' modifier ignored since `D' is the default (see `U')
/root/.guix-profile/bin/x86_64-linux-gnu-ar: `u' modifier ignored since `D' is the default (see `U')
CXXLD crypto/libbitcoin_crypto_avx2.la
CXXLD crypto/libbitcoin_crypto_x86_shani.la
CXXLD leveldb/libleveldb.la
/root/.guix-profile/bin/x86_64-linux-gnu-ar: `u' modifier ignored since `D' is the default (see `U')
CXXLD crc32c/libcrc32c.la
/root/.guix-profile/bin/x86_64-linux-gnu-ar: `u' modifier ignored since `D' is the default (see `U')
CXXLD leveldb/libmemenv.la
/root/.guix-profile/bin/x86_64-linux-gnu-ar: `u' modifier ignored since `D' is the default (see `U')
/root/.guix-profile/bin/x86_64-linux-gnu-ar: `u' modifier ignored since `D' is the default (see `U')
/root/.guix-profile/bin/x86_64-linux-gnu-ar: `u' modifier ignored since `D' is the default (see `U')
AR libbitcoin_cli.a
```
Libtool 2.4.7 release notes:
https://lists.gnu.org/archive/html/autotools-announce/2022-03/msg00000.html
Mostly changes to remove src/univalue exceptions from the various linters,
and the required code changes to make them happy. As well as minor doc
changes.
Now that we use GCC 10 for release builds, we no-longer need to
pass-Wl,-z,noexecstack to get a non-executable stack in RISC-V binaries.
This was originally removed in #21036, but then re-added in #21799, when
we reverted to using GCC 8.
Add an argument `-a` to provide a asmap file to do the IP to ASN
lookups.
This speeds up the script greatly, and makes the output deterministic.
Also removes the dependency on `dns.lookup`.
I've annotated the output with ASxxxx comments to provide a way to
verify the functionality.
For now I've added instructions in README.md to download and use the
`demo.map` from the asmap repository. When we have some other mechanism
for distributing asmap files we could switch to that.
This continues #24824. I've removed all the fallbacks and extra
complexity, as everyone will be using the same instructions anyway.
Co-authored-by: Pieter Wuille <pieter.wuille@gmail.com>
Co-authored-by: James O'Beirne <james.obeirne@pm.me>
Co-authored-by: russeree <reese.russell@ymail.com>
7e9fe6d800 windeploy: Renewed windows code signing certificate (Andrew Chow)
Pull request description:
The current windows code signing certificate expires on May 26 23:59:59 2022 GMT. I have purchased a new code signing certificate which will expire on May 29 23:59:59 2024 GMT.
ACKs for top commit:
laanwj:
ACK 7e9fe6d800
fanquake:
ACK 7e9fe6d800 - tested above with OpenSSL 3 & faketime.
Tree-SHA512: 283eb863d4db0573c7e78fe9d8f1b855533fc45b0995cd2d66e40b5242eb9bc9317b01e1b151fe49d512cd4aa6c48e2390017070f79db46493813fdd0a0f568a
Compiling glibc 2.24 and 2.27 with the new GCC 10 results in a number of new warnings,
i.e:
```bash
libc-tls.c: In function ‘__libc_setup_tls’:
libc-tls.c:208:30: error: array subscript 1 is outside the bounds of an interior zero-length array ‘struct dtv_slotinfo[0]’ [-Werror=zero-length-bounds]
208 | static_slotinfo.si.slotinfo[1].map = main_map;
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~
In file included from ../sysdeps/x86_64/ldsodefs.h:54,
from ../sysdeps/gnu/ldsodefs.h:46,
from ../sysdeps/unix/sysv/linux/ldsodefs.h:25,
from libc-tls.c:20:
../sysdeps/generic/ldsodefs.h:398:7: note: while referencing ‘slotinfo’
398 | } slotinfo[0];
| ^~~~~~~~
```
While we could try and backport all the patches required to fix these up, it would
currently seem easier to disable -Werror, which Guix uses by default when building
glibc.
There are two reasons to perform this bump:
* Fixes#25082 by bumping to a commit that includes a fix for time-dependent unit
tests in libgit2 (f5fe0082abe4547f3fb9f29d8351473cfb3a387b).
* Gives us access to clang-toolchain-14 (14.0.3, 998eda3067c7d21e0d9bb3310d2f5a14b8f1c681),
which is useful for the Guix portion of #21778.
Note that with this bump:
Linux kernels headers update from 5.15.28 to 5.15.37.
eb02713efc doc: add minimum required kernel version to dependencies.md (fanquake)
dcad5f70f1 guix: consolidate kernel headers to 5.15 (fanquake)
Pull request description:
Our minimum supported kernel version is currently defined by Guix, as the version passed to the [`--enable-kernel=`](https://www.gnu.org/software/libc/manual/html_node/Configuring-and-compiling.html) option when configuring glibc. That version is [currently set to 3.2.0](https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/base.scm?id=34e9eae68c9583acce5abc4100add3d88932a5ae#n776):
```scheme
;; This is the default for most architectures as of GNU libc 2.26,
;; but we specify it explicitly for clarity and consistency. See
;; "kernel-features.h" in the GNU libc for details.
"--enable-kernel=3.2.0"
```
and has been that way since we started using Guix (i.e from Guix 1.3.0, with the release v22.0).
Passing `--enable-kernel` defines `__LINUX_KERNEL_VERSION` inside glibc, which is then used to determine supported features & syscall usage. For example, some defines in `unix/sysv/linux/kernel-features.h`, from glibc version 2.24, where glibcs default supported kernel version was still 2.6.32 (it's more modern as of recent releases):
```cpp
#ifndef __LINUX_KERNEL_VERSION
/* We assume the worst; all kernels should be supported. */
# define __LINUX_KERNEL_VERSION0
#endif
/* Support for various CLOEXEC and NONBLOCK flags was added in
2.6.23. */
#define __ASSUME_O_CLOEXEC1
/* prlimit64 is available in 2.6.36. */
#if __LINUX_KERNEL_VERSION >= 0x020624
# define __ASSUME_PRLIMIT641
#endif
```
Note that because we currently specify the `5.15` headers, the exact version being used, i.e 5.15.x, changes when we update our time-machine commit, as Guix updates all it's header packages as new point releases become available. Currently it is [`5.15.28`](https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/linux.scm?id=34e9eae68c9583acce5abc4100add3d88932a5ae#n380). The changelog for the 5.15 headers is available [here](https://cdn.kernel.org/pub/linux/kernel/v5.x/ChangeLog-5.15.36).
Similar to glibc, it may currently be possible to build and run bitcoind against older kernels, however, for the purposes of documenting what we support for our release binaries, I can't see a reason to document anything other than the version that we are targeting when building the glibc used to build the release binaries.
Guix Build (on x86_64):
```bash
22ff2d3a72d337c4eccbfa4c834a67c7c3397f225aedb71a3c636f2708964e93 guix-build-eb02713efc17/output/aarch64-linux-gnu/SHA256SUMS.part
e316b2d0806183e0e51a25722f48af85d145b1581f44f68b925d9f484a5aa0d3 guix-build-eb02713efc17/output/aarch64-linux-gnu/bitcoin-eb02713efc17-aarch64-linux-gnu-debug.tar.gz
da667d935d9331c5cbca2e0e334cd7e56202ee294553459672fa50f13f501c4d guix-build-eb02713efc17/output/aarch64-linux-gnu/bitcoin-eb02713efc17-aarch64-linux-gnu.tar.gz
f44f2921f3dac2c545806dffb579137fb9eeeee15671395f6a7f817ed6213143 guix-build-eb02713efc17/output/arm-linux-gnueabihf/SHA256SUMS.part
101bab25ab2a6c36729cdf840264a85700cc3cfa23d3900b0bee0ac9ae637e8d guix-build-eb02713efc17/output/arm-linux-gnueabihf/bitcoin-eb02713efc17-arm-linux-gnueabihf-debug.tar.gz
9f7338ee42234949ef3104c6cb2b8a723d616a46d0047d833062adc92e3b6b72 guix-build-eb02713efc17/output/arm-linux-gnueabihf/bitcoin-eb02713efc17-arm-linux-gnueabihf.tar.gz
5ca08e7b38c4dd4456145602f25b015c164c0cd0317dd2a26855dc0495605418 guix-build-eb02713efc17/output/arm64-apple-darwin/SHA256SUMS.part
186c9e2df4034472107964e835ee7a05777c7a0fa5e0db12b5740f18e732d7d5 guix-build-eb02713efc17/output/arm64-apple-darwin/bitcoin-eb02713efc17-arm64-apple-darwin-unsigned.dmg
4e252c7775662777ddcb0a1b0efac6b6e71c25479d6b44b821199ae00abd18ca guix-build-eb02713efc17/output/arm64-apple-darwin/bitcoin-eb02713efc17-arm64-apple-darwin-unsigned.tar.gz
4360342994d54839bbf5fb4d86c6c0b0a3cbcef68b0d2c991aab6e81301638e7 guix-build-eb02713efc17/output/arm64-apple-darwin/bitcoin-eb02713efc17-arm64-apple-darwin.tar.gz
f354822050cd625de7445cfa317475cfad90a7e39d135c5b99950ee69969f445 guix-build-eb02713efc17/output/dist-archive/bitcoin-eb02713efc17.tar.gz
aa864574dd692bb40df95e17c08113f5bcd04b7d5997a2f74ca0557cae3edfcd guix-build-eb02713efc17/output/powerpc64-linux-gnu/SHA256SUMS.part
45df2a4ec5592834fd08d36068cd0968de281870cb9df48cc4783078b85985dd guix-build-eb02713efc17/output/powerpc64-linux-gnu/bitcoin-eb02713efc17-powerpc64-linux-gnu-debug.tar.gz
2bc1c481635ac073d61f51db425576a42a54b16218a00e5f80579426dbf9677b guix-build-eb02713efc17/output/powerpc64-linux-gnu/bitcoin-eb02713efc17-powerpc64-linux-gnu.tar.gz
403d7790e611d3e07b25e02549c9f50e51fff2e1a323605db4f9a569712771a7 guix-build-eb02713efc17/output/powerpc64le-linux-gnu/SHA256SUMS.part
db40032c3b25d95012496f1b3fa5df7f207dcbeefa510bd140b96df4dfd84c88 guix-build-eb02713efc17/output/powerpc64le-linux-gnu/bitcoin-eb02713efc17-powerpc64le-linux-gnu-debug.tar.gz
eea8b4ee96dc8a9813b727550bf07202a6f9cba99605247813beb5251c7f2623 guix-build-eb02713efc17/output/powerpc64le-linux-gnu/bitcoin-eb02713efc17-powerpc64le-linux-gnu.tar.gz
7164e539d25c300b993a620caacdcef659bb6a7c4775a873e30ee645c9ceed15 guix-build-eb02713efc17/output/riscv64-linux-gnu/SHA256SUMS.part
3ae271e6fb94e5d4e46a402508a02d659e879d222c6696e57c78530157eb39ae guix-build-eb02713efc17/output/riscv64-linux-gnu/bitcoin-eb02713efc17-riscv64-linux-gnu-debug.tar.gz
8c37d0b790c28b692804b360605baec4371af4f080c0024ba75f06c0096a4356 guix-build-eb02713efc17/output/riscv64-linux-gnu/bitcoin-eb02713efc17-riscv64-linux-gnu.tar.gz
51bbf04cb32b579ba5609fe3ef24e9901f8d49e3311fe9776ee1fdb644f7e0b1 guix-build-eb02713efc17/output/x86_64-apple-darwin/SHA256SUMS.part
8e81f8badb0cff1aa430a899065cf9744b4b2d45addb8e30606a2f8bf08faa26 guix-build-eb02713efc17/output/x86_64-apple-darwin/bitcoin-eb02713efc17-x86_64-apple-darwin-unsigned.dmg
8b4120b6d83c03dae34b0b5a189522d01c523ab005d816339fdfddf9c412ef15 guix-build-eb02713efc17/output/x86_64-apple-darwin/bitcoin-eb02713efc17-x86_64-apple-darwin-unsigned.tar.gz
26e633faba4f05f51f4e0bffaa2bbbf8c2d5d134d6777c9395bf9b65af6a808c guix-build-eb02713efc17/output/x86_64-apple-darwin/bitcoin-eb02713efc17-x86_64-apple-darwin.tar.gz
16c96d1f349ca3fbf5ffb8e00d5defe1af5a14abb6f61abdbd367e9a5e99bf33 guix-build-eb02713efc17/output/x86_64-linux-gnu/SHA256SUMS.part
faa203c9c3943c2f30ca3f4f30c3eee52e38ac9a2f15c6303b0c8ff0be146e07 guix-build-eb02713efc17/output/x86_64-linux-gnu/bitcoin-eb02713efc17-x86_64-linux-gnu-debug.tar.gz
b5cf5154ac0e2138a4ccbc7639026d909e606b9f55c5859ae54d941eb950759b guix-build-eb02713efc17/output/x86_64-linux-gnu/bitcoin-eb02713efc17-x86_64-linux-gnu.tar.gz
bc14a09399ef3a6d9696116ddda6509b6cb1726719dfd462106cb9d2fde32efc guix-build-eb02713efc17/output/x86_64-w64-mingw32/SHA256SUMS.part
7c0a773f0e892e41fe8f7b299be655e53f110a64bd6e77c2e7a6b4c699605498 guix-build-eb02713efc17/output/x86_64-w64-mingw32/bitcoin-eb02713efc17-win64-debug.zip
27f6a50394c61c0efa2f3afe655a265c64a34249041ef0090f9043cd4cdc8c71 guix-build-eb02713efc17/output/x86_64-w64-mingw32/bitcoin-eb02713efc17-win64-setup-unsigned.exe
dd5bb661a9d99bbc2c2c7256996b26bef116b2e61b7497ada26b3322550d53cb guix-build-eb02713efc17/output/x86_64-w64-mingw32/bitcoin-eb02713efc17-win64-unsigned.tar.gz
84d5068dd59180498473263d757192a422859c0704a2e45a762d1635e49efb80 guix-build-eb02713efc17/output/x86_64-w64-mingw32/bitcoin-eb02713efc17-win64.zip
```
Guix build (on arm64):
```bash
2bbcf455381d4be6bb402c705dba5655de3e1b62a0ed1dbbfcb573450a63d148 guix-build-eb02713efc17/output/arm-linux-gnueabihf/SHA256SUMS.part
c9d982eb6c9efc752fffb3a1d2b14e8cc4d9e2cf4c03c2f02eb320d04f52a86d guix-build-eb02713efc17/output/arm-linux-gnueabihf/bitcoin-eb02713efc17-arm-linux-gnueabihf-debug.tar.gz
0c0b3122dcdc051bee1022dc9b0cf7771b7f6b30fa3e7369c97907d8c10d7ea3 guix-build-eb02713efc17/output/arm-linux-gnueabihf/bitcoin-eb02713efc17-arm-linux-gnueabihf.tar.gz
5388f8a7525ed49d11569988e598a0ec68ddcaf9b35cff0c8bcb02187b0fbaad guix-build-eb02713efc17/output/arm64-apple-darwin/SHA256SUMS.part
16b745e2b2e036d65b549be740116d9b6e819730cc76075f01bdbc4beb166724 guix-build-eb02713efc17/output/arm64-apple-darwin/bitcoin-eb02713efc17-arm64-apple-darwin-unsigned.dmg
18eaede02d12dcbb83003272b5b79a08a10067a326542687ab445bfc623ce9e8 guix-build-eb02713efc17/output/arm64-apple-darwin/bitcoin-eb02713efc17-arm64-apple-darwin-unsigned.tar.gz
5fb73968c7ea50c9642d3cddcd745a512be3043ada314b8a1fc94f179744a1d8 guix-build-eb02713efc17/output/arm64-apple-darwin/bitcoin-eb02713efc17-arm64-apple-darwin.tar.gz
f354822050cd625de7445cfa317475cfad90a7e39d135c5b99950ee69969f445 guix-build-eb02713efc17/output/dist-archive/bitcoin-eb02713efc17.tar.gz
fad672b9e5d372ba5511c14ed48ef77bcf303d475f35680bd4a668fee150225a guix-build-eb02713efc17/output/powerpc64-linux-gnu/SHA256SUMS.part
246aa854e87675a0b90cc14f7b6affcfefabfc0f79edd3dc96ae6b98010b8b1c guix-build-eb02713efc17/output/powerpc64-linux-gnu/bitcoin-eb02713efc17-powerpc64-linux-gnu-debug.tar.gz
90e968e0bab84e80a9f2fe9498eea7c59d8908f5a16accd93d7f9318a7098ce0 guix-build-eb02713efc17/output/powerpc64-linux-gnu/bitcoin-eb02713efc17-powerpc64-linux-gnu.tar.gz
65f0c018d882d7fc845a9bb1581824b17e7ecf0df7081ab2538f0e617e120a8b guix-build-eb02713efc17/output/powerpc64le-linux-gnu/SHA256SUMS.part
0bf7fcd127180e5e04112914747496db535226bf05126690f259fa0cf2a96642 guix-build-eb02713efc17/output/powerpc64le-linux-gnu/bitcoin-eb02713efc17-powerpc64le-linux-gnu-debug.tar.gz
14fc3e17dfa903f83e44f970c8b4e4726e7476c59d0fffdec815a1c80ec1b51a guix-build-eb02713efc17/output/powerpc64le-linux-gnu/bitcoin-eb02713efc17-powerpc64le-linux-gnu.tar.gz
9018f95b54d0643d734260b6eb69ee5f086c98e62f25dd579675b467a844793d guix-build-eb02713efc17/output/riscv64-linux-gnu/SHA256SUMS.part
e1f2b3678c22103d7b89cbbeec9b2863c9c6f749ff4cbedd74cb6e62598c0a04 guix-build-eb02713efc17/output/riscv64-linux-gnu/bitcoin-eb02713efc17-riscv64-linux-gnu-debug.tar.gz
bf9fa35119344dfc93048196dd9cd5bb230b0785350ae5150bb4bdb28fd8423d guix-build-eb02713efc17/output/riscv64-linux-gnu/bitcoin-eb02713efc17-riscv64-linux-gnu.tar.gz
51bbf04cb32b579ba5609fe3ef24e9901f8d49e3311fe9776ee1fdb644f7e0b1 guix-build-eb02713efc17/output/x86_64-apple-darwin/SHA256SUMS.part
8e81f8badb0cff1aa430a899065cf9744b4b2d45addb8e30606a2f8bf08faa26 guix-build-eb02713efc17/output/x86_64-apple-darwin/bitcoin-eb02713efc17-x86_64-apple-darwin-unsigned.dmg
8b4120b6d83c03dae34b0b5a189522d01c523ab005d816339fdfddf9c412ef15 guix-build-eb02713efc17/output/x86_64-apple-darwin/bitcoin-eb02713efc17-x86_64-apple-darwin-unsigned.tar.gz
26e633faba4f05f51f4e0bffaa2bbbf8c2d5d134d6777c9395bf9b65af6a808c guix-build-eb02713efc17/output/x86_64-apple-darwin/bitcoin-eb02713efc17-x86_64-apple-darwin.tar.gz
6e5828e2efa4e951b147b8de42f79dee1652933e04c50093bd31ee375c0c4ca9 guix-build-eb02713efc17/output/x86_64-linux-gnu/SHA256SUMS.part
3d3af1d078eee6f66aac2af891fd7d7a77abc3d7164a807d0a7cc44f15e52b9d guix-build-eb02713efc17/output/x86_64-linux-gnu/bitcoin-eb02713efc17-x86_64-linux-gnu-debug.tar.gz
13f452bc65194de16fa91ed87be9790ed0d1a178deefb102fa54d3f9832b8c25 guix-build-eb02713efc17/output/x86_64-linux-gnu/bitcoin-eb02713efc17-x86_64-linux-gnu.tar.gz
1673b5fca6687ea0f196a5f2ce2b79662b3efe01b71f341fc596069a1ade610c guix-build-eb02713efc17/output/x86_64-w64-mingw32/SHA256SUMS.part
d6228bdfb4fbc7b895ed4f0c30e1343c3392bd6e8e5ed33a973887ba0bb749ba guix-build-eb02713efc17/output/x86_64-w64-mingw32/bitcoin-eb02713efc17-win64-debug.zip
27f6a50394c61c0efa2f3afe655a265c64a34249041ef0090f9043cd4cdc8c71 guix-build-eb02713efc17/output/x86_64-w64-mingw32/bitcoin-eb02713efc17-win64-setup-unsigned.exe
dd5bb661a9d99bbc2c2c7256996b26bef116b2e61b7497ada26b3322550d53cb guix-build-eb02713efc17/output/x86_64-w64-mingw32/bitcoin-eb02713efc17-win64-unsigned.tar.gz
50a68d3644dbe5cb5de21993cd0e8992ab9c6aa88c009a75e5d5a55180476ea6 guix-build-eb02713efc17/output/x86_64-w64-mingw32/bitcoin-eb02713efc17-win64.zip
```
ACKs for top commit:
laanwj:
ACK eb02713efc
dongcarl:
Code Review ACK eb02713efc
vincenzopalazzo:
ACK eb02713efc
Tree-SHA512: afee459d881d3231b72711b0beec5410d6b37deb0d94aa0aaca13010f54bf95fadaf7e5081ea8b9c6975a9e2b8be48b761e3b6ce284c06f82be2210db1156e96
b42643c253 doc: update init.cpp -conf help text (josibake)
970b9987ad doc: update devtools, release-process readmes (josibake)
50635d27b4 build: include bitcoin.conf in build outputs (josibake)
6aac946f49 doc: update bitcoin-conf.md (Josiah Baker)
1c7e820ded script: add script to generate example bitcoin.conf (josibake)
b483084d86 doc: replace bitcoin.conf with placeholder file (josibake)
Pull request description:
create a script for parsing the output from `bitcoind --help` to create an example conf file for new users
## problem
per #10746 , `bitcoin.conf` not being put into the data directory during installation causes some confusion for users when running bitcoin. in the discussion on the issue, one proposed solution was to have an example config file and instruct users to `cp` it into their data directory after startup. in addition to #10746 , there have been other requests for a "skeleton config file" (https://github.com/bitcoin/bitcoin/issues/19641) to help users get started with configuring bitcoind.
the main issue with an example config file is that it creates a second source of truth regarding what options are available for configuring bitcoind. this means any changes to the options (including the addition or removal of options) would have to be updated for the command line and also updated in the example file.
this PR addresses this issue by providing a script to generate an example file directly from the `bitcoind --help` on-demand by running `contrib/devtools/gen-bitcoin-conf.sh`. this solution was originally proposed on #10746 and would also solve #19641 . this guarantees any changes made to the command-line options or the command-line options help would also be reflected in the example file after compiling and running the script.
the main purpose of this script is to generate a config file to be included with releases, same as `gen-manpages.sh`. this ensures every release also includes an up-to-date, full example config file for users to edit. the script is also available for users who compile from source for generating an example config for their compiled binary.
## special considerations
this removes the `bitcoin.conf` example file from the repo as it is now generated by this script. the original example file did contain extra text related to how to use certain options but going forward all option help docs should be moved into `init.cpp`
this also edits `init.cpp` to have the option help indicate that `-conf` is not usable from the config file. this is similar to how `-includeconf` 's help indicates it cannot be used from the command line
ACKs for top commit:
laanwj:
Tested and code review ACK b42643c253
Tree-SHA512: 4546e0cef92aa1398da553294ce4712d02e616dd72dcbe0b921af474e54f24750464ec813661f1283802472d1e8774e634dd1cc26fbf1f13286d3e0406c02c09
copy over bitcoin.conf during the build process.
this means `contrib/devtools/gen-bitcoin-conf.sh` will need
to be run and the generated file committed during the release process.
this is the same process used for generating man pages for each release.
027aab663a test, contrib, refactor: use `with` when opening a file (brunoerg)
Pull request description:
When manipulating a file in Python without using `with()`, you have to close the file manually, so this PR does it in `get_block_hashes` (`contrib/linearize/linearize-data.py`).
Edit: this PR does it for all occurances that previously weren't using `with`.
ACKs for top commit:
laanwj:
Code review ACK 027aab663a
Tree-SHA512: 879400968e0013e8678ec16f1fe5d0963a73c1e0d442ca34802d885214f0783d2e9a9b500fc6be7c3b93560a367b6a3d685eee24d2f9ce53fddf064ea6feecf8
this ensures bitcoind option help is the source of truth and also
gives an example conf file for users to customize and copy to their
data directory.
closes#10746
9b0a13a289 tidy: Add include-what-you-use (fanquake)
74cd038e30 refactor: fix includes in src/init (fanquake)
c79ad935f0 refactor: fix includes in src/compat (fanquake)
Pull request description:
We recently added a [`clang-tidy` job](https://github.com/bitcoin/bitcoin/blob/master/ci/test/00_setup_env_native_tidy.sh) to the CI, which generates a compilation database. We can leverage that now existing database to begin running [include-what-you-use](https://include-what-you-use.org/) over the codebase.
This PR demonstrates using a mapping_file to indicate fixups / includes that may differ from IWYU suggestions. In this case, I've added some fixups for glibc includes that I've [upstreamed changes for](https://github.com/include-what-you-use/include-what-you-use/pull/1026):
```bash
# Fixups / upstreamed changes
[
{ include: [ "<bits/termios-c_lflag.h>", private, "<termios.h>", public ] },
{ include: [ "<bits/termios-struct.h>", private, "<termios.h>", public ] },
{ include: [ "<bits/termios-tcflow.h>", private, "<termios.h>", public ] },
]
```
The include "fixing" commits of this PR:
* Adds missing includes.
* Swaps C headers for their C++ counterparts.
* Removes the pointless / unmaintainable `//for abc, xyz` comments. When using IWYU, if anyone wants to see / generate those comments, to see why something is included, it is trivial to do so (IWYU outputs them by default). i.e:
```cpp
// The full include-list for compat/stdin.cpp:
#include <compat/stdin.h>
#include <poll.h> // for poll, pollfd, POLLIN
#include <termios.h> // for tcgetattr, tcsetattr
#include <unistd.h> // for isatty, STDIN_FILENO
```
TODO:
- [ ] Qt mapping_file. There is one in the IWYU repo, but it's for Qt 5.11. Needs testing.
- [ ] Boost mapping_file. There is one in the IWYU repo, but it's for Boost 1.75. Needs testing.
I'm not suggesting we turn this on the for entire codebase, or immediately go-nuts refactoring all includes. However I think our dependency includes are now slim enough, and our CI infrastructure in place such that we can start doing this in some capacity, and just automate away include fixups / refactorings etc.
ACKs for top commit:
MarcoFalke:
review ACK 9b0a13a289
jonatack:
ACK 9b0a13a289 reviewed changes and run CI output in https://cirrus-ci.com/task/4750910332076032
Tree-SHA512: 00beab5a5f2a6fc179abf08321a15391ecccaa91ab56f3c50c511e7b29a0d7c95d8bb43eac2c31489711086f6f77319d43d803cf8ea458e7cd234a780d9ae69e
914076ed5f guix: Improve error message about missed macOS SDK (Hennadii Stepanov)
Pull request description:
The error message now mentions another option for users to specify the path to the macOS SDK.
ACKs for top commit:
fanquake:
ACK 914076ed5f
Tree-SHA512: e8b9a6c8aa984f02cdb4ad1eeb6c22510c0c74c47c831104ac410ef68b3bc40024a52400b5065a3e25603f8a040e72c8c464436cb9a0a018ca73b0d28e0d609d
This change makes naming of `install_name_tool` consistent across
the whole build system.
-BEGIN VERIFY SCRIPT-
sed --in-place --expression='s/INSTALLNAMETOOL/INSTALL_NAME_TOOL/g' $(git grep --files-with-matches 'INSTALLNAMETOOL')
-END VERIFY SCRIPT-
1dd8cbfbc6 build: don't compress macOS DMG (fanquake)
Pull request description:
Skip compressing the macOS DMG, and drop related build steps and dependencies. Uncompressed the DMG increases from ~16mb to ~30mb, which compared to other software a user may download, (Firefox 125mb, VLC 52mb, Open Office 176mb), is still relatively small. When contrasted against the 100's of GB of blockchain data a node will download, an additional 15mb to get the release binary, isn't much additional overhead. Note that if / when we build with LTO enabled for releases, this size will shrink back down significantly again.
`native_libdmg-hfsplus` is not maintained, and I doubt the DMG creation feature will ever be fixed. If at some point `xorrisofs` supports compressing dmgs, we could enable that.
Guix Build on x86_64:
```bash
25b7c8bb7bc8ea014d43cebb844a842d2ac8d5a343039a820d24b649c9e6bc8a guix-build-1dd8cbfbc631/output/arm64-apple-darwin/SHA256SUMS.part
16beb5c52c9bf51b5ce9ef5a0d17c0038238a833383586a1b14acbca78533e4b guix-build-1dd8cbfbc631/output/arm64-apple-darwin/bitcoin-1dd8cbfbc631-arm64-apple-darwin-unsigned.dmg
d8f89a61a7448d6334dbb3639386a7b6340542393933f35421a9e6dfc724e455 guix-build-1dd8cbfbc631/output/arm64-apple-darwin/bitcoin-1dd8cbfbc631-arm64-apple-darwin-unsigned.tar.gz
11617dc261ef602433f5bb29956a40a9085dbc783f519f75fbe06e80970148d0 guix-build-1dd8cbfbc631/output/arm64-apple-darwin/bitcoin-1dd8cbfbc631-arm64-apple-darwin.tar.gz
aa8550d4a394d3161d14ec5e6012ed07354135afb022e905a1946785b4665664 guix-build-1dd8cbfbc631/output/dist-archive/bitcoin-1dd8cbfbc631.tar.gz
2b837f2f971a9738d0b7b8497f7ded740ef5e67c8baa7f30ca33e6b7d826eec8 guix-build-1dd8cbfbc631/output/x86_64-apple-darwin/SHA256SUMS.part
db972b2c06dbde5525a3f9e6ceb9c20a8120bc9a6f15e1d852a4bfac09d88569 guix-build-1dd8cbfbc631/output/x86_64-apple-darwin/bitcoin-1dd8cbfbc631-x86_64-apple-darwin-unsigned.dmg
50fe990c3f9923ee92195125faf6517396e7c1b017a8f4f7d52e991ebce52f0c guix-build-1dd8cbfbc631/output/x86_64-apple-darwin/bitcoin-1dd8cbfbc631-x86_64-apple-darwin-unsigned.tar.gz
1d9022b0ae46ead41046c40f82291ce363760660a3cd6e6ef6a5b1128b90faef guix-build-1dd8cbfbc631/output/x86_64-apple-darwin/bitcoin-1dd8cbfbc631-x86_64-apple-darwin.tar.gz
```
Guix Build on arm64:
```bash
```
ACKs for top commit:
Sjors:
re-tACK 1dd8cbfbc6 on Intel macOS
laanwj:
Build system changes code review ACK 1dd8cbfbc6, I don't know anything about MacOS application formats and their internals so do not have an opinion on the contents of this change.
jarolrod:
ACK 1dd8cbfbc6
Tree-SHA512: 04c5bf78f26a9877777093ec4c50c457107bef59d720839ea5e7d7e4f7961dfee9f86b40cf791524a9e60e9e77403a797e9fcdae3849b60b759f9f66cc31b6ab
ba30a5407e contrib: macdeploy: monkey-patch gen-sdk to be deterministic (Pavol Rusnak)
1868a17e5a contrib: macdeploy: make gen-sdk deterministic (Pavol Rusnak)
Pull request description:
This PR attempts to make `contrib/macdeploy/gen-sdk` deterministic
Can anyone with the `Xcode_12.2.xip` confirm that `gen-sdk` produces the same hash? => `e7ca56bc8804d16624fad68be2e71647747d6629cacaaa3de5fbfa7f444e9eae `
ACKs for top commit:
laanwj:
Tested ACK ba30a5407e
jarolrod:
Tested ACK ba30a5407e
Tree-SHA512: 1638ceaf28e87ef0d21a1a71ef02989f75942b60a12f07236ac709bde96f08f39f816767e35a0fe68c26bf5978e63e74f5385be9d4b8f80a2e89b30f163f4526
107582039a doc: Add gpg key import instructions for Windows (Dave Scotese)
Pull request description:
This is a single commit to replace the three commits from #23916
I propose this change so that Windows users can more easily import signers' keys.
ACKs for top commit:
sipsorcery:
tACK 107582039a.
Tree-SHA512: 7d4ec77ce10f751748c49f1453fa8baf0976b15af4f87dc27f4e2715ad73fbd7dc1f07fcf3e660d63a6b9eb895a5e4105774613d39a2328f73b92d9e6cff4ebd
I have some qualms with maintaining a suspicious hosts list as part as
the repository. But also, it's stale and irrelevant. I've checked the
entire list and none of them is connectable. Only one still appars in
`nodes_main.txt` but with low uptime and an old subversion string so it
wouldn't be picked in the first place.
c457fb144c improve clarity and up max ipv6 ASNs (Baas)
Pull request description:
This PR attempts to address some of the areas of improvement raised in #17020 . Concretely, my proposed change is fairly minor but addresses the following changes to [`makeseeds.py`](https://github.com/bitcoin/bitcoin/blob/master/contrib/seeds/makeseeds.py):
- Increase max seeds per ASN for IPv6 to 10 as recommended [here](https://github.com/bitcoin/bitcoin/pull/16999#issuecomment-536999544), while keeping max seeds per ASN for IPv4 at 2.
- Bump `MIN_BLOCKS` to 730000.
- Improved script clarity: added function types and more docs to functions, added progress indicator when performing ASN lookup, and change string formatting to better align with [bitcoin python style guidelines](https://github.com/bitcoin/bitcoin/blob/master/test/functional/README.md#style-guidelines)
With the different ASN limits for IPv4 and IPv6, and the new minimum block requirement, the current stats look look like:
```
IPv4 IPv6 Onion Pass
470689 73238 0 Initial
470689 73238 0 Skip entries with invalid address
470689 73238 0 After removing duplicates
470688 73238 0 Skip entries from suspicious hosts
6098 1676 0 Enforce minimal number of blocks
5252 1443 0 Require service bit 1
3812 898 0 Require minimum uptime
3738 877 0 Require a known and recent user agent
3715 869 0 Filter out hosts with multiple bitcoin ports
512 512 0 Look up ASNs and limit results per ASN and per net
```
The new ASN max seeds of 10 allows for 512 IPv6 addresses to be included, up from the ~150 that was filtered by the previous version.
While there is more to do for #17020 , these changes I think are fairly isolated from the rest and should make it a bit easier for others to get up to speed with what the functions in the script do.
ACKs for top commit:
laanwj:
Concept and code review ACK c457fb144c
Tree-SHA512: 3ed67868443cc50544e23b27e2341758c3a8866997b0dba47b137032d5e1a13428855daaeed682626ed471542b44435635178d54848a2cd6fe73777679428032
65c49ac750 test: throw `ValueError` for invalid base58 checksum (Sebastian Falbesoner)
219d2c7ee1 contrib: testgen: use base58 methods from test framework (Sebastian Falbesoner)
605fecfb66 scripted-diff: rename `chars` to `b58chars` in test_framework.address (Sebastian Falbesoner)
11c63e374d contrib: testgen: import OP_* constants from test framework (Sebastian Falbesoner)
7d755bb31c contrib: testgen: avoid need for manually setting PYTHONPATH (Sebastian Falbesoner)
Pull request description:
This PR removes the redundant base58 implementation [contrib/testgen/base58.py](https://github.com/bitcoin/bitcoin/blob/master/contrib/testgen/base58.py) for the test generation script `gen_key_io_test_vectors.py` and uses the one from the test framework instead. Additionally, three other cleanups/improvements are done:
- import script operator constants `OP_*` from test framework instead of manually defining them
- add Python path to test framework directly in the script (via `sys.path.append(...)`) instead of needing the caller to specify `PYTHONPATH=...` on the command line (the same approach is done for the signet miner and the message capture scripts)
- rename `chars` to `b58chars` in the test_framework.address module (is more explicit and makes the diff for the base58 replacement smaller)
ACKs for top commit:
laanwj:
Code review ACK 65c49ac750
Tree-SHA512: 92e1534cc320cd56262bf455de7231c6ec821bfcd0ed58aa5718271ecec1a89df7951bf31527a2306db6398e7f2664d2ff8508200c28163c0b164d3f5aaf8b0e
af74e061c0 guix: make it possible to override gpg binary (Pavol Rusnak)
Pull request description:
For example on Qubes OS one might want to use qubes-gpg-client-wrapper instead
Fixes https://github.com/bitcoin/bitcoin/issues/24346
ACKs for top commit:
laanwj:
Concept and code review ACK af74e061c0
Tree-SHA512: 9e56b5fab231f8908fff15c88fe5b356ac4a31a14a27ae2dd3b6e876f32628910a666a4e2da5bf7c5d159de66cf57652c94c81cdc3b1c3d39a23c23e2c77dd03
15893a0781 supp: remove Boost Valgrind suppression (fanquake)
b0740fdcb8 ci: use DWARF-4 for Valgrind CI job (fanquake)
Pull request description:
[clang-14 defaults to using DWARF-5](https://releases.llvm.org/14.0.0/tools/clang/docs/ReleaseNotes.html#dwarf-support-in-clang), which breaks vlagrinds (3.18) ability
to parse debug info. Valgrind [claims to support DWARF-5](https://valgrind.org/docs/manual/dist.news.html) from version
3.18 onwards, but maybe that only works when compiling with GCC.
Explicitly use DWARF-4 for now. Note that from 11.0 [GCC also defaults to
using DWARF-5](https://www.gnu.org/software/gcc/gcc-11/changes.html).
Also remove a Boost related suppression.
Top commit has no ACKs.
Tree-SHA512: e4f476170ac5ccbb43d26e990b24753bda3985b2ac5c8a32e74d2d1d64d1b3a2d80a90fbab345f0a9e404eac7fbd783c20147379208e615d526657e8a57890ca
254a63e097 contrib: refactor: replace `hex_switchEndian` in linearize scripts (Sebastian Falbesoner)
3f863cfff1 contrib: refactor: simplify block header string routine in linearize-data.py (Sebastian Falbesoner)
Pull request description:
This PR simplifies the linearization scripts `linearize-data.py` and `linearize-hashes.py` by replacing overly complicated cruft (block header hash string calculation, hex string reverse) with means of the Python3 standard library.
ACKs for top commit:
laanwj:
Code review ACK 254a63e097
Tree-SHA512: 4a0e20e63bd11f23a190480b22dbc2f2a3070e2a4f3a01b8797f99bb5fc830185e91e6712c8ae97562f9a24a98aa4f19d52f02a3f5fcbe4c578ee88a41382d1d
97af652788 guix: Drop code for the unsupported `i686-linux-gnu` host (Hennadii Stepanov)
Pull request description:
Now GUIX build for the `i686-linux-gnu` host is broken, and [there are no plans to re-add it](https://github.com/bitcoin/bitcoin/pull/24448).
ACKs for top commit:
fanquake:
ACK 97af652788
Tree-SHA512: 968181aff65e607a7c1a1b06ac7dfd79f6e2ce49b3c4c3828def020e925769fdbab1859d37ea924ded7632405b30539ac3ec81ac714cb9a01a2f7d5c93301dd9
12cc0201c2 contrib: fix signet miner (sighash mismatch) (Sebastian Falbesoner)
Pull request description:
gruve-p reported that the signet miner doesn't work anymore (see https://github.com/bitcoin/bitcoin/issues/24501#issuecomment-1062088351), failing with the following error of the `walletprocesspsbt` RPC:
```
error code: -22
error message:
Specified sighash value does not match value stored in PSBT
.....
subprocess.CalledProcessError: Command '['bitcoin-cli', '-signet', '-stdin', 'walletprocesspsbt']' returned non-zero exit status 22
```
PSBT signing was changed to use SIGHASH_DEFAULT by default in #22514. The signet miner script sets the sighash type of the created PSBT to SIGHASH_ALL (3 is the per-input type PSBT_IN_SIGHASH_TYPE, following a little-endian 32 unsigned integer of the sighash type):
e04720ec33/contrib/signet/miner (L169-L170)
hence this leads to a sighash mismatch when the `walletprocesspsbt` RPC is called. Fix this by explicitly passing the correct sighash type. The same change was needed in one of our functional tests, see commit d3992669df.
Note that instead of feeding the PSBT via `-stdin` it is directly passed as parameter, as I couldn't figure out a way to pass multiple parameters otherwise (separating by newline also didn't work).
ACKs for top commit:
kallewoof:
ACK 12cc0201c2
ajtowns:
ACK 12cc0201c2 ; code review only
Tree-SHA512: 8509e768e96f85e28c0ca0dc2d35874aa29623febddc46bf90472ec38f38cb3a1b5407c563fd9101d07088775d0fdb18e9137cc38955e847885b83c16591c736
PSBT signing was changed to use SIGHASH_DEFAULT by default in #22514.
The signet miner script sets the sighash type of the created PSBT to
SIGHASH_ALL, hence this leads to a sighash mismatch when the
`walletprocesspsbt` RPC is called. Fix this by explicitly passing the
correct sighash type.
Note that the same change was needed in one of our functional tests,
see commit d3992669df.
Reported by gruve-p.
The macOS and Windows builds do not require a GCC 7 toolchain, and this
is actually causing build issues, i.e #24211. So switch to using a GCC
10 native toolchain for both.
```bash
test3.c: In function 'main':
test3.c:6:21: warning: implicit declaration of function 'CoFreeUnusedLibrariesEx' [-Wimplicit-function-declaration]
6 | CoFreeUnusedLibrariesEx(0,0);
```
