Commit Graph

1586 Commits

Author SHA1 Message Date
Carl Dong
fcab35b229 guix-attest: Produce and sign normalized documents
That way we can easily combine the document and detached signature to
produce cleartext signature files for upload during the release process.

See subsequent commits which modify doc/release-process.md for more
details.
2021-07-19 18:24:36 -04:00
Carl Dong
c2541fd0ca guix: Overhaul README
- Added detailed Guix bootstrap/installation instructions
2021-07-19 18:24:36 -04:00
Carl Dong
46ce6ce378 tree-wide: Rename gitian-keys to builder-keys 2021-07-08 11:29:54 -04:00
Carl Dong
fc4f8449f3 guix: Update various check_tools lists 2021-07-08 11:29:26 -04:00
Carl Dong
263220a85c guix: Check for a sane services database
On bare systems, it is possible to be lacking a services database. Check
for basic entries before attempting a build.

See the error message in the diff for more context.
2021-07-08 11:28:07 -04:00
fanquake
797b3ed909
script: remove gitian reference from symbol-check.py 2021-07-06 20:50:47 +08:00
fanquake
15fc9a0299
guix: add additional documentation to patches 2021-07-06 20:50:47 +08:00
fanquake
de6ca41a52
guix: no-longer pass --enable-glibc-back-compat to Guix
Now that our Guix builds are performed on glibc 2.24 and 2.27 (RISCV),
we no-longer need to pass the --enable-glibc-back-compat option.

Replace it with --disable-threadlocal, to prevent the usage of symbols
from glibc 2.18.

None of the binaries produced required symbols later than 2.17, and 2.27
(RISCV).
2021-07-06 20:50:40 +08:00
Carl Dong
647f7e5f1d guix: Also sort SHA256SUMS.part 2021-07-02 14:10:33 -04:00
Carl Dong
dc4137a60c guix: Build depends/qt with our platform definition
Our 'bitcoin-linux-g++' definition better integrates with our depends
system than the stock linux-g++-64 definition.

This fixes a bug whereby Guix builds on x86_64 for x86_64 did not
produce a QMinimalIntegrationPlugin and led to bitcoin-qt not being
built.
2021-07-01 16:20:28 -04:00
Carl Dong
16b0a936e1 guix: Rebase toolchain on glibc 2.24 (2.27 for riscv64)
Support for riscv64 in glibc landed in 2.27 so it's unavoidable that we
use 2.27.

Running a Bitcoin build with toolchains based on 2.24 for platforms
other than riscv64 seem to produce binaries which do not have 2.17
symbols. So use 2.24 since it's more recent and maintained by Debian
Stretch.
2021-07-01 16:17:03 -04:00
MarcoFalke
d6a59166a1
Merge bitcoin/bitcoin#22257: test: refactor: various (de)serialization helpers cleanups/improvements
bdb8b9a347 test: doc: improve doc for `from_hex` helper (mention `to_hex` alternative) (Sebastian Falbesoner)
1914054208 scripted-diff: test: rename `FromHex` to `from_hex` (Sebastian Falbesoner)
a79396fe5f test: remove `ToHex` helper, use .serialize().hex() instead (Sebastian Falbesoner)
2ce7b47958 test: introduce `tx_from_hex` helper for tx deserialization (Sebastian Falbesoner)

Pull request description:

  There are still many functional tests that perform conversions from a hex-string to a message object (deserialization) manually. This PR identifies all those instances and replaces them with a newly introduced helper `tx_from_hex`.

  Instances were found via
  * `git grep "deserialize.*BytesIO"`

  and some of them manually, when it were not one-liners.

  Further, the helper `ToHex` was removed and simply replaced by `.serialize().hex()`, since now both variants are in use (sometimes even within the same test) and using the helper doesn't really have an advantage in readability. (see discussion https://github.com/bitcoin/bitcoin/pull/22257#discussion_r652404782)

ACKs for top commit:
  MarcoFalke:
    review re-ACK bdb8b9a347 😁

Tree-SHA512: e25d7dc85918de1d6755a5cea65471b07a743204c20ad1c2f71ff07ef48cc1b9ad3fe5f515c1efaba2b2e3d89384e7980380c5d81895f9826e2046808cd3266e
2021-06-24 12:47:13 +02:00
Sebastian Falbesoner
1914054208 scripted-diff: test: rename FromHex to from_hex
-BEGIN VERIFY SCRIPT-
sed -i 's/\<FromHex\>/from_hex/g' $(git grep -l FromHex)
-END VERIFY SCRIPT-

Co-authored-by: MarcoFalke <falke.marco@gmail.com>
2021-06-21 14:33:59 +02:00
Sebastian Falbesoner
a79396fe5f test: remove ToHex helper, use .serialize().hex() instead 2021-06-21 14:30:03 +02:00
Sebastian Falbesoner
2ce7b47958 test: introduce tx_from_hex helper for tx deserialization
`FromHex` is mostly used for transactions, so we introduce a
shortcut `tx_from_hex` for `FromHex(CTransaction, hex_str)`.
2021-06-21 14:28:05 +02:00
W. J. van der Laan
a305a687e7
Merge bitcoin/bitcoin#22244: devtools: Correctly extract symbol versions in symbol-check
e8cd3700ee devtools: Integrate ARCH_MIN_GLIBC_VER table into MAX_VERSIONS in symbol-check.py (W. J. van der Laan)
a33381acf5 devtools: Add xkb version to symbol-check (W. J. van der Laan)
19e598bab0 devtools: Fix verneed section parsing in pixie (W. J. van der Laan)

Pull request description:

  I misunderstood the ELF specification for version symbols (verneed): The `vn_aux` pointer is relative to the main verneed record, not the start of the section.

  This caused many symbols to not be versioned properly in the return value of `elf.dyn_symbols`. This was discovered in #21454.

  Fix it by correcting the offset computation.

  - xkb versions symbols (using the prefix `V`), as this library is used by bitcoin-qt, add it to the valid versions in `symbol-check.py`

  This unfortunately brings to light some symbols that have been introduced since and weren't caught (from a gitian compile of master):

  ```
  bitcoin-cli: symbol getrandom from unsupported version GLIBC_2.25
  bitcoin-cli: failed IMPORTED_SYMBOLS
  bitcoind: symbol getrandom from unsupported version GLIBC_2.25
  bitcoind: symbol log from unsupported version GLIBC_2.29
  bitcoind: symbol fcntl64 from unsupported version GLIBC_2.28
  bitcoind: symbol pow from unsupported version GLIBC_2.29
  bitcoind: symbol exp from unsupported version GLIBC_2.29
  bitcoind: failed IMPORTED_SYMBOLS
  bitcoin-qt: symbol exp from unsupported version GLIBC_2.29
  bitcoin-qt: symbol fcntl64 from unsupported version GLIBC_2.28
  bitcoin-qt: symbol log from unsupported version GLIBC_2.29
  bitcoin-qt: symbol pow from unsupported version GLIBC_2.29
  bitcoin-qt: symbol statx from unsupported version GLIBC_2.28
  bitcoin-qt: symbol getrandom from unsupported version GLIBC_2.25
  bitcoin-qt: symbol renameat2 from unsupported version GLIBC_2.28
  bitcoin-qt: symbol getentropy from unsupported version GLIBC_2.25
  bitcoin-qt: failed IMPORTED_SYMBOLS
  bitcoin-wallet: symbol exp from unsupported version GLIBC_2.29
  bitcoin-wallet: symbol log from unsupported version GLIBC_2.29
  bitcoin-wallet: symbol fcntl64 from unsupported version GLIBC_2.28
  bitcoin-wallet: failed IMPORTED_SYMBOLS
  test_bitcoin: symbol getrandom from unsupported version GLIBC_2.25
  test_bitcoin: symbol log from unsupported version GLIBC_2.29
  test_bitcoin: symbol fcntl64 from unsupported version GLIBC_2.28
  test_bitcoin: symbol pow from unsupported version GLIBC_2.29
  test_bitcoin: symbol exp from unsupported version GLIBC_2.29
  test_bitcoin: failed IMPORTED_SYMBOLS
  ```

ACKs for top commit:
  hebasto:
    ACK e8cd3700ee

Tree-SHA512: 8c15e3478eb642f01a1ddaadef03f80583f088f9fa8e3bf171ce16b0ec05ffb4675ec147d7ffc6a4360637ed47fca517c6ca2bac7bb30d794c03783cfb964b79
2021-06-21 07:58:12 +02:00
W. J. van der Laan
0f47e01d7d
Merge bitcoin/bitcoin#20923: signet miner followups
b3c712cb28 contrib/signet/miner: remove debug code (Anthony Towns)
297e35159f bitcoin-util: use AddCommand / GetCommand (Anthony Towns)
b6d493fd4d contrib/signet/README.md: Update miner description (Anthony Towns)
e66543827c contrib/signet/miner: Automatic timestamp for first block (Anthony Towns)
a383ce5b4a contrib/signet/miner: --grind-cmd is required for calibrate (Anthony Towns)
1a45cd2e51 contrib/signet: Fix typos (Anthony Towns)

Pull request description:

  Followups from #19937

ACKs for top commit:
  laanwj:
    Code review ACK b3c712cb28

Tree-SHA512: a1003f9ee3697438114b60872b50f4300c8b52f0d58551566eb61c421d787525807ae75be205dcab2c24358cd568f53260120880109a9d728773405ff987596f
2021-06-18 19:31:38 +02:00
W. J. van der Laan
e8cd3700ee devtools: Integrate ARCH_MIN_GLIBC_VER table into MAX_VERSIONS in symbol-check.py
The (ancient) versions specified here were deceptive. Entries older than
MAX_VERSIONS['GLIBC'], which is 2.17, are ignored here. So reorganize
the code to avoid confusion for other people reading this code.
2021-06-18 13:25:57 +02:00
fanquake
da69d9965a
Merge bitcoin/bitcoin#21871: scripts: add checks for minimum required OS versions
aa80b5759d scripts: check macOS SDK version is set (fanquake)
c972345bac scripts: check minimum required Windows version is set (fanquake)
29615aef52 scripts: check minimum required macOS vesion is set (fanquake)
8732f7b6c9 scripts: LIEF 0.11.5 (fanquake)

Pull request description:

  macOS:
  We use a compile flag ([-mmacosx-version-min=10.14](https://github.com/bitcoin/bitcoin/blob/master/depends/hosts/darwin.mk#L96)) to set the minimum required version of macOS needed to run our binaries. This adds a sanity check that the version is being set as expected.

  Clangs Darwin driver should infer the SDK version used during compilation, and forward that through to the linker. Add a check that this has been done, and the expected SDK version is set. Should help prevent issues like #21771 in future.

  Windows:
  We use linker flags ([-Wl,--major/minor-subsystem-version](https://github.com/bitcoin/bitcoin/blob/master/configure.ac#L683)) to set the minimum required version of Windows needed to run our binaries. This adds a sanity check that the version is being set as expected.

  Gitian builds:
  ```bash
  # macOS:
  8b6fcd61d75001c37b2af3fceb5ae09f5d2fe85e97d361f684214bd91c27954a  bitcoin-f015e1c2cac9-osx-unsigned.dmg
  3c1e412bc7f5a7a5d0f78e2cd84b7096831414e1304c1307211aa3e135d89bbf  bitcoin-f015e1c2cac9-osx-unsigned.tar.gz
  50b7b2804e8481f63c69c78e3e8a71c0d811bf2db8895dd6d3edae9c46a738ae  bitcoin-f015e1c2cac9-osx64.tar.gz
  fe6b5c0a550096b76b6727efee30e85b60163a41c83f21868c849fdd9876b675  src/bitcoin-f015e1c2cac9.tar.gz
  8a20f21b20673dfc8c23e22b20ae0839bcaf65bf0e02f62381cdf5e7922936f0  bitcoin-core-osx-22-res.yml

  # Windows:
  b01fcdc2a5673387050d6c6c4f96f1d350976a121155fde3f76c2af309111f9d  bitcoin-f015e1c2cac9-win-unsigned.tar.gz
  b95bdcbef638804030671d2332d58011f8c4ed4c1db87d6ffd211515c32c9d02  bitcoin-f015e1c2cac9-win64-debug.zip
  350bf180252d24a3d40f05e22398fec7bb00e06d812204eb5a421100a8e10638  bitcoin-f015e1c2cac9-win64-setup-unsigned.exe
  2730ddabe246d99913c9a779e97edcadb2d55309933d46f1dffd0d23ecf9aae5  bitcoin-f015e1c2cac9-win64.zip
  fe6b5c0a550096b76b6727efee30e85b60163a41c83f21868c849fdd9876b675  src/bitcoin-f015e1c2cac9.tar.gz
  aa60d7a753e8cb2d4323cfbbf4d964ad3645e74c918cccd66862888f8646d80f  bitcoin-core-win-22-res.yml
  ```

ACKs for top commit:
  hebasto:
    ACK aa80b5759d, tested by breaking tests:

Tree-SHA512: 10150219910e8131715fbfe20edaa15778387616ef3bfe1a5152c7acd3958fe8f88c74961c3d3641074eb72824680c22764bb1dc01a19e92e946c2d4962a8d2c
2021-06-18 15:21:47 +08:00
fanquake
d50302625e
Merge bitcoin/bitcoin#22182: guix: Overhaul how guix-{attest,verify} works and hierarchy
e2c40a4ed5 guix-attest: Error out if SHA256SUMS is unexpected (Carl Dong)
4cc35daed5 Rewrite guix-{attest,verify} for new hier (Carl Dong)
28a9c9b839 Make SHA256SUMS fragment right after build (Carl Dong)

Pull request description:

  Based on:  #22075
  Code reviewers: I recommend reading the new `guix-{attest,verify}` files instead of trying to read the diff

  The following changes resolve many usability improvements which were pointed out to me:
  1. Some maintainers like to extract their "uncodesigned tarball" inside the `output/` directory, resulting in the older `guix-attest` mistakenly attesting to the extracted contents
  2. Maintainers whose GPG keys reside on an external smartcard often need to physically interact with the smartcard as a way to approve the signing operation, having one signature per platform means a lot of fidgeting
  3. Maintainers wishing to sign on a separate machine now has the option of transferring only a subtree of `output/`, namely `output/*/SHA256SUMS.part`, in order to perform a signature (you may need to specify an `$OUTDIR_BASE` env var)
  4. An `all.SHA256SUMS` file should be usable as the base `SHA256SUMS` in bitcoin core torrents and on the release server.

  For those who sign on an separate machine than the one you do builds on, the following steps will work:
  1. `env GUIX_SIGS_REPO=/home/achow101/guix.sigs SIGNER=achow101 NO_SIGN=1 ./contrib/guix/guix-attest`
  2. Copy `/home/achow101/guix.sigs/<tag>/achow101` (which does not yet have signatures) to signing machine
  3. Sign the `SHA256SUMS` files:
      ```bash
      for i in "<path-to-achow101>/*.SHA256SUMS"; do
          gpg --detach-sign --local-user "<your-key-here>" --armor --output "$i"{.asc,}
      done
      ```
  5. Upload `<path-to-achow101>` (now with signatures) to `guix.sigs`

  -----

  After this change, output directories will now include a `SHA256SUMS.part` fragment, created immediately after a successful build:
  ```
  output
  └── x86_64-w64-mingw32
      ├── bitcoin-4e069f7589da-win64-debug.zip
      ├── bitcoin-4e069f7589da-win64-setup-unsigned.exe
      ├── bitcoin-4e069f7589da-win64.zip
      ├── bitcoin-4e069f7589da-win-unsigned.tar.gz
      └── SHA256SUMS.part
  ```

  These `SHA256SUMS.part` fragments look something like:
  ```
  3ebd7262b1a0a5bb757fef1f70e7e14033c70f98c059bc4dbfee5d1992b25825  dist-archive/bitcoin-4e069f7589da.tar.gz
  def2e7d3de5ab3e3f955344e75151df4f33713f9101f5295bd13c9375bdf633b  x86_64-w64-mingw32/bitcoin-4e069f7589da-win64-debug.zip
  643049fe3ee4a4e83a1739607e67b11b7c9b1a66208a6f35a9ff634ba795500e  x86_64-w64-mingw32/bitcoin-4e069f7589da-win64-setup-unsigned.exe
  a247a1ccec0ccc2e138c648284bd01f6a761f2d8d6d07d91b5b4a6670ec3f288  x86_64-w64-mingw32/bitcoin-4e069f7589da-win-unsigned.tar.gz
  fab76a836dcc592e39c04fd2396696633fb6eb56e39ecbf6c909bd173ed4280c  x86_64-w64-mingw32/bitcoin-4e069f7589da-win64.zip
  ```

  Meaning that they are valid `SHA256SUMS` files when `sha256sum --check`'d at the `guix-build-*/output` directory level

  When `guix-attest` is invoked, these `SHA256SUMS.part` files are combined and sorted (by `-k2`, `LC_ALL=C`) to create:

  1. `noncodesigned.SHA256SUMS` for a manifest of all non-codesigned outputs, and
  3. `all.SHA256SUMS` for a manifest of all outputs including non-codesigned outputs

  Then both files are signed, resulting in the following `guix.sigs` hierarchy:
  ```
  4e069f7589da/
  └── dongcarl
      ├── all.SHA256SUMS
      ├── all.SHA256SUMS.asc
      ├── noncodesigned.SHA256SUMS
      └── noncodesigned.SHA256SUMS.asc
  ```

ACKs for top commit:
  achow101:
    ACK e2c40a4ed5
  hebasto:
    ACK e2c40a4ed5, tested on Linux Mint 20.1 (x86_64) with and w/o `NO_SIGN=1`. Changes in `contrib/guix/libexec/codesign.sh` and `contrib/guix/guix-verify` are reviewed only.

Tree-SHA512: 618aacefb0eb6595735a9ab6a98ea6598fce65f9ccf33fa1e7ef93bf140c0f6cfc16e34870c6aa3e4777dd3f004b92a82a994141879870141742df948ec59c1f
2021-06-17 13:10:37 +08:00
Carl Dong
e2c40a4ed5 guix-attest: Error out if SHA256SUMS is unexpected 2021-06-14 17:05:26 -04:00
W. J. van der Laan
a33381acf5 devtools: Add xkb version to symbol-check
xkb versions symbols (using the prefix `V`), as this library is used by
bitcoin-qt, add it to the valid versions in `symbol-check.py`.
2021-06-14 20:32:09 +02:00
W. J. van der Laan
19e598bab0 devtools: Fix verneed section parsing in pixie
I misunderstood the ELF specification for version symbols (verneed):
The `vn_aux` pointer is relative to the main verneed record, not the
start of the section.

This caused many symbols to not be versioned properly in the return
value of `elf.dyn_symbols`. This was discovered in #21454.

Fix it by correcting the offset computation.
2021-06-14 20:31:58 +02:00
fanquake
ce8276b531
Merge bitcoin/bitcoin#22190: Use latest signapple commit
683d197970 Use latest signapple commit (Andrew Chow)

Pull request description:

  Update gitian and guix to use the same latest signapple commit.

  Also changed guix to use the actual repo. The changes from the fork were incorporated upstream.

ACKs for top commit:
  fanquake:
    ACK 683d197970 - sanity checked that the updated package is built:

Tree-SHA512: a4981f8bbe33e6c5654632bc9b9f6f2f1e675741a19ac7296205e370f1e64a747101ecb632e0cc82a0134e4c2e9ce47b3f7b4d8c8f75f0f06dd069c078303759
2021-06-11 09:09:26 +08:00
fanquake
aa80b5759d
scripts: check macOS SDK version is set
Clangs Darwin driver should infer the SDK version used during compilation, and
forward that through to the linker. Add a check that this has been done, and the
expected SDK version is set.

Should help prevent issues like #21771 in future.
2021-06-10 15:43:50 +08:00
fanquake
c972345bac
scripts: check minimum required Windows version is set
We use linker flags (-Wl,--major/minor-subsystem-version) to set the
minimum required version of Windows needed to run our binaries. This
adds a sanity check that the version is being set as expected.
2021-06-10 10:40:53 +08:00
fanquake
29615aef52
scripts: check minimum required macOS vesion is set
We use a compile flag (-mmacosx-version-min) to set the minimum required
version of macOS needed to run our binaries. This adds a sanity check
that the version is being set as expected.
2021-06-10 10:40:53 +08:00
fanquake
8732f7b6c9
scripts: LIEF 0.11.5 2021-06-10 10:40:52 +08:00
Carl Dong
4cc35daed5 Rewrite guix-{attest,verify} for new hier 2021-06-09 11:06:00 -04:00
Carl Dong
28a9c9b839 Make SHA256SUMS fragment right after build 2021-06-09 11:06:00 -04:00
fanquake
7cac26246a
Merge bitcoin/bitcoin#22075: guix: Misc leftover usability improvements
108a6be92a guix: Check for disk space availability before building (Carl Dong)
d7dec89091 guix: Remove dest if OUTDIR mv fails (Carl Dong)

Pull request description:

  There seems to be some corner cases that can be hit when guix scripts unexpectedly fail in the middle of operation, see: https://gnusha.org/bitcoin-builds/2021-05-24.log

  - Perform an early disk space check for `guix-build`
  - Overwrite existing output directory after a successful build (the existing one might be malformed), and cleanup output directory if the `mv` somehow fails

ACKs for top commit:
  laanwj:
    Tested ACK 108a6be92a
  achow101:
    ACK 108a6be92a

Tree-SHA512: cf6438317da40bf55714cd2d8cce859b3d435cc66cabefe8d4a53552d7880966acfe84ffe8fadf1c80e368ae6b037992258a6d409df85ffc6ce8bf780e98e2e5
2021-06-09 09:09:38 +08:00
Andrew Chow
683d197970 Use latest signapple commit
Update gitian and guix to use the same latest signapple commit
2021-06-08 16:46:56 -04:00
W. J. van der Laan
07ededa30c
Merge bitcoin/bitcoin#22050: p2p: remove tor v2 support
5d82a57db4 contrib: remove torv2 seed nodes (Jon Atack)
5f7e086dac contrib: update generate-seeds.py to ignore torv2 addresses (Jon Atack)
8be56f0f8e p2p, refactor: extract OnionToString() from CNetAddr::ToStringIp() (Jon Atack)
5f9d3c09b4 p2p: remove torv2 from CNetAddr::ToStringIP() (Jon Atack)
3d39042144 p2p: remove torv2 in SetIP() and ADDR_TORV2_SIZE constant (Jon Atack)
cff5ec477a p2p: remove pre-addrv2 onions from SerializeV1Array() (Jon Atack)
4192a74413 p2p: ignore torv2-in-ipv6 addresses in SetLegacyIPv6() (Jon Atack)
1d631e956f p2p: remove BIP155Network::TORV2 from GetBIP155Network() (Jon Atack)
7d1769bc45 p2p: remove torv2 from SetNetFromBIP155Network() (Jon Atack)
eba9a94b9f fuzz: rename CNetAddr/CService deserialize targets (Jon Atack)
c56a1c9b18 p2p: drop onions from IsAddrV1Compatible(), no longer relay torv2 (Jon Atack)
f8e94002fc p2p: remove torv2/ADDR_TORV2_SIZE from SetTor() (Jon Atack)
0f1c58ae87 test: update feature_proxy to torv3 (Jon Atack)

Pull request description:

  ![image](https://user-images.githubusercontent.com/2415484/120018909-4d425a00-bfd7-11eb-83c9-95a3dac97926.jpeg)

  This patch removes support in Bitcoin Core for Tor v2 onions, which are already removed from the release of Tor 0.4.6.

  - no longer serialize/deserialize and relay Tor v2 addresses
  - ignore incoming Tor v2 addresses
  - remove Tor v2 addresses from the addrman and peers.dat on node launch
  - update generate-seeds.py to ignore Tor v2 addresses
  - remove Tor v2 hard-coded seeds

  Tested with tor-0.4.6.1-alpha (no v2 support) and 0.4.5.7 (v2 support). With the latest Tor (no v2 support), this removes all the warnings like those reported with current master in https://github.com/bitcoin/bitcoin/issues/21351

  ```
  <bitcoind debug log>
  Socks5() connect to […].onion:8333 failed: general failure

  <tor log>
  Invalid hostname [scrubbed]; rejecting
  ```

  and the addrman no longer has Tor v2 addresses on launching bitcoind.
  ```rake
  $ ./src/bitcoin-cli -addrinfo
  {
    "addresses_known": {
      "ipv4": 44483,
      "ipv6": 8467,
      "torv2": 0,
      "torv3": 2296,
      "i2p": 6,
      "total": 55252
    }
  }
  ```
  After recompiling back to current master and restarting with either of the two Tor versions (0.4.5.7 or 0.4.6.1), -addrinfo initially returns 0 Tor v2 addresses and then begins finding them again.

  Ran nodes on this patch over the past week on mainnet/testnet/signet/regtest after building with DEBUG_ADDRMAN.

  Verified that this patch bootstraps an onlynet=onion node from the Tor v3 hardcoded fixed seeds on mainnet and testnet and connects to blocks and v3 onion peers: `rm ~/.bitcoin/testnet3/peers.dat ; ./src/bitcoind -testnet -dnsseed=0 -onlynet=onion`

  ![Screenshot from 2021-05-28 00-26-17](https://user-images.githubusercontent.com/2415484/119905021-ea02ea00-bf3a-11eb-875f-27ef57640c49.png)

  Tested using `addnode`, `getaddednodeinfo`,`addpeeraddress`, `disconnectnode` and `-addrinfo` that a currently valid, connectable Tor v2 peer can no longer be added:

  ![Screenshot from 2021-05-30 11-32-05](https://user-images.githubusercontent.com/2415484/120099282-29435d80-c12a-11eb-81b6-5084244d7d2a.png)

  Thanks to Vasil Dimov, Carl Dong, and Wladimir J. van der Laan for their work on BIP155 and Tor v3 that got us here.

ACKs for top commit:
  laanwj:
    Code review ACK 5d82a57db4

Tree-SHA512: 590ff3d2f6ef682608596facb4b01f44fef69716d2ab3552ae1655aa225f4bf104f9ee08d6769abb9982a8031de93340df553279ce1f5023771f9f2b651178bb
2021-06-03 18:43:55 +02:00
fanquake
8837f1ebde
Merge bitcoin/bitcoin#21654: build, qt: Make Qt rcc output always deterministic
a58868d201 build: Makes rcc output always deterministic (Hennadii Stepanov)

Pull request description:

  The Qt Resource Compiler ([rcc](https://doc.qt.io/qt-5/rcc.html)) has a command-line option `--format-version` which has the [default value](https://code.qt.io/cgit/qt/qtbase.git/tree/src/tools/rcc/main.cpp?h=5.12.10#n172) 2.

  The only difference from `--format-version 1` is adding a [last modified timestamp](https://code.qt.io/cgit/qt/qtbase.git/tree/src/tools/rcc/rcc.cpp?h=5.12.10#n207) to the output file ([credits](https://github.com/bitcoin/bitcoin/pull/21654#issuecomment-819198228) to **fanquake**). That, in turn, forces us to use `QT_RCC_SOURCE_DATE_OVERRIDE=1` to get deterministic builds (#13732).

  This change makes rcc output always deterministic by using `--format-version 1` option that makes usage of the
  `QT_RCC_SOURCE_DATE_OVERRIDE` needless.

  ---

  Also it improves interaction with ccache:

  On master (f6c44e999b):
  ```
  $ make && make clean && ccache --zero-stats && make && ccache --show-stats
  ...
  cache directory                     /home/hebasto/.ccache
  primary config                      /home/hebasto/.ccache/ccache.conf
  secondary config      (readonly)    /etc/ccache.conf
  stats updated                       Sun Apr 11 15:45:43 2021
  stats zeroed                        Sun Apr 11 15:45:05 2021
  cache hit (direct)                   638
  cache hit (preprocessed)               0
  cache miss                             1
  cache hit rate                     99.84 %
  called for link                       10
  cleanups performed                     0
  files in cache                     20023
  cache size                          13.2 GB
  max cache size                      15.0 GB
  ```

  The missed file is always `qt/libbitcoinqt_a-qrc_bitcoin_locale.o`.

  With this PR:
  ```
  $ make && make clean && ccache --zero-stats && make && ccache --show-stats
  ...
  cache directory                     /home/hebasto/.ccache
  primary config                      /home/hebasto/.ccache/ccache.conf
  secondary config      (readonly)    /etc/ccache.conf
  stats updated                       Sun Apr 11 15:28:46 2021
  stats zeroed                        Sun Apr 11 15:28:21 2021
  cache hit (direct)                   639
  cache hit (preprocessed)               0
  cache miss                             0
  cache hit rate                    100.00 %
  called for link                       10
  cleanups performed                     0
  files in cache                     20012
  cache size                          13.2 GB
  max cache size                      15.0 GB
  ```

ACKs for top commit:
  fanquake:
    ACK a58868d201

Tree-SHA512: 52f4a3267f41883d13025c0de79b6da22e92d60c729e01b986935c6812bbfe7fadc40b742bd715bfdf09df94af6838d4fbbe8208c6123f366108e38c8e1121c5
2021-06-03 21:25:30 +08:00
Jon Atack
5d82a57db4
contrib: remove torv2 seed nodes 2021-06-03 14:04:06 +02:00
Jon Atack
5f7e086dac
contrib: update generate-seeds.py to ignore torv2 addresses 2021-06-03 14:04:04 +02:00
MarcoFalke
9999e4c64b
doc: Fix external links (IRC, ...) 2021-05-31 17:27:57 +02:00
W. J. van der Laan
2e8f3928f1
Merge bitcoin/bitcoin#22017: Update Windows code signing certificate
167fb1fc72 Update Windows code signing certificate (Andrew Chow)

Pull request description:

  Updates the Windows code signing certificate to a new one issued by Digicert. This certificate has been issued to Bitcoin Core Code Signing LLC registered in Delaware, US. Note that this is different from the previous Bitcoin Core Code Signing Association registered in Zurich, Switzerland as it was unable to meet the validation requirements in time.

ACKs for top commit:
  Sjors:
    utACK 167fb1f
  laanwj:
    ACK 167fb1fc72

Tree-SHA512: 8d5308c710ef94330417955b9bc82c5894d282798cebece82b84b425e3354e566aa6a68693ec359391ea40ddd7e2032d35ce28d104683d75ec3010ddf00be209
2021-05-27 21:51:58 +02:00
W. J. van der Laan
811aa24c71
Merge bitcoin/bitcoin#22060: contrib: add torv3 seed nodes for testnet, drop v2 ones
6fe0516858 contrib: add torv3 seed nodes for testnet, drop v2 ones (Jon Atack)

Pull request description:

  Replace the ancient (2015) Tor V2 hardcoded seeds with new Tor V3 ones. This needs to be done before 0.22 to make sure onion-only testnet nodes can still connect to the network. Continues #21560.

  Ways to test:
  - Re-generate ` src/chainparamsseeds.h` with `cd contrib/seeds && python3 generate-seeds.py . > ../../src/chainparamsseeds.h`, check if git tree stays the same.
  - Create a new testnet node with `bitcoind -testnet -onlynet=onion -proxy=127.0.0.1:9050` (or delete `~/.bitcoin/testnet3/peers.dat`), check if it is able to connect to the network and get blocks.
  - Check if the addresses are connectable for ex.:
  ```python3
  #!/usr/bin/env python3
  import subprocess
  with open('contrib/seeds/nodes_test.txt') as f:
      for line in (line for line in (line.rstrip().split('#', 1)[0] for line in f) if line):
          subprocess.call(["nc", "-v", "-x", "127.0.0.1:9050", "-z"] + line.split(':'))
  ```

  Thanks to jonatack for providing the list.

ACKs for top commit:
  jonatack:
    ACK 6fe0516858

Tree-SHA512: 61bfdb44dfab9d02b75e5cb06c089a3b1a1fe7134875e1d09166c4116e961d809aa25422fe03f068876e9423b571ecc4a0c7a7eeacba4aac3b2768717f3ee6d6
2021-05-27 11:25:29 +02:00
Carl Dong
108a6be92a guix: Check for disk space availability before building 2021-05-25 22:37:50 -04:00
Carl Dong
d7dec89091 guix: Remove dest if OUTDIR mv fails 2021-05-25 16:40:46 -04:00
Jon Atack
6fe0516858 contrib: add torv3 seed nodes for testnet, drop v2 ones 2021-05-25 16:07:56 +02:00
W. J. van der Laan
b295395664
Merge bitcoin/bitcoin#21239: guix: Add codesignature attachment support for osx+win
ee883201cf guix: repro: Sort find output in libtool for gcc-8 (Carl Dong)
ee0a67c32a codesigning: Use SHA256 as digest for osslsigncode (Windows) (Carl Dong)
38eb91eb06 guix: Add codesigning functionality (Carl Dong)
bac2690e6f guix: Package codesigning tools (Carl Dong)
0a2176d477 guix: Reindent existing manifest.scm (Carl Dong)
c090a3e923 Makefile.am: use APP_DIST_DIR instead of hard-coding dist (Carl Dong)

Pull request description:

  This is the last PR before we reach feature-parity with the Gitian process!

  Note: I tried using the `Makefile` inside the distsrc to make the dmg instead of manually listing out the commands, but `make` seems to want to re-make a lot of other files which broke the dmg.

  The workflow looks something like this:
  1. `env [ FOO=bar... ] ./contrib/guix/guix-build` (add additional env vars as necessary)
  2. Codesigners only:
      1.  Copy `guix-build-<short-id>/output/x86_64-apple-darwin18/bitcoin-<short-id>-osx-unsigned.tar.gz` and `guix-build-<short-id>/output/x86_64-w64-mingw32/bitcoin-<short-id>-win-unsigned.tar.gz` to signing computer
      2. Codesign with `./detached-sig-create.sh` inside the tarball
      3. Upload contents of `signature-{osx,win}.tar.gz` to https://github.com/bitcoin-core/bitcoin-detached-sigs (as a new tag)
  3. Checkout new tag for `bitcoin-core/bitcoin-detached-sigs` with the detached signatures
  4. `env [ FOO=bar... ] DETACHED_SIGS_REPO=<path/to/bitcoin-detached-sigs> ./contrib/guix/guix-codesign` (modify env vars as necessary)
  5. Make sure `guix.sigs` is cloned and updated
  6. `env GUIX_SIGS_REPO=<path/to/guix.sigs> SIGNER=0x96AB007F1A7ED999=dongcarl ./contrib/guix/guix-attest` (modify env vars as necessary)
  7. Commit your new signatures and SHA256SUMS in `guix.sigs`
  8. Optionally, after there are multiple signatures in `guix.sigs`: `env GUIX_SIGS_REPO=<path/to/guix.sigs> ./contrib/guix/guix-verify`

ACKs for top commit:
  laanwj:
    Tested ACK ee883201cf
  achow101:
    ACK ee883201cf

Tree-SHA512: e812a07a5f19f900600c70cb9c717769ef544a6c0c12760b5558b76b6b37df863257f3dbf38b0757e6e06e334470267e94c9f2bdbc27409d6837b1a0bfc6acbc
2021-05-24 15:33:16 +02:00
Andrew Chow
167fb1fc72 Update Windows code signing certificate 2021-05-21 11:49:01 -04:00
Carl Dong
ee883201cf guix: repro: Sort find output in libtool for gcc-8
Otherwise the resulting .a static libraries (e.g. libstdc++.a) will not
be reproducible and end up making the Bitcoin binaries non-reproducible
as well.

See: https://reproducible-builds.org/docs/archives/#gnu-libtool
2021-05-19 15:29:10 -04:00
W. J. van der Laan
ecf5f2c1a0
Merge bitcoin/bitcoin#21889: macho: check for control flow instrumentation
42b589d18f scripts: test for MACHO control flow instrumentation (fanquake)
469a5bc4fa build: build Boost with -fcf-protection when targeting Darwin (fanquake)

Pull request description:

  Addresses the macOS portion of #21888.

  Build Boost with `-fcf-protection` when targeting Darwin. This should be ok, because our cross-compiler (Clang 10) supports the option, and I'd expect all versions of Apple Clang being used to compile Core would also support it. Building Boost with this option is required so that the `main` provided to `test_bitcoin` has instrumentation.

  Note that the presence of instrumentation does not mean it will be used, as that is determined at runtime by the CPU.
  From the Intel control flow enforcement documentation:

  > The ENDBR32 and ENDBR64 instructions will have the same effect as the NOP instruction on Intel 64 processors that do not support CET. On processors supporting CET, these instructions do not change register or flag state. This allows CET instrumented programs to execute on processors that do not support CET. Even when CET is supported and enabled, these NOP–like instructions do not affect the execution state of the program, do not cause any additional register pressure, and are minimally intrusive from power and performance perspectives.

  Follow up from #21135.

  Guix builds:
  ```bash
  663df8471400f06d4da739e39a886aa17f56a36d66e0ff7cc290686294ef39c9  guix-build-42b589d18fed/output/dist-archive/bitcoin-42b589d18fed.tar.gz
  45e841661e1659a634468b6f8c9fb0a7956c31ba296f1fd0c02cd880736d6127  guix-build-42b589d18fed/output/x86_64-apple-darwin18/bitcoin-42b589d18fed-osx-unsigned.dmg
  0ea85c99fef35429a5048fa14850bce6b900eaa887aeea419b019852f8d2be78  guix-build-42b589d18fed/output/x86_64-apple-darwin18/bitcoin-42b589d18fed-osx-unsigned.tar.gz
  85857a5a4a5d4d3a172d6c361c12c4a94f6505fc12b527ea63b75bfe54ee1001  guix-build-42b589d18fed/output/x86_64-apple-darwin18/bitcoin-42b589d18fed-osx64.tar.gz
  ```

  Gitian builds:
  ```bash
  # macOS:
  bdfd677a6b88273a741b433e1e7f554af50cc76b3342d44ab0c441e2b40efc96  bitcoin-42b589d18fed-osx-unsigned.dmg
  f3b2d09f3bea7a5cc489b02e8e53dd76a9922338500fae79cad0506655af56f9  bitcoin-42b589d18fed-osx-unsigned.tar.gz
  29d5ad5e46bc9fb0056922a8b47c026e5e9f71e6cf447203b74644587d6fb6f7  bitcoin-42b589d18fed-osx64.tar.gz
  663df8471400f06d4da739e39a886aa17f56a36d66e0ff7cc290686294ef39c9  src/bitcoin-42b589d18fed.tar.gz
  366f8d7a2fc1f3e22cb1018043099126a71ce65380cc27b1c3280cce42d06c98  bitcoin-core-osx-22-res.yml
  ```

ACKs for top commit:
  laanwj:
    Code review ACK 42b589d18f

Tree-SHA512: 12cb8d462d64d845b9fe48c5c6978892adff8bf5b5572bb29f35df1f6176e47b32a68bcb6e4883c7d9454e76e8868851005a7325916852a2d0d32659ac7dae3f
2021-05-14 12:26:38 +02:00
Carl Dong
ee0a67c32a codesigning: Use SHA256 as digest for osslsigncode (Windows) 2021-05-13 15:41:56 -04:00
Carl Dong
38eb91eb06 guix: Add codesigning functionality 2021-05-13 15:41:56 -04:00
Carl Dong
bac2690e6f guix: Package codesigning tools 2021-05-13 15:41:56 -04:00
Carl Dong
0a2176d477 guix: Reindent existing manifest.scm 2021-05-13 15:41:56 -04:00