Commit Graph

182 Commits

Author SHA1 Message Date
Hennadii Stepanov
1362d6173f
scripted-diff: Insert missed copyright headers
-BEGIN VERIFY SCRIPT-
./contrib/devtools/copyright_header.py insert contrib/guix/libexec/build.sh
./contrib/devtools/copyright_header.py insert contrib/guix/libexec/codesign.sh
./contrib/devtools/copyright_header.py insert contrib/tracing/log_raw_p2p_msgs.py
./contrib/devtools/copyright_header.py insert contrib/tracing/log_utxocache_flush.py
./contrib/devtools/copyright_header.py insert contrib/tracing/p2p_monitor.py
./contrib/devtools/copyright_header.py insert test/lint/lint-files.sh
-END VERIFY SCRIPT-
2021-12-30 19:46:41 +02:00
fanquake
1b76b18f8a
Merge bitcoin/bitcoin#23585: scripted-diff: Drop Darwin version for better maintainability
2f356a0ca8 scripted-diff: Drop Darwin version for better maintainability (Hennadii Stepanov)

Pull request description:

  After this PR, any macOS tools version bumping in the future will touch fewer files in the repo.

  Pointing a Darwin version for the `--host` system does not matter for the following reasons:

  - in terms of the resulted binaries, we should only care about the minimum supported macOS version which is a separated parameter in our build system.

  - in terms of the build system itself, the usage of the `$(host)` variable is self-consistent enough. Btw `$(host_os)` value already has the version dropped:
  ```
  $ make -C depends --no-print-directory print-host_os HOST=x86_64-apple-darwin19
  host_os=darwin
  ```

ACKs for top commit:
  gruve-p:
    ACK 2f356a0ca8
  promag:
    ACK 2f356a0ca8.
  fanquake:
    ACK 2f356a0ca8

Tree-SHA512: 374896ab0ba02b0d8b4b21431fe963bd213b0d09586e0898c13a4c5fa294c1b693f1b2c92880c245c4157c14217b4825b36522f461930477f4d2a727086ebb2a
2021-12-09 16:13:33 +08:00
Hennadii Stepanov
5202bd1dc0
test: Bump shellcheck version to 0.8.0 2021-11-30 21:15:40 +02:00
W. J. van der Laan
c5712d172e
Merge bitcoin/bitcoin#23506: test: Make more shell scripts verifiable by the shellcheck tool
a3f61676e8 test: Make more shell scripts verifiable by the `shellcheck` tool (Hennadii Stepanov)

Pull request description:

  Some shell scripts from `contrib/guix` and `contrib/shell` are not verifiable by the `shellcheck` tool for the following reasons:
  - they have no extension (see 4eccf063b2 from bitcoin/bitcoin#21375)
  - they have the `.bash` extension while `.sh` is expected

  This PR adds these scripts to the input for the `shellcheck` tool, and it fixes discovered `shellcheck` warnings.

ACKs for top commit:
  dongcarl:
    Code Review ACK a3f61676e8, this is a good robustness improvement for our shell scripts.
  jamesob:
    crACK a3f61676e8

Tree-SHA512: 6703f5369d9c04c1a174491f381afa5ec2cc4d37321c1b93615abcdde4dfd3caae82868b699c25b72132d8c8c6f2e9cf24d38eb180ed4d0f0584d8c282e58935
2021-11-30 20:05:44 +01:00
Hennadii Stepanov
a3f61676e8
test: Make more shell scripts verifiable by the shellcheck tool 2021-11-28 14:08:54 +02:00
Hennadii Stepanov
2f356a0ca8
scripted-diff: Drop Darwin version for better maintainability
-BEGIN VERIFY SCRIPT-
sed -i 's/darwin19/darwin/g' $(git grep --files-with-matches 'darwin19')
-END VERIFY SCRIPT-
2021-11-25 01:12:46 +02:00
fanquake
dbfca4a815
build: require glibc 2.18+ for release builds
From what I can see the only platform this drops support for is CentOS
7. CentOS 7 reached the end of it's "full update" support at the end of
2020. It does receive maintenance updates until 2024, however I don't
think supporting glibc 2.17 until 2024 is realistic. Note that anyone
wanting to self-compile and target a glibc 2.17 runtime could build with
--disable-threadlocal.

glibc 2.18 was released in August 2013.
https://sourceware.org/legacy-ml/libc-alpha/2013-08/msg00160.html
2021-11-16 07:51:24 +08:00
fanquake
a7f28af437
Merge bitcoin/bitcoin#22646: build: tighter Univalue integration, remove --with-system-univalue
0f95247246 Integrate univalue into our buildsystem (Cory Fields)
9b49ed656f Squashed 'src/univalue/' changes from 98fadc0909..a44caf65fe (fanquake)

Pull request description:

  This PR more tightly integrates building Univalue into our build system. This follows the same approach we use for [LevelDB](https://github.com/bitcoin-core/leveldb/), ([`Makefile.leveldb.include`](https://github.com/bitcoin/bitcoin/blob/master/src/Makefile.leveldb.include)), and [CRC32C](https://github.com/bitcoin-core/crc32c) ([`Makefile.crc32c.include`](https://github.com/bitcoin/bitcoin/blob/master/src/Makefile.crc32c.include)), and will be the same approach we use for [minisketch](https://github.com/sipa/minisketch); see #23114.

  This approach yields a number of benefits, including:
  * Faster configuration due to one less subconfigure being run during `./configure` i.e 22s with this PR vs 26s
  * Faster autoconf i.e 13s with this PR vs 17s
  * Improved caching
  * No more issues with compiler flags i.e https://github.com/bitcoin/bitcoin/pull/12467
  * More direct control means we can build exactly the objects we want

  There might be one argument against making this change, which is that builders should have the option to use "proper shared/system libraries". However, I think that falls down for a few reasons. The first being that we already don't support building with a number of system libraries (secp256k1, leveldb, crc32c); some for good reason. Univalue is really the odd one out at the moment.

  Note that the only fork of Core I'm aware of, that actively patches in support for using system libs, also explicitly marks them as ["DANGEROUS"](a886811721/configure.ac (L1430)) and ["NOT SUPPORTED"](a886811721/configure.ac (L1312)). So it would seem they exist more to satisfy a distro requirement, as opposed to something that anyone should, or would actually use in practice.

  PRs like #22412 highlight the "issue" with us operating with our own Univalue fork, where we actively fix bugs, and make improvements, when upstream (https://github.com/jgarzik/univalue) may not be taking those improvements, and by all accounts, is not currently actively maintained. Bitcoin Core should not be hamstrung into not being able to fix bugs in a library, and/or have to litter our source with "workarounds", i.e #22412, for bugs we've already fixed, based on the fact that an upstream project is not actively being maintained. Allowing builders to use system libs is really only exacerbating this problem, with little benefit to our project. Bitcoin Core is not quite like your average piece of distro packaged software.

  There is the potential for us to give the same treatment to libsecp256k1, however it seems doing that is currently less straightforward.

ACKs for top commit:
  dongcarl:
    ACK 0f95247246 less my comment above, always nice to have an include-able `sources.mk` which makes integration easier.
  theuni:
    ACK 0f95247246. Thanks fanquake for keeping this going.

Tree-SHA512: a7f2e41ee7cba06ae72388638e86b264eca1b9a8b81c15d1d7b45df960c88c3b91578b4ade020f8cc61d75cf8d16914575f9a78fa4cef9c12be63504ed804b99
2021-10-20 11:01:38 +08:00
Carl Dong
b96adcbfae guix: Fix powerpc64(le) dynamic linker name
I used Guix's values for the powerpc64(le) dynamic linkers, and the
/lib-prefix seems to be a Guix-ism rather than standard. The standard
path for the linker-loaders start with /lib64.

I've taken the new loader values from SYSDEP_KNOWN_INTERPRETER_NAMES in
glibc's sysdeps/unix/sysv/linux/powerpc/ldconfig.h file.

For future reference, loader path values can also be found on glibc's
website: https://sourceware.org/glibc/wiki/ABIList?action=recall&rev=16
2021-10-13 08:38:40 -04:00
Cory Fields
0f95247246
Integrate univalue into our buildsystem
This addresses issues like the one in #12467, where some of our compiler flags
end up being dropped during the subconfigure of Univalue. Specifically, we're
still using the compiler-default c++ version rather than forcing c++17.

We can drop the need subconfigure completely in favor of a tighter build
integration, where the sources are listed separately from the build recipes,
so that they may be included directly by upstream projects. This is
similar to the way leveldb build integration works in Core.

Core benefits of this approach include:
- Better caching (for ex. ccache and autoconf)
- No need for a slow subconfigure
- Faster autoconf
- No more missing compile flags
- Compile only the objects needed

There are no benefits to Univalue itself that I can think of. These changes
should be a no-op there, and to downstreams as well until they take advantage
of the new sources.mk.

This also removes the option to use an external univalue to avoid similar ABI
issues with mystery binaries.

Co-authored-by: fanquake <fanquake@gmail.com>
2021-10-11 20:46:25 +08:00
fanquake
8f022a59b8
Merge bitcoin/bitcoin#22993: build: set OSX_MIN_VERSION to 10.15
a43b8e9555 build: set OSX_MIN_VERSION to 10.15 (fanquake)

Pull request description:

  Taken out of #20744, as splitting up some of the build changes was mentioned [here](https://github.com/bitcoin/bitcoin/pull/22937#discussion_r707303172).

  This is required to use `std::filesystem` on macOS, as support for it only landed in the libc++.dylib shipped with 10.15. So if we want to move to using `std::filesystem` for `23.0`, this bump is required.

  See also: https://developer.apple.com/documentation/xcode-release-notes/xcode-11-release-notes

  > Clang now supports the C++17 \<filesystem\> library for iOS 13, macOS 10.15, watchOS 6, and tvOS 13.

  macOS 10.15 was released in October 2019. macOS OS's seem to have a life of about 3 years, so it's possible that 10.14 will become officially unsupported by the end of 2021 and prior to the release of 23.0.

  Guix builds:
  ```bash
  bash-5.1# find guix-build-$(git rev-parse --short=12 HEAD)/output/ -type f -print0 | env LC_ALL=C sort -z | xargs -r0 sha256sum
  abc8b749be65f1339dcdf44bd1ed6ade2533b8e3b5030ad1dde0ae0cede78136  guix-build-a43b8e955558/output/dist-archive/bitcoin-a43b8e955558.tar.gz
  1edcc301eb4c02f3baa379beb8d4c78e661abc24a293813bc9d900cf7255b790  guix-build-a43b8e955558/output/x86_64-apple-darwin19/SHA256SUMS.part
  e9dbb5594a664519da778dde9ed861c3f0f631525672e17a67eeda599f16ff44  guix-build-a43b8e955558/output/x86_64-apple-darwin19/bitcoin-a43b8e955558-osx-unsigned.dmg
  11b23a17c630dddc7594c25625eea3de42db50f355733b9ce9ade2d8eba3a8f3  guix-build-a43b8e955558/output/x86_64-apple-darwin19/bitcoin-a43b8e955558-osx-unsigned.tar.gz
  257ba64a327927f94d9aa0a68da3a2695cf880b3ed1a0113c5a966dcc426eb5e  guix-build-a43b8e955558/output/x86_64-apple-darwin19/bitcoin-a43b8e955558-osx64.tar.gz
  ```

ACKs for top commit:
  hebasto:
    ACK a43b8e9555
  jarolrod:
    ACK a43b8e9

Tree-SHA512: 9ac77be7cb56c068578860a3b2b8b7487c9e18b71b14aedd77a9c663f5d4bb19756d551770c02ddd12f1797beea5757b261588e7b67fb53509bb998ee8022369
2021-09-21 15:37:12 +08:00
Jeremy Rand
9bdda50151
Enable TLS in links in documentation 2021-09-16 22:00:20 +00:00
fanquake
a43b8e9555
build: set OSX_MIN_VERSION to 10.15
This is required to use std::filesystem on macOS as support for it only
landed in the libc++ dylib shipped with 10.15.

See also: https://developer.apple.com/documentation/xcode-release-notes/xcode-11-release-notes

Clang now supports the C++17 <filesystem> library for iOS 13, macOS 10.15, watchOS 6, and tvOS 13.
2021-09-16 17:50:19 +08:00
Carl Dong
96cc6bb04f guix/prelude: Override VERSION with FORCE_VERSION
Previously, if the builder exported $VERSION in their environment (as
past Gitian-building docs told them to), but their HEAD does not
actually point to v$VERSION, their build outputs will differ from those
of other builders.

This is because the contrib/guix/guix-* scripts only ever act on the
current git worktree, and does not try to check out $VERSION if $VERSION
is set in the environment.

Setting $VERSION only makes the scripts pretend like the current
worktree is $VERSION.

This problem was seen in jonatack's attestation for all.SHA256SUMS,
where only his bitcoin-22.0rc3-osx-signed.dmg differed from everyone
else's.

Here is my deduced sequence of events:

1. Aug 27th: He guix-builds 22.0rc3 and uploads his attestations up to
   guix.sigs

2. Aug 30th, sometime after POSIX time 1630310848: he pulls the latest
   changes from master in the same worktree where he guix-built 22.0rc3
   and ends up at 7be143a960

3. Aug 30th, sometime before POSIX time 1630315907: With his worktree
   still on 7be143a960, he guix-codesigns. Normally, this would result
   in outputs going in guix-build-7be143a960e2, but he had
   VERSION=22.0rc3 in his environment, so the guix-* scripts pretended
   like he was building 22.0rc3, and used 22.0rc3's guix-build directory
   to locate un-codesigned outputs and dump codesigned ones.

   However, our SOURCE_DATE_EPOCH defaults to the POSIX time of HEAD
   (7be143a960), which made all timestamps in the resulting codesigned
   DMG 1630310848, 7be143a960e2's POSIX timestamp. This differs from the
   POSIX timestamp of 22.0rc3, which is 1630348517. Note that the
   windows codesigning procedure does not consider SOURCE_DATE_EPOCH.

We resolve this by only allowing VERSION overrides via the FORCE_VERSION
environment variable.
2021-08-31 10:57:04 -04:00
Andrew Chow
fb17c99e35 guix: Don't include directory name in SHA256SUMS
The SHA256SUMS file can be used in a sha256sum -c command to verify
downloaded binaries. However users are likely to download just a single
file and not place this file in the correct directory relative to the
SHA256SUMS file for the simple verification command to work. By not
including the directory name in the SHA256SUMS file, it will be easier
for users to verify downloaded binaries.

Co-authored-by: Carl Dong <contact@carldong.me>
2021-08-18 20:07:32 -04:00
fanquake
8193294cab
Merge bitcoin/bitcoin#22642: release: Release with separate SHA256SUMS and sig files
90b3e482e9 release: Release with separate SHA256SUMS and sig files (Carl Dong)

Pull request description:

  This allows us to:
  - remove the rfc4880 EOL hacks, and
  - release with a SHA256SUMS.asc file that's a combination of all signer signatures

ACKs for top commit:
  achow101:
    ACK 90b3e482e9
  laanwj:
    Concept and code review ACK 90b3e482e9

Tree-SHA512: 5d5086063d303aa0cbd590e5fdf2ae8f555e25f4e43bf67545e33384449b990e94834c711622530ad0eb3dcc83f52746884a5081dadb0acff8dd799cfadafac7
2021-08-09 16:19:49 +08:00
Carl Dong
d451b60d22 guix-verify: Non-zero exit code when anything fails
Previously, if verification fails, the correct message will be printed,
but the exit code would still be 0.
2021-08-05 19:05:16 -04:00
Carl Dong
90b3e482e9 release: Release with separate SHA256SUMS and sig files
This allows us to remove the rfc4880 EOL hacks and release with a
SHA256SUMS.asc file that's a combination of all signer signatures.
2021-08-05 16:58:54 -04:00
fanquake
8f76f9f613
Merge bitcoin/bitcoin#22531: guix: Fixes to guix-{attest,verify}
9b313dfef1 guix: Ensure EPOCH_SOURCE_DATE does not include GPG information (Andrew Chow)
43225f0a2a guix: Remove extra \r from all.SHA256SUMS line ending (Andrew Chow)
d080c27066 guix, doc: Add a note that codesigners need to rebuild after tagging (Andrew Chow)
4a466388a0 guix: Allow changing the base manifest in guix-verify (Andrew Chow)
33455c7696 guix: Make all.SHA256SUMS rather than codesigned.SHA256SUMS (Andrew Chow)

Pull request description:

  `guix-verify` expects `all.SHA256SUMS` but `guix-attest` produces `codesigned.SHA256SUMS`. Since `all.SHA256SUMS` makes more sense (as the file contains all the sha256sums, not just the codesigned ones), `guix-attest` has been changed to output a file of that name.

  As a quality of life improvement, `guix-verify` can take `SIGNER` and use the signer's manifest as the base to compare against. This makes it easier to compare a single person's attestations with everyone else's and can make it more obvious when one builder is clearly mismatching with everyone else.

  Lastly `release-process.md` is updated with a note about a gotcha that can cause a mismatch in the codesigned attestation.

ACKs for top commit:
  fanquake:
    ACK 9b313dfef1

Tree-SHA512: 0d60627def38288dbd3059ad1e72cad224f9205da11b1a561c082ef28250a074df5cc5f2797c91a7be027bc486a3fda3319c2e496a8724e5b539337236c6f990
2021-07-29 11:13:12 +08:00
Andrew Chow
9b313dfef1 guix: Ensure EPOCH_SOURCE_DATE does not include GPG information
If the user has set log.showSignature=true in their git config, then the
git log will always output GPG signature information. Since git log is
used to set EPOCH_SOURCE_DATE, this will mistakenly have GPG signature
information in it which causes issues for the build. To avoid this
issue, we override the config and force log.showSignature=false.
2021-07-28 12:59:47 -04:00
Andrew Chow
43225f0a2a guix: Remove extra \r from all.SHA256SUMS line ending
guix-attest mistakenly added an extra \r to the line endings in
all.SHA256SUMS, causing guix-verify to erroneously fail.

Co-Authored-By: Carl Dong <contact@carldong.me>
2021-07-28 12:59:47 -04:00
jonatack
198ceb82f9 script, doc: guix touchups 2021-07-23 21:23:37 +05:30
h
d7b7f610a5 Updated Readme, Corrected the codesign typo 2021-07-23 20:15:16 +05:30
fanquake
2b5563bb1e
Merge bitcoin/bitcoin#22533: guix/build: Remove vestigial SKIPATTEST.TAG
9f01feda0a guix/build: Remove vestigial SKIPATTEST.TAG (Carl Dong)

Pull request description:

  No longer needed or referenced by anything. A relic from prior to the great hierarchy overhaul of #22182

ACKs for top commit:
  achow101:
    ACK 9f01feda0a
  fanquake:
    ACK 9f01feda0a

Tree-SHA512: a94cf63f0c5cb8dbacf1025b6c0e81b219c2a3c93b3cbcefc239ccde29e602ecd4b717b1d93dbe53cb791a5017236fb09823c034aec42b0c31894fc9e0ab8b21
2021-07-23 14:03:24 +08:00
Andrew Chow
4a466388a0 guix: Allow changing the base manifest in guix-verify
When verifying guix attestations, it is useful to set a particular
signer's manifest as the base to compare against.
2021-07-22 18:48:25 -04:00
Andrew Chow
33455c7696 guix: Make all.SHA256SUMS rather than codesigned.SHA256SUMS 2021-07-22 17:57:35 -04:00
Carl Dong
9f01feda0a guix/build: Remove vestigial SKIPATTEST.TAG 2021-07-22 17:47:22 -04:00
Carl Dong
a884a1edcd guix/INSTALL: Misc fixups 2021-07-22 17:42:46 -04:00
Carl Dong
3c4d2c418e guix: Silence getent(1) invocation 2021-07-20 19:47:28 -04:00
fanquake
201c5e4aec
Merge bitcoin/bitcoin#22465: guix: Pin kernel-header version, time-machine to upstream 1.3.0 commit
e6a94d4446 guix: Bump to version-1.3.0 from upstream (Carl Dong)
90fd13b954 guix: Pin kernel header version (Carl Dong)

Pull request description:

  ```
  - Use 4.19 for riscv64 (earliest LTS release w/ riscv64 support)
  - Use 4.9 for all others (second-oldest LTS release, released in
    combination with glibc glibc 2.24 in Debian stretch)
  ```

  ```
  The chosen commit is the HEAD of Guix's version-1.3.0 branch as of July
  15th, 2021.

  Also fix visual indenting.
  ```

  -----

  This + the documentation PR should make our Guix system ready for release!

ACKs for top commit:
  MarcoFalke:
    review ACK e6a94d4446 to change to vanilla guix. Did not review the kernel change.
  laanwj:
    ACK e6a94d4446
  fanquake:
    ACK e6a94d4446

Tree-SHA512: a175e4ddb3ee786a39f5e800ce336932ad2f6797a3a28400a6f723875d0f19833fd36cedc41b3580e4604110517211bd9f557be36adf7265fd8e591c434ae032
2021-07-20 12:23:56 +08:00
fanquake
e7441a6a45
Merge bitcoin/bitcoin#21711: guix: Add full installation and usage documentation
fac4814106 doc/release-process: Add torrent creation details (Carl Dong)
5d24cc3d82 guix/INSTALL: Guix installs init scripts in libdir (Carl Dong)
5da2ee49d5 guix/INSTALL: Add coreutils/inotify-dir-recreate troubleshooting (Carl Dong)
318c60700b guix: Adapt release-process.md to new Guix process (Carl Dong)
fcab35b229 guix-attest: Produce and sign normalized documents (Carl Dong)
c2541fd0ca guix: Overhaul README (Carl Dong)
46ce6ce378 tree-wide: Rename gitian-keys to builder-keys (Carl Dong)
fc4f8449f3 guix: Update various check_tools lists (Carl Dong)
263220a85c guix: Check for a sane services database (Carl Dong)

Pull request description:

  Based on: #21462

  Keeping the README in one file so that it's easy to search through. Will add more jumping links later so navigation is easier.

  Current TODOs:
  - [x] Shell installer option: prompt user to re-login for `/etc/profile.d` entry to be picked up
  - [x] Binary tarball option: prompt user to create `/etc/profile.d` entry and re-login
  - [x] Fanquake docker option: complete section
  - [x] Arch Linux AUR option: prompt to start `guix-daemon-latest` unit after finishing "optional setup" section
  - [x] Building from source option: Insert dependency tree diagram that I made
  - [x] Building from source option: redo sectioning, kind of a mess right now
  - [x] Optional setup: make clear which parts are only needed if building from source
  - [x] Workaround 1 for GnuTLS: perhaps mention how to remove Guix build farm's key
  - [x] Overall (after everything): Make the links work.

  Note to self: wherever possible, tell user how to check that something is true rather than branching by installation option.

ACKs for top commit:
  fanquake:
    ACK fac4814106 - going to go ahead and merge this now. It's a lot of documentation, and could probably be nit-picked / improved further, however, that can continue over the next few weeks. I'm sure more (backportable) improvements / clarifications will be made while we progress through RCs towards a new release.

Tree-SHA512: dc46c0ecdfc67c7c7743ca26e4a603eb3f54adbf81be2f4c1f4c20577ebb84b5250b9c9ec89c0e9860337ab1c7cff94d7963c603287267deecfe1cd987fa070a
2021-07-20 11:09:39 +08:00
Carl Dong
5d24cc3d82 guix/INSTALL: Guix installs init scripts in libdir 2021-07-19 18:24:36 -04:00
Carl Dong
5da2ee49d5 guix/INSTALL: Add coreutils/inotify-dir-recreate troubleshooting 2021-07-19 18:24:36 -04:00
Carl Dong
fcab35b229 guix-attest: Produce and sign normalized documents
That way we can easily combine the document and detached signature to
produce cleartext signature files for upload during the release process.

See subsequent commits which modify doc/release-process.md for more
details.
2021-07-19 18:24:36 -04:00
Carl Dong
c2541fd0ca guix: Overhaul README
- Added detailed Guix bootstrap/installation instructions
2021-07-19 18:24:36 -04:00
fanquake
4371e635d6
Merge bitcoin/bitcoin#22410: Avoid GCC 7.1 ABI change warning in guix build
1edddf5de4 Avoid GCC 7.1 ABI change warning in guix build (Pieter Wuille)

Pull request description:

  The arm-linux-gnueabihf guix build output is littered with warnings like:

  ```
  /gnu/store/7a96hdqdb2qi8a39f09n84xjy2hr23rs-gcc-cross-arm-linux-gnueabihf-8.4.0/include/c++/bits/stl_vector.h:1085:4: note:
                   parameter passing for argument of type '__gnu_cxx::__normal_iterator<CRecipient*, std::vector<CRecipient> >' changed in GCC 7.1
  ```

  These are irrelevant for us. Disable them using `-Wno-psabi`.

ACKs for top commit:
  laanwj:
    ACK 1edddf5de4
  hebasto:
    ACK 1edddf5de4, after thorough reading related materials, I agree this change can be merged. As I mentioned above, I have been compiling my arm-32bit binaries with `-Wno-psabi` flag for two years, and no related flaws were observed.

Tree-SHA512: 485c7500547ac5da567ad23847341c18ff832607f5a1002676404cc647e437cf3445b6894ecff5b52929ca52bea946c06bd90eace1997c895e56204e787065e4
2021-07-18 14:33:22 +08:00
Carl Dong
e6a94d4446 guix: Bump to version-1.3.0 from upstream
The chosen commit is the HEAD of Guix's version-1.3.0 branch as of July
15th, 2021.

Also fix visual indenting.
2021-07-15 21:50:48 -04:00
Carl Dong
90fd13b954 guix: Pin kernel header version
- Use 4.19 for riscv64 (earliest LTS release w/ riscv64 support)
- Use 4.9 for all others (second-oldest LTS release, released in
  combination with glibc glibc 2.24 in Debian stretch)
2021-07-15 21:47:57 -04:00
Carl Dong
fc4f8449f3 guix: Update various check_tools lists 2021-07-08 11:29:26 -04:00
Carl Dong
263220a85c guix: Check for a sane services database
On bare systems, it is possible to be lacking a services database. Check
for basic entries before attempting a build.

See the error message in the diff for more context.
2021-07-08 11:28:07 -04:00
Carl Dong
5b4703c6a7
guix: Test security-check sanity before performing them 2021-07-07 20:29:05 +08:00
Carl Dong
678348db51
guix: Patch binutils to add security-related disable flags
We use these flags in our test-security-check make target, but they are
only available because debian patches them in.

We can patch them in for our Guix builds so that we can check the sanity
of our security/symbol checking suite before running them.
2021-07-07 19:31:37 +08:00
Pieter Wuille
1edddf5de4 Avoid GCC 7.1 ABI change warning in guix build 2021-07-06 10:41:43 -07:00
fanquake
15fc9a0299
guix: add additional documentation to patches 2021-07-06 20:50:47 +08:00
fanquake
de6ca41a52
guix: no-longer pass --enable-glibc-back-compat to Guix
Now that our Guix builds are performed on glibc 2.24 and 2.27 (RISCV),
we no-longer need to pass the --enable-glibc-back-compat option.

Replace it with --disable-threadlocal, to prevent the usage of symbols
from glibc 2.18.

None of the binaries produced required symbols later than 2.17, and 2.27
(RISCV).
2021-07-06 20:50:40 +08:00
Carl Dong
647f7e5f1d guix: Also sort SHA256SUMS.part 2021-07-02 14:10:33 -04:00
Carl Dong
dc4137a60c guix: Build depends/qt with our platform definition
Our 'bitcoin-linux-g++' definition better integrates with our depends
system than the stock linux-g++-64 definition.

This fixes a bug whereby Guix builds on x86_64 for x86_64 did not
produce a QMinimalIntegrationPlugin and led to bitcoin-qt not being
built.
2021-07-01 16:20:28 -04:00
Carl Dong
16b0a936e1 guix: Rebase toolchain on glibc 2.24 (2.27 for riscv64)
Support for riscv64 in glibc landed in 2.27 so it's unavoidable that we
use 2.27.

Running a Bitcoin build with toolchains based on 2.24 for platforms
other than riscv64 seem to produce binaries which do not have 2.17
symbols. So use 2.24 since it's more recent and maintained by Debian
Stretch.
2021-07-01 16:17:03 -04:00
fanquake
da69d9965a
Merge bitcoin/bitcoin#21871: scripts: add checks for minimum required OS versions
aa80b5759d scripts: check macOS SDK version is set (fanquake)
c972345bac scripts: check minimum required Windows version is set (fanquake)
29615aef52 scripts: check minimum required macOS vesion is set (fanquake)
8732f7b6c9 scripts: LIEF 0.11.5 (fanquake)

Pull request description:

  macOS:
  We use a compile flag ([-mmacosx-version-min=10.14](https://github.com/bitcoin/bitcoin/blob/master/depends/hosts/darwin.mk#L96)) to set the minimum required version of macOS needed to run our binaries. This adds a sanity check that the version is being set as expected.

  Clangs Darwin driver should infer the SDK version used during compilation, and forward that through to the linker. Add a check that this has been done, and the expected SDK version is set. Should help prevent issues like #21771 in future.

  Windows:
  We use linker flags ([-Wl,--major/minor-subsystem-version](https://github.com/bitcoin/bitcoin/blob/master/configure.ac#L683)) to set the minimum required version of Windows needed to run our binaries. This adds a sanity check that the version is being set as expected.

  Gitian builds:
  ```bash
  # macOS:
  8b6fcd61d75001c37b2af3fceb5ae09f5d2fe85e97d361f684214bd91c27954a  bitcoin-f015e1c2cac9-osx-unsigned.dmg
  3c1e412bc7f5a7a5d0f78e2cd84b7096831414e1304c1307211aa3e135d89bbf  bitcoin-f015e1c2cac9-osx-unsigned.tar.gz
  50b7b2804e8481f63c69c78e3e8a71c0d811bf2db8895dd6d3edae9c46a738ae  bitcoin-f015e1c2cac9-osx64.tar.gz
  fe6b5c0a550096b76b6727efee30e85b60163a41c83f21868c849fdd9876b675  src/bitcoin-f015e1c2cac9.tar.gz
  8a20f21b20673dfc8c23e22b20ae0839bcaf65bf0e02f62381cdf5e7922936f0  bitcoin-core-osx-22-res.yml

  # Windows:
  b01fcdc2a5673387050d6c6c4f96f1d350976a121155fde3f76c2af309111f9d  bitcoin-f015e1c2cac9-win-unsigned.tar.gz
  b95bdcbef638804030671d2332d58011f8c4ed4c1db87d6ffd211515c32c9d02  bitcoin-f015e1c2cac9-win64-debug.zip
  350bf180252d24a3d40f05e22398fec7bb00e06d812204eb5a421100a8e10638  bitcoin-f015e1c2cac9-win64-setup-unsigned.exe
  2730ddabe246d99913c9a779e97edcadb2d55309933d46f1dffd0d23ecf9aae5  bitcoin-f015e1c2cac9-win64.zip
  fe6b5c0a550096b76b6727efee30e85b60163a41c83f21868c849fdd9876b675  src/bitcoin-f015e1c2cac9.tar.gz
  aa60d7a753e8cb2d4323cfbbf4d964ad3645e74c918cccd66862888f8646d80f  bitcoin-core-win-22-res.yml
  ```

ACKs for top commit:
  hebasto:
    ACK aa80b5759d, tested by breaking tests:

Tree-SHA512: 10150219910e8131715fbfe20edaa15778387616ef3bfe1a5152c7acd3958fe8f88c74961c3d3641074eb72824680c22764bb1dc01a19e92e946c2d4962a8d2c
2021-06-18 15:21:47 +08:00
fanquake
d50302625e
Merge bitcoin/bitcoin#22182: guix: Overhaul how guix-{attest,verify} works and hierarchy
e2c40a4ed5 guix-attest: Error out if SHA256SUMS is unexpected (Carl Dong)
4cc35daed5 Rewrite guix-{attest,verify} for new hier (Carl Dong)
28a9c9b839 Make SHA256SUMS fragment right after build (Carl Dong)

Pull request description:

  Based on:  #22075
  Code reviewers: I recommend reading the new `guix-{attest,verify}` files instead of trying to read the diff

  The following changes resolve many usability improvements which were pointed out to me:
  1. Some maintainers like to extract their "uncodesigned tarball" inside the `output/` directory, resulting in the older `guix-attest` mistakenly attesting to the extracted contents
  2. Maintainers whose GPG keys reside on an external smartcard often need to physically interact with the smartcard as a way to approve the signing operation, having one signature per platform means a lot of fidgeting
  3. Maintainers wishing to sign on a separate machine now has the option of transferring only a subtree of `output/`, namely `output/*/SHA256SUMS.part`, in order to perform a signature (you may need to specify an `$OUTDIR_BASE` env var)
  4. An `all.SHA256SUMS` file should be usable as the base `SHA256SUMS` in bitcoin core torrents and on the release server.

  For those who sign on an separate machine than the one you do builds on, the following steps will work:
  1. `env GUIX_SIGS_REPO=/home/achow101/guix.sigs SIGNER=achow101 NO_SIGN=1 ./contrib/guix/guix-attest`
  2. Copy `/home/achow101/guix.sigs/<tag>/achow101` (which does not yet have signatures) to signing machine
  3. Sign the `SHA256SUMS` files:
      ```bash
      for i in "<path-to-achow101>/*.SHA256SUMS"; do
          gpg --detach-sign --local-user "<your-key-here>" --armor --output "$i"{.asc,}
      done
      ```
  5. Upload `<path-to-achow101>` (now with signatures) to `guix.sigs`

  -----

  After this change, output directories will now include a `SHA256SUMS.part` fragment, created immediately after a successful build:
  ```
  output
  └── x86_64-w64-mingw32
      ├── bitcoin-4e069f7589da-win64-debug.zip
      ├── bitcoin-4e069f7589da-win64-setup-unsigned.exe
      ├── bitcoin-4e069f7589da-win64.zip
      ├── bitcoin-4e069f7589da-win-unsigned.tar.gz
      └── SHA256SUMS.part
  ```

  These `SHA256SUMS.part` fragments look something like:
  ```
  3ebd7262b1a0a5bb757fef1f70e7e14033c70f98c059bc4dbfee5d1992b25825  dist-archive/bitcoin-4e069f7589da.tar.gz
  def2e7d3de5ab3e3f955344e75151df4f33713f9101f5295bd13c9375bdf633b  x86_64-w64-mingw32/bitcoin-4e069f7589da-win64-debug.zip
  643049fe3ee4a4e83a1739607e67b11b7c9b1a66208a6f35a9ff634ba795500e  x86_64-w64-mingw32/bitcoin-4e069f7589da-win64-setup-unsigned.exe
  a247a1ccec0ccc2e138c648284bd01f6a761f2d8d6d07d91b5b4a6670ec3f288  x86_64-w64-mingw32/bitcoin-4e069f7589da-win-unsigned.tar.gz
  fab76a836dcc592e39c04fd2396696633fb6eb56e39ecbf6c909bd173ed4280c  x86_64-w64-mingw32/bitcoin-4e069f7589da-win64.zip
  ```

  Meaning that they are valid `SHA256SUMS` files when `sha256sum --check`'d at the `guix-build-*/output` directory level

  When `guix-attest` is invoked, these `SHA256SUMS.part` files are combined and sorted (by `-k2`, `LC_ALL=C`) to create:

  1. `noncodesigned.SHA256SUMS` for a manifest of all non-codesigned outputs, and
  3. `all.SHA256SUMS` for a manifest of all outputs including non-codesigned outputs

  Then both files are signed, resulting in the following `guix.sigs` hierarchy:
  ```
  4e069f7589da/
  └── dongcarl
      ├── all.SHA256SUMS
      ├── all.SHA256SUMS.asc
      ├── noncodesigned.SHA256SUMS
      └── noncodesigned.SHA256SUMS.asc
  ```

ACKs for top commit:
  achow101:
    ACK e2c40a4ed5
  hebasto:
    ACK e2c40a4ed5, tested on Linux Mint 20.1 (x86_64) with and w/o `NO_SIGN=1`. Changes in `contrib/guix/libexec/codesign.sh` and `contrib/guix/guix-verify` are reviewed only.

Tree-SHA512: 618aacefb0eb6595735a9ab6a98ea6598fce65f9ccf33fa1e7ef93bf140c0f6cfc16e34870c6aa3e4777dd3f004b92a82a994141879870141742df948ec59c1f
2021-06-17 13:10:37 +08:00