Commit Graph

262 Commits

Author SHA1 Message Date
Hennadii Stepanov
fd7caae35f
build: Disable --disable-fuzz-binary for gitian builds 2021-02-08 15:38:26 +02:00
fanquake
2ecaf21433
gitian: remove execstack workaround for ricv64 & powerpc64le
When building with g++-10 (or 8) on Focal, binaries are being produced
with noexecstack by default, so we can remove the workaround of
explicitly passing "-Wl,-z,noexecstack" for risvc46 and powerpc64le.

When building for powerpc64 this is still required.
2021-02-03 21:47:16 +08:00
fanquake
5baff2b318
build: use focal in gitian descriptors
Compilers used change as follows:
Linux native GCC 7.5 -> GCC 8.4
Linux cross GCC 8.4 -> GCC 8.4
Windows mingw-w64 7.3 -> mingw-w64 9.3
macOS Clang 8.0.0 -> Clang 8.0.0

The macOS and Win cross builds in the CI are updated to use Focal, and
per the op, running the security tests is disabled in the Windows
build.
2021-02-03 21:39:54 +08:00
Wladimir J. van der Laan
543bf745d3 gitian-linux: Extend noexec-stack workaround to powerpc 2021-01-24 23:33:16 +01:00
Luke Dashjr
00f67c8aa1 gitian-linux: Build binaries for 64-bit POWER 2021-01-24 23:27:39 +01:00
Luke Dashjr
63fc2b1782 gitian: Properly quote arguments in wrappers 2021-01-19 14:41:22 +01:00
Andrew Chow
2c403279e2 gitian: Remove codesign_allocate and pagestuff from MacOS build 2021-01-07 15:33:19 -05:00
Andrew Chow
95b06d2185 gitian: use signapple to apply the MacOS code signature 2021-01-07 15:33:19 -05:00
Andrew Chow
42bb1ea363 gitian: install signapple in gitian-osx-signer.yml 2021-01-07 15:33:19 -05:00
fanquake
0df9819126 build: Replace genisoimage with xorriso
xorriso and its mkisofs/genisoimage emulation alter-ego xorrisofs are
more maintained, and has the right toggles for us to achieve output
determinism without using blunt tools like faketime.

In this commit, we use xorrisofs from the build environment rather than
building it ourselves using depends. This is not necessary and can be
changed in the future.

From https://wiki.debian.org/genisoimage?action=recall&rev=11 :

> The classical command line interface for production of ISO 9660
> filesystem images is the option set established by program mkisofs.
> For reasons of licensing and other problems with its author, Debian
> ships a fork of mkisofs, called genisoimage, which was split off in
> 2006 and then developed independently.
>
> Meanwhile, genisoimage gets no new features and not even bug fixes. It
> is first choice only if its options -udf or -hfs are needed.
>
> Replacement in most uses cases, especially for bootable ISO 9660
> filesystems, archiving, and backup, is xorrisofs which starts the -as
> mkisofs emulation mode of program xorriso.
2020-12-16 12:29:14 -05:00
Carl Dong
22437fc72e build: Run libdmg-hfsplus's DMG tool in make deploy
Previously, the compression of the .iso file to a .dmg file was done
outside of `make deploy' in order to use the faketime-wrapped version of
libdmg-hfsplus's DMG tool.

Specifying the faketime-wrapped version of the DMG tool to ./configure
fixes this and simplifies build scripts.
2020-12-11 11:43:11 -05:00
Wladimir J. van der Laan
555b5d1bf9
Merge #20419: build: set minimum supported macOS to 10.14
a52ecc936a build: set minimum supported macOS to 10.14 (fanquake)

Pull request description:

  This is a requirement for C++17 support. See my comments [here](https://github.com/bitcoin/bitcoin/issues/16684#issuecomment-643722538):

  > You cannot use std::get with std::variant on macOS < 10.14, because Apples libc++ doesn't support the std::bad_variant_access exception. [Relevant comment](https://github.com/bitcoin/bitcoin/pull/19183#discussion_r439794318) in #19183.

  > While we could work around this in our own code, using std::get_if, this would still be a problem for 3rd-party dependencies.

  > I've been testing Qt 5.15LTS (we'll have to enable C++17 in qt, and may upgrade to a newer version at the same time), and you can't enable -std c++17, while targeting a macOS deployment version < 10.14, configuring will fail. They are making use of std::get with std::variant throughout their cocoa code.

  We would have to had to have bumped to at least 10.13 in any case, as Qt 5.15 (#19716) [requires 10.13+](https://doc.qt.io/qt-5/supported-platforms.html).

ACKs for top commit:
  hebasto:
    ACK a52ecc936a, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: f669b2fc777aeea1e9afdbbc7bd9afe3997418211db6ba53c934cae0e62a9b999603da539518c229f34961d275c9e2f315c7b022cf5fb97bd201a69c85d470cc
2020-11-23 14:24:58 +01:00
Andrew Chow
8f7b930475 Drop the leading 0 from the version number
Removes the leading 0 from the version number. The minor version, which
we had been using as the major version, is now the major version. The
revision, which we had been using as the minor version, is now the minor
version. The revision number is dropped. The build number is promoted to
being part of the version number. This also avoids issues where it was
accidentally not included in the version number.

The CLIENT_VERSION remains the same format as previous as previously,
the Major version was 0 so that was never a factor in CLIENT_VERSION.
2020-11-18 12:00:57 -05:00
fanquake
a52ecc936a
build: set minimum supported macOS to 10.14 2020-11-18 21:46:09 +08:00
fanquake
8a715a6b17
build: Bump gitian descriptors to 0.22 2020-11-18 20:52:44 +08:00
MarcoFalke
faa2f06f5e
scripted-diff: [build] Ensure source tarball has leading directory name
-BEGIN VERIFY SCRIPT-
sed -i 's|git archive --|git archive --prefix="${DISTNAME}/" --|g'                          $(git grep -l 'git archive' ./contrib)
sed -i 's|tar -xf "\?${\?GIT_ARCHIVE}\?"\?|tar --strip-components=1 -xf "${GIT_ARCHIVE}"|g' $(git grep -l 'tar -xf'     ./contrib)
-END VERIFY SCRIPT-
2020-11-05 17:31:24 +01:00
Andrew Chow
e87df82580 Add sqlite to travis and depends 2020-10-14 11:18:13 -04:00
Hennadii Stepanov
33a84e8f40
build: Update and sort package list in gitian-linux.yml
Virtual package 'binutils-gold' replaced with 'binutils'.
Explicitly added 'patch' package.
2020-08-05 18:08:47 +03:00
Hennadii Stepanov
95051682be
build: Drop old hack which is unneeded now
This hack is described in #8188. The current implementation was
introduced in #8315.
2020-07-31 17:10:40 +03:00
Cory Fields
2418f739f7
macos: Bump to xcode 11.3.1 and 10.15 SDK
This gets us a newer SDK with c++17 support and retains 10.12
back-compat.

Co-authored-by: Carl Dong <contact@carldong.me>
2020-06-22 10:14:33 -04:00
Carl Dong
3381e4a189
Adapt rest of tooling to new SDK naming scheme 2020-06-22 10:13:55 -04:00
Carl Dong
bfe1ba2f5b
rel-builds: Specify core.abbrev for git-rev-parse
Chose 12 because the kernel uses it:

  https://public-inbox.org/git/CA+55aFy0_pwtFOYS1Tmnxipw9ZkRNCQHmoYyegO00pjMiZQfbg@mail.gmail.com/raw

And also because it's a nice number.
2020-05-04 13:00:08 -04:00
Carl Dong
d256f91cb1
rel-builds: Directly deploy win installer to OUTDIR 2020-05-01 12:30:20 -04:00
Carl Dong
395c1137f6
gitian: Limit sourced script to just assignments
Previously, the sourced script would create the source tarball. Now, it
only assigns variables and the source-ing script has more flexibility in
determining what to do with these variables.

See later commit showing how this flexibility is useful in our Guix
builds.
2020-04-28 10:36:35 -04:00
fanquake
65fb3dfc8d
Merge #18556: build: Drop make dist in gitian builds
2aa48edec0 refactor: Drop unused ${WRAP_DIR}/${HOST} directory (Hennadii Stepanov)
1362be0447 build: Drop make dist in gitian builds (Hennadii Stepanov)

Pull request description:

  After the merge of #18331, the packaged source tarball is created by `git archive`, but the binaries are built from another one which is made by `make dist`.

  With this PR the only source tarball, created by `git archive`, is used both for binaries building and for packaging to users.

  Close #16588.
  Close #18547.

  As a good side-effect, #18349 becomes redundant.

  **Change in behavior**

  The following variables 1b151e3ffc/configure.ac (L2-L6)

  are no longer used for naming of directories and tarballs.

  Instead of them the gitian descriptors use a git tag (if available) or a commit hash.

  ---

  Also a small refactor commit picked from #18404.

ACKs for top commit:
  dongcarl:
    ACK 2aa48edec0
  MarcoFalke:
    ACK 2aa48edec0
  fanquake:
    ACK 2aa48edec0 - I've had a quick look over this, and don't want to block merging if this actually gets as closer to finally having this all sorted out. Obviously we've still got #18741, and after speaking to Carl this morning, there will likely be even more changes after that (not Guix specific).

Tree-SHA512: d3b16f87e48d1790a3264940c28acd5d881bfd10f3ce94fb0c8a6af76d8039289d01e0cd4972adac49ae24362857251f6c1e5e09e3e9fbf636c10708b4015a7c
2020-04-28 16:44:17 +08:00
Hennadii Stepanov
2aa48edec0
refactor: Drop unused ${WRAP_DIR}/${HOST} directory
This commit removes the directory that is no longer used since #16667.
2020-04-17 16:09:11 +03:00
Hennadii Stepanov
1362be0447
build: Drop make dist in gitian builds 2020-04-17 16:09:04 +03:00
Wladimir J. van der Laan
8f2497941e
Merge #18598: gitian: Add missing automake package to gitian-win-signer.yml
e44aeefaae gitian: Add missing automake package to gitian-win-signer.yml (Andrew Chow)

Pull request description:

  automake is needed to build osslsigncode otherwise autogen.sh fails with the docker virtualization method.

ACKs for top commit:
  hebasto:
    ACK e44aeefaae, for `osslsigncode-1.7.1` we did not run `autogen.sh` in the past.
  fanquake:
    ACK e44aeefaae
  jonatack:
    ACK e44aeef

Tree-SHA512: a0e615c1b099ee1c469ce41f886f2ece6746234a5a800743a4e8be671e4114fd30e1c35bc0ddcb75778409564129d0fde7ac4e3d70b0f7691f97f729f34c8e0c
2020-04-16 22:27:46 +02:00
fanquake
b8b050a8d6
build: add linker optimization flags to gitian descriptors
Any -O argument will enable optimizations in GNU ld. We can use -O2
here, as this matches our compile flags. Note that this would also
enable additional optimizations if using the lld or gold linkers,
when compared to -O0.
2020-04-12 18:36:56 +08:00
Andrew Chow
e44aeefaae gitian: Add missing automake package to gitian-win-signer.yml
automake is needed to build osslsigncode otherwise autogen.sh fails.
2020-04-11 14:15:05 -04:00
Wladimir J. van der Laan
dabe2bb11a build: Bump gitian descriptors to 0.21
Per the release process.
2020-04-10 19:46:39 +02:00
Wladimir J. van der Laan
3e50fdbe4e
Merge #18395: scripts: add PE dylib checking to symbol-check.py
1a0993ae35 scripts: add PE dylib checking to symbol-check.py (fanquake)

Pull request description:

  Uses `objdump -x` and looks for `DLL Name:` lines. i.e:
  ```bash
  objdump -x src/qt/bitcoin-qt.exe | grep "DLL Name:"
  	DLL Name: ADVAPI32.dll
  	DLL Name: dwmapi.dll
  	DLL Name: GDI32.dll
  	DLL Name: IMM32.dll
  	DLL Name: IPHLPAPI.DLL
  	DLL Name: KERNEL32.dll
  	DLL Name: msvcrt.dll
  	DLL Name: ole32.dll
  	DLL Name: OLEAUT32.dll
  	DLL Name: SHELL32.dll
  	DLL Name: SHLWAPI.dll
  	DLL Name: USER32.dll
  	DLL Name: UxTheme.dll
  	DLL Name: VERSION.dll
  	DLL Name: WINMM.dll
  	DLL Name: WS2_32.dll
  ```

ACKs for top commit:
  dongcarl:
    Concept ACK 1a0993ae35
  hebasto:
    ACK 1a0993ae35, tested on Linux Mint 19.3:

Tree-SHA512: 0099a50e2c616d5239a15cafa9a7c483e9c40244af41549e4738be0f5360f27a2afb956eb50b47cf446b242f4cfc6dc9d111306a056fb83789eefbd71eddabd2
2020-03-25 15:35:36 +01:00
fanquake
1a0993ae35
scripts: add PE dylib checking to symbol-check.py 2020-03-22 10:47:38 +08:00
Hennadii Stepanov
e4d366788b
build: Drop needless EXTRA_DIST content
Some EXTRA_DIST content is needless since a git archive is used as the
source tarball.
2020-03-15 18:34:27 +02:00
Hennadii Stepanov
6c4da59f5b
build: Drop SOURCEDIST reordering
Making SOURCEDIST deterministic is needless since a git archive is used
as the source tarball.
2020-03-12 11:43:16 +02:00
Hennadii Stepanov
5e6b8b3912
build: Use git archive as source tarball 2020-03-12 11:34:43 +02:00
fanquake
530d02addb
build: pass -fno-ident in Windows gitian descriptor
This prevents compilers from emitting compiler name and
version number info that can needlessly bloat binaries.

Accepted by Clang and GCC. See:

https://clang.llvm.org/docs/ClangCommandLineReference.html#cmdoption-clang-qn

https://gcc.gnu.org/onlinedocs/gcc/Code-Gen-Options.html#index-fno-ident
2020-02-13 18:20:43 +08:00
fanquake
98264e2ccb
Merge #18104: build: Skip i686 build by default in guix and gitian
fae9084ac5 build: Skip i686 build by default in guix and gitian (MarcoFalke)
fa55a2554c depends: Remove reference to win32 (MarcoFalke)

Pull request description:

  Closes #17504

  Now that we no longer provide downloads for i686 on our website (https://bitcoincore.org/en/download/), there is no need to build them by default.

  i686 can still be built in depends (tested by ci/travis) and in guix/gitian by setting the appropriate `HOSTS`.

ACKs for top commit:
  practicalswift:
    ACK fae9084ac5 -- patch looks correct
  dongcarl:
    ACK fae9084ac5 patch looks correct
  laanwj:
    Code review ACK fae9084ac5
  hebasto:
    ACK fae9084ac5, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: b000c19a2cd2a596a52028fa298c4022c24cfdfc1bdb3795a90916d0a00a32e4dd22278db93790b6a11724e08ea8451f4f05c77bc40d1664518e11a8c82d6e29
2020-02-11 16:32:17 +08:00
MarcoFalke
fae9084ac5
build: Skip i686 build by default in guix and gitian 2020-02-09 13:22:53 -08:00
fanquake
7e2104433c
build: use macOS 10.14 SDK
Co-Authored-By: Carl Dong <accounts@carldong.me>
2020-02-03 19:49:46 +08:00
Wladimir J. van der Laan
a3d198c93c
Merge #17863: scripts: Add MACHO dylib checks to symbol-check.py
c491368d8c scripts: add MACHO dylib checking to symbol-check.py (fanquake)
76bf97213f scripts: fix check-symbols & check-security argument passing (fanquake)

Pull request description:

  Based on #17857.

  This adds dynamic library checks for MACHO executables to symbol-check.py. The script has been modified to function more like `security-check.py`. The error output is now also slightly different. i.e:
  ```bash
  # Linux x86
  bitcoin-cli: symbol operator new[](unsigned long) from unsupported version GLIBCXX_3.4
  bitcoin-cli: export of symbol vtable for std::basic_ios<char, std::char_traits<char> > not allowed
  bitcoin-cli: NEEDED library libstdc++.so.6 is not allowed
  bitcoin-cli: failed IMPORTED_SYMBOLS EXPORTED_SYMBOLS LIBRARY_DEPENDENCIES

  # RISCV (skips exported symbols checks)
  bitcoin-tx: symbol operator new[](unsigned long) from unsupported version GLIBCXX_3.4
  bitcoin-tx: NEEDED library libstdc++.so.6 is not allowed
  bitcoin-tx: failed IMPORTED_SYMBOLS LIBRARY_DEPENDENCIES

  # macOS
  Checking macOS dynamic libraries...
  libboost_filesystem.dylib is not in ALLOWED_LIBRARIES!
  bitcoind: failed DYNAMIC_LIBRARIES
  ```

  Compared to `v0.19.0.1` the macOS allowed dylibs has been slimmed down somewhat:
  ```diff
   src/qt/bitcoin-qt:
   /usr/lib/libSystem.B.dylib
  -/System/Library/Frameworks/DiskArbitration.framework/Versions/A/DiskArbitration
   /System/Library/Frameworks/IOKit.framework/Versions/A/IOKit
   /System/Library/Frameworks/Foundation.framework/Versions/C/Foundation
   /System/Library/Frameworks/CoreServices.framework/Versions/A/CoreServices
   /System/Library/Frameworks/AppKit.framework/Versions/C/AppKit
   /System/Library/Frameworks/ApplicationServices.framework/Versions/A/ApplicationServices
   /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation
  -/System/Library/Frameworks/Security.framework/Versions/A/Security
  -/System/Library/Frameworks/SystemConfiguration.framework/Versions/A/SystemConfiguration
   /System/Library/Frameworks/CoreGraphics.framework/Versions/A/CoreGraphics
  -/System/Library/Frameworks/OpenGL.framework/Versions/A/OpenGL
  -/System/Library/Frameworks/AGL.framework/Versions/A/AGL
   /System/Library/Frameworks/Carbon.framework/Versions/A/Carbon
   /usr/lib/libc++.1.dylib
  -/System/Library/Frameworks/CFNetwork.framework/Versions/A/CFNetwork
   /System/Library/Frameworks/CoreText.framework/Versions/A/CoreText
   /System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO
   /usr/lib/libobjc.A.dylib
  ```

ACKs for top commit:
  laanwj:
    ACK c491368d8c

Tree-SHA512: f8624e4964e80b3e0d34e8d3cc33f3107938f3ef7a01c07828f09b902b5ea31a53c50f9be03576e1896ed832cf2c399e03a7943a4f537a1e1c705f3804aed979
2020-01-22 20:33:44 +01:00
MarcoFalke
0a8b68cdf7
Merge #17483: build: Set gitian arch back to amd64
fae75306ba scripted-diff: Set gitian arch back to amd64 (MarcoFalke)

Pull request description:

  This was required to allow gitian builds on non-amd64 architecture, however, it seems to break the current builds (with lxc), see https://github.com/bitcoin/bitcoin/pull/17409#issuecomment-554099626

  Also, the gititan builds wouldn't be deterministic across arches anyway, see #17468

  So instead of wasting more time on this, revert the change and hope that guix allows to compile on non-amd64 architectures.

Top commit has no ACKs.

Tree-SHA512: 801e9a30ae1b0882ef45d5eb3a3cf80f3ace3b99db046069dbd95b6162119e977e3cf3134287d1ac5d09483906206acc71e1ac34d6b74dbc533d46aaf73f5cc2
2020-01-22 09:50:26 -05:00
fanquake
c491368d8c
scripts: add MACHO dylib checking to symbol-check.py 2020-01-04 11:24:42 +08:00
fanquake
4ca92dc6d3
scripts: add MACHO PIE check to security-check.py 2020-01-02 14:42:21 +08:00
willyk
c966ff14c7
gitian: fixed SC2001 regex
the `-` is not a special symbol and should not have `%` in front of it.
2019-12-05 13:00:34 -08:00
Wladimir J. van der Laan
1f59885d27
Merge #17361: script: Lint Gitian descriptors with ShellCheck
17f81e9648 script: Enable SC2001 rule for Gitian scripts (Hennadii Stepanov)
61bb21b418 script: Enable SC2155 rule for Gitian scripts (Hennadii Stepanov)
577682d9e8 script: Enable SC2006 rule for Gitian scripts (Hennadii Stepanov)
14aded46df script: Lint Gitian descriptors with ShellCheck (Hennadii Stepanov)

Pull request description:

  This PR extracts shell scripts from Gitian descriptors (`contrib/gitian-descriptors/`) and checks for ShellCheck warnings as any other one.

  Some non-controversial warnings are fixed.

ACKs for top commit:
  practicalswift:
    ACK 17f81e9648 -- diff looks correct

Tree-SHA512: bdfa3d35bbb65ff634c90835d75c3df63e958b558599771d21366724f5cf64da83a68957d926e926a99c3704b9529e96a17697dc8d9ff3adf7154d9cb1999a8d
2019-11-28 10:18:22 +01:00
Hennadii Stepanov
17f81e9648
script: Enable SC2001 rule for Gitian scripts 2019-11-27 19:27:56 +02:00
Hennadii Stepanov
61bb21b418
script: Enable SC2155 rule for Gitian scripts
Also pwd command is replaced with $PWD variable everywhere for
consistency.
2019-11-27 19:25:29 +02:00
Wladimir J. van der Laan
eafd259367 build: Add NX workaround for RV64
Work around https://bugs.launchpad.net/ubuntu/+source/gcc-8-cross-ports/+bug/1853740.
2019-11-24 13:40:11 +01:00
fanquake
7d7bf2ff4a
build: set minimum supported macOS to 10.12 2019-11-21 11:08:47 -05:00