c799a19b4b build, qt: No longer need to set QT_RCC_TEST=1 for determinism (Hennadii Stepanov)
Pull request description:
The Qt Resource Compiler (rcc) output order relies on [`QHash`](https://doc.qt.io/qt-5/qhash.html):
> This randomization of `QHash` is enabled by default. Even though programs should never depend on a particular `QHash` ordering, there may be situations where you temporarily need deterministic behavior, for example for debugging or regression testing. To disable the randomization, define the environment variable `QT_HASH_SEED` to have the value 0.
Since #3620 we use `QT_RCC_TEST=1` to achieve a deterministic output.
Since Qt 5.3.1 hash seeding is disabled for all of the bootstrapped tools, including rcc. Therefore, `QT_RCC_TEST=1` is no longer needed.
See commit [5283a6c87beac5a43f612786fefd6e43f2c70bf6](5283a6c87b).
ACKs for top commit:
fanquake:
ACK c799a19b4b
Tree-SHA512: 9d116ac1e8c605ee3e8ed7f618586f0de85d8b06bbbb70fe8c298939ce203d2a7e97264a9afac037179993ab54c5f69a65ebb9ab27ca7f45acb963011bd45743
We now use a script named gen_id to generate the base build_id/host_id.
This solves 2 problems:
1. GNU Make special-casing exit code 127 (command not found) meant that
warnings about missing tools would propagate to the user's terminal
and broke our opportunistic build_id construction.
2. This change ensures that we don't have arbitrary characters in our
make variables that would be misinterpreted by Make.
See comments in depends/Makefile and depends/gen_id for more
information.
Currently, if you build depends using `NO_BDB=1` (only sqlite wallets),
./configure will fail as it still tries to find bdb. i.e:
```bash
checking for Berkeley DB C++ headers... default
configure: error: Found Berkeley DB other than 4.8, required for portable BDB wallets (--with-incompatible-bdb to ignore or --without-bdb to disable BDB wallet support)
```
This PR fixes the build such that you can build depends, opting out of
bdb without opting out of wallets entirely, and still configure
successfully.
7476b46f18 guix: Build dmg as a static binary (Carl Dong)
06d6cf6784 depends: libdmg-hfsplus: Skip CMake RPATH patching (Carl Dong)
65176ab573 guix: Remove codesign_allocate+pagestuff from unsigned tarball (Carl Dong)
ca85679eb4 guix: Use clang-toolchain instead of clang (Carl Dong)
1aec0eda8f guix: Fallback to local build for substitute-enabled Guix users (Carl Dong)
1742f8e12d guix: Add early health check for guix-daemon (Carl Dong)
c1ae726a13 guix: More thoroughly control native toolchain (Carl Dong)
39741128d3 guix: Supply --link-profile (Carl Dong)
d55a1056ee guix: Add troubleshooting documentation entries (Carl Dong)
7f401c953f guix: Adapt guix-build to prelude, restructure hier (Carl Dong)
4eccf063b2 guix: Remove guix-build.sh filename extension (Carl Dong)
7753357a7b guix: Add source-able bash prelude and utils (Carl Dong)
e5b49a01f5 guix: Create windeploy inside distsrc-* (Carl Dong)
3e9982ab38 contrib: Silence git-describe when looking for tag (Carl Dong)
d5a71e9785 guix: Use --cores instead of --max-jobs (Carl Dong)
Pull request description:
This PR addresses a few hiccups encountered by the brave souls who've been experimenting with the Guix scripts:
- Resolves confusion between `--cores=` and `--max-jobs=`
- `guix`'s `--cores=` actually corresponds to make's `--jobs=`, so let's just control `--cores=` with our overridable env var
- `git-describe` will scream `fatal: no tag exactly matches '<hash>'` when looking for a tag, but we don't care, so silence that
- `windeploy/unsigned` should be inside `distsrc-*` and created idempotently (sorry I know this one annoyed people)
- Add troubleshooting documentation to `README.md`
- Add early health check for `guix-daemon` in case user forgot to start a `guix-daemon`
- Depending on configuration, a `--fallback` flag may be needed to tell Guix to not fail if substitutes fail but fallback to building locally
- `codesign_allocate` and `pagestuff` are now unnecessary for codesigning as we're now using `signapple`
A few robustness changes are also included:
- We supply the `--link-profile` flag, as some Guix packages may expect the profile to be available under `$HOME/.guix-profile`
- We now clear and manually set all toolchain-related env vars (e.g. `C*_INCLUDE_PATH`) ourselves, after patching a Qt::moc bug
- We use the native `clang-toolchain` package for darwin builds instead of `clang`, lining up with all our other toolchain packages.
Finally, we restructure the guix building hierarchy such that it looks something like:
```
guix-build-<short-hash-or-version-tag>
├── distsrc-<short-hash-or-version-tag>-${HOST}
│ ├── contrib
│ ├── depends
│ ├── src
│ └── ...
├── distsrc-<short-hash-or-version-tag>-...
└── output
├── dist-archive
│ └── bitcoin-<short-hash-or-version-tag>.tar.gz
├── *-linux-*
│ ├── bitcoin-<short-hash-or-version-tag>-*-linux-*-debug.tar.gz
│ └── bitcoin-<short-hash-or-version-tag>-*-linux-*.tar.gz
├── x86_64-apple-darwin18
│ ├── bitcoin-<short-hash-or-version-tag>-osx64.tar.gz
│ ├── bitcoin-<short-hash-or-version-tag>-osx-unsigned.dmg
│ └── bitcoin-<short-hash-or-version-tag>-osx-unsigned.tar.gz
└── x86_64-w64-mingw32
├── bitcoin-<short-hash-or-version-tag>-win64-debug.zip
├── bitcoin-<short-hash-or-version-tag>-win64-setup-unsigned.exe
├── bitcoin-<short-hash-or-version-tag>-win64.zip
└── bitcoin-<short-hash-or-version-tag>-win-unsigned.tar.gz
```
Separating guix builds by their version identifier (basically namespacing them) allows us to change the layout in the future without worry about potential naming conflicts.
ACKs for top commit:
sipa:
ACK 7476b46f18
laanwj:
ACK 7476b46f18
Tree-SHA512: 0e899aa941aafdf552b2a7e8a08131ee9283180bbef7334439e2461a02aa7235ab7b9ca9c149b80fc5d0a9f4bbd35bc80fcee26197c0836ba8eaf2d86ffa0386
This relatively easy change eliminates all runtime dependencies (except
for the kernel) for dmg, which is the only native build tool that gets
put in our output tarballs.
This allows much more flexibility when constructing the codesigning
environment, and is much more robust.
After reading
https://www.gnu.org/savannah-checkouts/gnu/autoconf/manual/autoconf-2.70/autoconf.html#Specifying-Target-Triplets,
my understanding is that this change should mostly be a no-op, as
--build defaults to the output of config.guess, however, this may be
slightly more correct
> For historical reasons, whenever you specify --host, be sure to
> specify --build too; this will be fixed in the future.
and will quell some warnings in depends (#16354). If anything, this
also explicitly enables cross-compilation mode when `--host` differs
from `--build`.
As for "fixed in the future", this is the case for Autoconf 2.70+.
246774e264 depends: fix Qt precompiled headers bug (Igor Cota)
8e7ad4146d depends: disable Qt Vulkan support on Android (Igor Cota)
ba46adaa1a CI: add Android APK build to cirrus (Igor Cota)
7563720e30 CI: add Android APK build script (Igor Cota)
ebfb10cb75 Qt: add Android packaging support (Igor Cota)
Pull request description:
This PR is the third and final piece of the basic Android support puzzle - it depends on https://github.com/bitcoin/bitcoin/pull/16110 and is related to https://github.com/bitcoin/bitcoin/pull/16883. It introduces an `android` directory under `qt` and a simple way to build an Android package of `bitcoin-qt`:
1. Build depends for Android as described in the [README](https://github.com/bitcoin/bitcoin/blob/master/depends/README.md)
2. Configure with one of the resulting prefixes
3. Run `make && make apk` in `src/qt`
The resulting APK files will be in `android/build/outputs/apk`. You can install them manually or with [adb](https://developer.android.com/studio/command-line/adb). One can also open the `android` directory in Android Studio for that integrated development and debugging experience. `BitcoinQtActivity` is your starting point.
Under the hood makefile `apk` target:
1. Renames the `bitcoin-qt` binary to `libbitcoin-qt.so` and copies it over to a folder under `android/libs` depending on which prefix and corresponding [ABI](https://developer.android.com/ndk/guides/abis.html#sa) `bitcoin-qt` was built for
2. Takes `libc++_shared.so` from the Android NDK and puts in the same place. It [must be included](https://developer.android.com/ndk/guides/cpp-support) in the APK
3. Extracts Qt for Android Java support files from the `qtbase` archive in `depends/sources` to `android/src`
There is also just a tiny bit of `ifdef`'d code to make the Qt Widgets menus usable. It's not pretty but it works and is a stepping stone towards https://github.com/bitcoin/bitcoin/pull/16883.
ACKs for top commit:
MarcoFalke:
cr ACK 246774e264
laanwj:
Code review ACK 246774e264
Tree-SHA512: ba30a746576a167545223c35a51ae60bb0838818779fc152c210f5af1413961b2a6ab6af520ff92cbc8dcd5dcb663e81ca960f021218430c1f76397ed4cead6c
180dc3c886 build: miniupnpc 2.2.2 (fanquake)
Pull request description:
Creating the dll subdir is no-longer required.
We can drop our wingen patch.
One issue that came up in [#19867](https://github.com/bitcoin/bitcoin/pull/19867#discussion_r483516359):
> unrelated to this change but: why are we inserting the architecture in here, seems like something not necessary to reveal
> My assumption is that it was being inserted to make depends more deterministic. However I think we can improve this, as there's no reason to reveal more of the version information either. Could leave the version as is /2.0 and either drop the architecture, or insert something else?
I've dropped our `sed` and added a patch that just removes the OS string and miniupnpc version from the User-Agent. i.e:
```bash
# master
strings depends/x86_64-apple-darwin19.6.0/lib/libminiupnpc.a | rg -i User-Agent
User-Agent: x86_64-apple-darwin19.6.0, UPnP/1.1, MiniUPnPc/2.0.20180203
User-Agent: x86_64-apple-darwin19.6.0, UPnP/1.1, MiniUPnPc/2.0.20180203
# this PR
strings depends/x86_64-apple-darwin19.6.0/lib/libminiupnpc.a | rg -i User-Agent
User-Agent: UPnP/1.1
User-Agent: UPnP/1.1
```
Note that built unmodified (22c1386351), the User-Agent would be:
```bash
strings libminiupnpc.dylib | rg User-Agent
User-Agent: Darwin/19.6.0, UPnP/1.1, MiniUPnPc/2.2.0
User-Agent: Darwin/19.6.0, UPnP/1.1, MiniUPnPc/2.2.0
```
ACKs for top commit:
laanwj:
Code review ACK 180dc3c886
hebasto:
ACK 180dc3c886.
Tree-SHA512: b0b6e623dbc5499e28faedf992d84278d6a11887a45a3806957b9e08886c5e56044cdfa2e7d7ec81cb1dd55f89be99834367905315d6bc611ba530e91d889ad1
See:
- QTBUG-72903, commit 9864d2c6f3b628ca9f07a56b197e77bd43931cca
- QTBUG-78873, commit e55a61a77f0c87c05661a0335dfdb12673c6a27f
Could be dropped for Qt 5.14+.
This tries to invoke xcrun, which is not available when cross-compiling.
Given we are in control of the SDK versions being used, removing this
check has minimal-no effect.
remove fix_configure_mac.patch
Fixed upstream: https://bugreports.qt.io/browse/QTBUG-67286
remove fix_riscv64_arch.patch
Was fixed upstream in 6a39e49a6cdeb28a04a3657bb6a22f848d5dfa9d
remove fix_rcc_determinism.patch
Fixed upstream in https://bugreports.qt.io/browse/QTBUG-62511
remove freetype_back_compat.patch
By the time we ship a release with Qt 5.12, we'll certainly no-longer be
supporting Ubuntu 14.04 and Ubuntu 16.04 ships with FreeType 2.6.1,
which is new enough that using the symbol is no-longer an issue.
The renaming of FT_Get_X11_Font_Format() happened in FreeType 2.6
remove xkb-default.patch
This was removed upstream in d5abf545971da717014d316127045fc19edbcd65
Co-authored-by: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
A fix for QTBUG-34748 was introduced in #5915 (v0.11.0, Qt 5.2.1).
QTBUG-34748 was fixed in version 5.3.0.
The separated patch file, provided by #5915, was dropped in #12971 while
bumping Qt to 5.9.4 (5.9.6). But libxcb.mk remained unchanged.
This change reverts #5915 for libxcb.mk.
7af25024e9 build: compile libnatpmp with -DNATPMP_STATICLIB on Windows (fanquake)
ee35745754 build: use newer source for libnatpmp (fanquake)
Pull request description:
The source we are currently using is from 2015. The upstream repo has
received a small number of bug fixes and improvements since then.
Including one that fixes an issue for Windows users: https://github.com/miniupnp/libnatpmp/pull/13.
The source we are currently using is the most recent "official" release,
however I don't think it's worth waiting for a new one. The maintainer
was prompted to do so in Oct 2020, then again in Jan of this year, and
no release has eventuated. Given libnatpmp is a new inclusion into our
repository, I think we should be using this newer source.
This also cleans up a few warnings we currently see in Windows depends builds:
```bash
Extracting libnatpmp...
/home/ubuntu/bitcoin/depends/sources/libnatpmp-20150609.tar.gz: OK
Preprocessing libnatpmp...
Configuring libnatpmp...
Building libnatpmp...
make[1]: Entering directory '/home/ubuntu/bitcoin/depends/work/build/x86_64-w64-mingw32/libnatpmp/20150609-13efa1beb87'
x86_64-w64-mingw32-gcc -Os -fPIC -Wall -DENABLE_STRNATPMPERR -c -o natpmp.o natpmp.c
x86_64-w64-mingw32-gcc -Os -fPIC -Wall -DENABLE_STRNATPMPERR -c -o getgateway.o getgateway.c
natpmp.c:42: warning: "EWOULDBLOCK" redefined
42 | #define EWOULDBLOCK WSAEWOULDBLOCK
|
In file included from natpmp.c:38:
/usr/share/mingw-w64/include/errno.h:166: note: this is the location of the previous definition
166 | #define EWOULDBLOCK 140
|
natpmp.c:43: warning: "ECONNREFUSED" redefined
43 | #define ECONNREFUSED WSAECONNREFUSED
|
In file included from natpmp.c:38:
/usr/share/mingw-w64/include/errno.h:110: note: this is the location of the previous definition
110 | #define ECONNREFUSED 107
|
natpmp.c:271:5: warning: ‘readnatpmpresponseorretry’ redeclared without dllimport attribute: previous dllimport ignored [-Wattributes]
271 | int readnatpmpresponseorretry(natpmp_t * p, natpmpresp_t * response)
| ^~~~~~~~~~~~~~~~~~~~~~~~~
ar crs libnatpmp.a natpmp.o getgateway.o
make[1]: Leaving directory '/home/ubuntu/bitcoin/depends/work/build/x86_64-w64-mingw32/libnatpmp/20150609-13efa1beb87'
Staging libnatpmp...
Postprocessing libnatpmp...
Caching libnatpmp...
```
ACKs for top commit:
hebasto:
ACK 7af25024e9
Tree-SHA512: 6939014ea986149a5bfdd42b516d563a65ae643516e234579d3f28e7c2f877b0270cc4305ae7c7cb131d6d946a6e0aedc84b4cc880a412612a878a333398b9d7
The source we are currently using is from 2015. The upstream repo has
received a small number of bug fixes and improvements since then.
Including one that fixes an issue for Windows users:
https://github.com/miniupnp/libnatpmp/pull/13.
The source we are currently using is the most recent "official" release,
however I don't think it's worth waiting for a new one. The maintainer
was prompted to do so in Oct 2020, then again in Jan of this year, and
no release has eventuated. Given libnatpmp is a new inclusion into our
repository, I think we should be using this newer source.
This also cleans up a few warnings we currently see in depends builds:
```bash
Extracting libnatpmp...
/home/ubuntu/bitcoin/depends/sources/libnatpmp-20150609.tar.gz: OK
Preprocessing libnatpmp...
Configuring libnatpmp...
Building libnatpmp...
make[1]: Entering directory '/home/ubuntu/bitcoin/depends/work/build/x86_64-w64-mingw32/libnatpmp/20150609-13efa1beb87'
x86_64-w64-mingw32-gcc -Os -fPIC -Wall -DENABLE_STRNATPMPERR -c -o natpmp.o natpmp.c
x86_64-w64-mingw32-gcc -Os -fPIC -Wall -DENABLE_STRNATPMPERR -c -o getgateway.o getgateway.c
natpmp.c:42: warning: "EWOULDBLOCK" redefined
42 | #define EWOULDBLOCK WSAEWOULDBLOCK
|
In file included from natpmp.c:38:
/usr/share/mingw-w64/include/errno.h:166: note: this is the location of the previous definition
166 | #define EWOULDBLOCK 140
|
natpmp.c:43: warning: "ECONNREFUSED" redefined
43 | #define ECONNREFUSED WSAECONNREFUSED
|
In file included from natpmp.c:38:
/usr/share/mingw-w64/include/errno.h:110: note: this is the location of the previous definition
110 | #define ECONNREFUSED 107
|
natpmp.c:271:5: warning: ‘readnatpmpresponseorretry’ redeclared without dllimport attribute: previous dllimport ignored [-Wattributes]
271 | int readnatpmpresponseorretry(natpmp_t * p, natpmpresp_t * response)
| ^~~~~~~~~~~~~~~~~~~~~~~~~
ar crs libnatpmp.a natpmp.o getgateway.o
make[1]: Leaving directory '/home/ubuntu/bitcoin/depends/work/build/x86_64-w64-mingw32/libnatpmp/20150609-13efa1beb87'
Staging libnatpmp...
Postprocessing libnatpmp...
Caching libnatpmp...
```
bd49ac4168 build: fix libnatpmp macos cross compile (fanquake)
Pull request description:
Currently, our cross-compile of libnatpmp for macOS doesn't work at all.
The wrong archiver is used, which produces an archive the linker doesn't like.
This becomes clear when configuring:
```bash
configure:25722: checking for initnatpmp in -lnatpmp
configure:25747: env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH -u LIBRARY_PATH /home/ubuntu/bitcoin/depends/x86_64-apple-darwin18/native/bin/clang++ --target=x86_64-apple-darwin18 <trim> -Wl,-headerpad_max_install_names -Wl,-dead_strip -Wl,-dead_strip_dylibs conftest.cpp -lnatpmp >&5
ld: archive has no table of contents for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
```
Fix this by using the right `ar` (we do the same for upnp).
While we're at it, fix the build so that we are using our c/ppflags.
In practice this basically means building with `-O2` rather than `-Os`.
Note that this fixes an issue that is also fixed by #21209. However, given there are reservations about updating to use a newer libnatpmp source, we should just fix this for now.
ACKs for top commit:
hebasto:
ACK bd49ac4168, tested:
Tree-SHA512: 2efc2c788ef3ebebfbf564ef07b6cf63a72d8a0bccc22b0ba36537216aa575436b7e87088477e85f6d9191ad34f0b13f1c22cf88c90e1cb81641bfee5dc3058a
76f52e3da3 build: Fix Qt processing of configure script for depends with DEBUG=1 (Hennadii Stepanov)
Pull request description:
This PR:
- makes the `configure` script correctly pickup Qt if depends is built with `DEBUG=1`:
- for Windows -- fix#19266
- for macOS -- fix#16391
- is an alternative to #18117 (without downsides)
ACKs for top commit:
fanquake:
ACK 76f52e3da3. Tested native darwin, and darwin/win cross compile with `DEBUG=1`.
Tree-SHA512: 8fde99302b4b06faf109315bddba9e3063b156c50f8f9863c2bd51718538c719429a63fdced071730c18022f2e559d3b25c1dcec3efa81fe79f657253680956a
Currently, our cross-compile of libnatpmp for macOS doesn't work at all.
The wrong archiver is used, which produces an archive the linker doesn't like.
This becomes clear when configuring:
```bash
configure:25722: checking for initnatpmp in -lnatpmp
configure:25747: env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH -u LIBRARY_PATH /home/ubuntu/bitcoin/depends/x86_64-apple-darwin18/native/bin/clang++ --target=x86_64-apple-darwin18 <trim> -Wl,-headerpad_max_install_names -Wl,-dead_strip -Wl,-dead_strip_dylibs conftest.cpp -lnatpmp >&5
ld: archive has no table of contents for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
```
Fix this by using the right `ar` (we do the same for upnp).
While we're at it, we fixe the build so that we are using our c/ppflags.
This means building with `-O2` rather than `-Os`.
Note that this fixes an issue that is also fixed by #21209.
However, given there are reservations about updating to use a newer libnatpmp source, we should just fix this for now.
5200929bfe depends: Include GUIX_ENVIRONMENT in id string (Carl Dong)
4c7d418588 depends: Improve id string robustness (Carl Dong)
b3bdff42b5 build: Proper quoting for var printing targets (Carl Dong)
Pull request description:
```
Environment variables and search paths can drastically effect the
operation of build tools.
Include these in our id string to mitigate against false cache hits.
```
Note to builders: This will invalidate all depends output caches in `BASE_CACHE`
ACKs for top commit:
laanwj:
re-ACK 5200929bfe
Tree-SHA512: e70c98da89cde90dc54bc3be89b925787cf94bbf246e27cc9345816b312073d78a02215448f731f21d8cf033c455234a2377ff1d66c00e1f3db69c9c9687d027
060a2a64d4 ci: remove boost thread installation (fanquake)
06e1d7d81d build: don't build or use Boost Thread (fanquake)
7097add83c refactor: replace Boost shared_mutex with std shared_mutex in sigcache (fanquake)
8e55981ef8 refactor: replace Boost shared_mutex with std shared_mutex in cuckoocache tests (fanquake)
Pull request description:
This replaces `boost::shared_mutex` and `boost::unique_lock` with [`std::shared_mutex`](https://en.cppreference.com/w/cpp/thread/shared_mutex) & [`std::unique_lock`](https://en.cppreference.com/w/cpp/thread/unique_lock).
Even though [some concerns were raised](https://github.com/bitcoin/bitcoin/issues/16684#issuecomment-726214696) in #16684 with regard to `std::shared_mutex` being unsafe to use across some glibc versions, I still think this change is an improvement. As I mentioned in #21022, I also think trying to restrict standard library feature usage based on bugs in glibc is not only hard to do, but it's not currently clear exactly how we do that in practice (does it also extend to patching out use in our dependencies, should we be implementing more runtime checks for features we are using, when do we consider an affected glibc "old enough" not to worry about? etc). If you take a look through the [glibc bug tracker](https://sourceware.org/bugzilla/describecomponents.cgi?product=glibc) you'll no doubt find plenty of (active) bug reports for standard library code we already using. Obviously not to say we shouldn't try and avoid buggy code where possible.
Two other points:
[Cory mentioned in #21022](https://github.com/bitcoin/bitcoin/pull/21022#issuecomment-769274179):
> It also seems reasonable to me to worry that boost hits the same underlying glibc bug, and we've just not happened to trigger the right conditions yet.
Moving away from Boost to the standard library also removes the potential for differences related to Boosts configuration. Boost has multiple versions of `shared_mutex`, and what you end up using, and what it's backed by depends on:
* The version of Boost.
* The platform you're building for.
* Which version of `BOOST_THREAD_VERSION` is defined: (2,3,4 or 5) default=2. (see [here](https://www.boost.org/doc/libs/1_70_0/doc/html/thread/build.html#thread.build.configuration) for some of the differences).
* Is `BOOST_THREAD_V2_SHARED_MUTEX` defined? (not by default). If so, you might get the ["less performant, but more robust"](https://github.com/boostorg/thread/issues/230#issuecomment-475937761) version of `shared_mutex`.
A lot of these factors are eliminated by our use of depends, but users will have varying configurations. It's also not inconceivable to think that a distro, or some package manager might start defining something like `BOOST_THREAD_VERSION=3`. Boost tried to change the default from 2 to 3 at one point.
With this change, we no longer use Boost Thread, so this PR also removes it from depends, the build system, CI etc.
Previous similar PRs were #19183 & #20922. The authors are included in the commits here.
Also related to #21022 - pthread sanity checking.
ACKs for top commit:
laanwj:
Code review ACK 060a2a64d4
vasild:
ACK 060a2a64d4
Tree-SHA512: 572d14d8c9de20bc434511f20d3f431836393ff915b2fe9de5a47a02dca76805ad5c3fc4cceecb4cd43f3ba939a0508178c4e60e62abdbaaa6b3e8db20b75b03
Environment variables and search paths can drastically effect the
operation of build tools.
Include these in our id string to mitigate against false cache hits.
This was missed in #20419, and the update before that, so just
make this un-versioned so that we don't have to worry about it.
This is fine, because it's just for downloading sources.
Instead of doing the awkward /bin path prepending at config.site
creation time, set darwin_{CC,CXX} in a way that fully determines the
program's path (clang/clang++) similar to how AC_PATH_{TOOL,PROG} would
do.
Also see the added comment block in depends/Makefile for more context on
determining $PATH for our config.site.
clang warns when a command line option is unused, and some of our tests
use Werror, so unfortunately we cannot use this flag to pin our linker
for now. Leaving this commit in for future reference, as it would be
great if there's more granularity to Werror and we can be explicit about
what linker we want to use.
Prior to this commit, when int_vars was called for packages, it would
immediately expand the "single-dollar variables", which may be defined
in terms of variables which are not yet determined (e.g. variables
defined in package/*.mk, which are included after int_vars is called).
This is required for the next commit as after that commit, for darwin
cross-builds:
0. int_vars is defined in terms of $(1)_cc
1. $(1)_cc is defined in terms of darwin_CC
2. ... which is defined in terms of clang_resource_dir
3. ... which is defined in terms of native_cctools_clang_version
4. which is undetermined at the time when int_vars is being expanded and evaluated
1112035d32 doc: fix various typos (Ikko Ashimine)
e8640849c7 doc: Use https URLs where possible (Sawyer Billings)
Pull request description:
Consolidates / fixes the changes from #20762, #20836, #20810. There is no output when `test/lint/lint-all.sh` is run.
Closes#20807.
ACKs for top commit:
MarcoFalke:
ACK 1112035d32
Tree-SHA512: 22ca824688758281a74e5ebc6a84a358142351434e34c88c6b36045d2d241ab95fd0958565fd2060f98317e62e683323b5320cc7ec13592bf340e6922294ed78
87fe104537 depends: Use more legible qmake commands in qt package (Hennadii Stepanov)
bf35a8da6e depends: Do not set build_subdir for qt package (Hennadii Stepanov)
Pull request description:
Rather than using `cd` to jump all over the place, perform all `(q)make` commands from the top level directory.
Looking at bash like `cd ../../../..` gives me a headache.
Credits to **fanquake**.
This PR is an alternative to #20504 that works without any additional [non-trivial hack](https://github.com/bitcoin/bitcoin/pull/20504#issuecomment-734730336).
ACKs for top commit:
promag:
Tested ACK 87fe104537.
fanquake:
ACK 87fe104537
Tree-SHA512: 1d2a13b5358fc7406c5363ddd62fd363dbc0ec5ace68946e4d3e6e8620419afaa64ef2837488aaed226174e01e8897495085540f7126b80f8b2372d21b5b29f9
a4118c6e20 Add patch to make codesign_allocate compatible with Apple's (Pieter Wuille)
Pull request description:
This is an alternative to #20638.
The problem is that Apple's codesign(_allocate) apparently rounds the "vmsize" attribute on the __LINKEDIT section to a multiple of 0x2000 on x86_64 rather than 0x1000 (as their published source code does). This divergence means that the binary signed by codesign is slightly different from the one recreated by our reattach-sig-to-gitian-output process, and the signature being invalid.
This fixes it by patching our codesign_allocate source code to also use 0x2000. In tests, this appears to result in matching binaries.
ACKs for top commit:
jonasschnelli:
Tested ACK a4118c6e20 - removed the osx cache, built commit a4118c6e20 for osx in gitian (dependency where built, patch was applied), signed on my signing mac (detach-sig-create), ran gitian osx signer with the produces signature and the a4118c6e20 build (detach-sig-apply), signature then was successful verified on my Mac (codesign -v /Volumes/Bitcoin-Core/Bitcoin-Qt.app)
MarcoFalke:
Concept ACK a4118c6e20
Tree-SHA512: 07b8cdf8216249ddfe4bd38b39f2b48b2e190d4002b84d8981e62197bbbc9f25ac5c137bcc32057b23fbf38cbb2889ef95101ce008edfbf608cd170b88b3acbc
7587d11ec9 build: remove cdrkit package from depends (fanquake)
0df9819126 build: Replace genisoimage with xorriso (fanquake)
22437fc72e build: Run libdmg-hfsplus's DMG tool in make deploy (Carl Dong)
Pull request description:
This is a redo of fanquake's https://github.com/bitcoin/bitcoin/pull/18151, which, aside from switching us from the deprecated `genisoimage` to the maintained `xorriso`, is also necessary for Guix to achieve determinism without using faketime.
> xorriso and its mkisofs/genisoimage emulation alter-ego xorrisofs are
> more maintained, and has the right toggles for us to achieve output
> determinism without using blunt tools like faketime.
>
> In this commit, we use xorrisofs from the build environment rather than
> building it ourselves using depends. This is not necessary and can be
> changed in the future.
>
> From wiki.debian.org/genisoimage?action=recall&rev=11 :
>
> > The classical command line interface for production of ISO 9660
> > filesystem images is the option set established by program mkisofs.
> > For reasons of licensing and other problems with its author, Debian
> > ships a fork of mkisofs, called genisoimage, which was split off in
> > 2006 and then developed independently.
> >
> > Meanwhile, genisoimage gets no new features and not even bug fixes. It
> > is first choice only if its options -udf or -hfs are needed.
> >
> > Replacement in most uses cases, especially for bootable ISO 9660
> > filesystems, archiving, and backup, is xorrisofs which starts the -as
> > mkisofs emulation mode of program xorriso.
ACKs for top commit:
laanwj:
ACK 7587d11ec9
Tree-SHA512: 62f3aad08fa8bf21192e951d7dd33b24975586d76834cfa3498f4b8cdb586cefec8cab2c073d1951a0884b5e182fd71ef2cf3accad98f84455016776ad3c5422
This change gets rid of multiple `../` that makes reasoning about the
script and its maintaining much easier.
Co-authored-by: fanquake <fanquake@gmail.com>
7d0271b5c3 depends: Set CMAKE_INSTALL_RPATH for native packages (Russell Yanofsky)
Pull request description:
This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/projects/10).
---
After #19685 started setting `LDFLAGS`, the `INSTALL_RPATH_USE_LINK_PATH` cmake option used in the libmultiprocess build no longer works, so it is neccessary to set `CMAKE_INSTALL_RPATH` as a fallback.
It's unclear currently whether the bad interaction between `INSTALL_RPATH_USE_LINK_PATH` and `LDFLAGS` is a bug, but the issue is reported:
- https://github.com/bitcoin/bitcoin/issues/19981#issuecomment-696680877
- https://discourse.cmake.org/t/install-rpath-use-link-path-not-working-when-cmake-exe-linker-flags-ldflags-is-set/1892Fixes#19981
ACKs for top commit:
fanquake:
ACK 7d0271b5c3 - I haven't looked in depth, but I've re-read through #19981 and checked the failure by testing #19160 (with this reverted):
dongcarl:
ACK 7d0271b Looked into this a bit, it makes sense that for the things we build in depends, we want the library search to start in depends. It seems reasonable to expect this to happen automatically when `CMAKE_INSTALL_PREFIX` and `INSTALL_RPATH_USE_LINK_PATH` are set, but oh well...
Tree-SHA512: 97cc5801c3204c14cd33004423631456ca0701e2127ee5146810a76e2f4aac9de1f4b5437402a4329cda54e022dc99270fee7e38c2995765f36b3848215fa78e
b685f60a08 build: mac_alias 2.1.1 (fanquake)
5d2cbdf772 macdeploy: use Python 3.6 (fanquake)
a42aa94c54 macdeploy: remove runHDIUtil in favor of directly calling subprocess.run (fanquake)
adaa26202b macdeploy: remove existing Bitcoin-Core.dmg if present (fanquake)
ccb0325b1b macdeploy: move qt_conf to where it's used (fanquake)
6390a04862 macdeploy: consolidate .DS_Store generation (fanquake)
32347cd56a macdeploy: assume plistlib is available (fanquake)
0ab4018c12 macdeploy: have a single level of logging output (fanquake)
827d382aa7 macdeploy: remove add-resources argument (fanquake)
464b34d4c3 macdeploy: remove codesigning argument (fanquake)
4d70d3d7fe build: automatically determine macOS translations (fanquake)
Pull request description:
This consolidates our macOS build code so that `.DS_Store` generation is the same when running `make deploy` for macOS when building on Linux and macOS, rather than maintaining two version of code that essentially do the same thing (just slightly differently).
It also removes unused code and any AppleScript usage, automates finding translation files and generally simplifies `macdeployqtplus`. It also gets rid of the annoying "popping up" behaviour during DMG generation, names the created image `Bitcoin-Core.dmg` rather than `Bitcoin-Qt.dmg`.
ACKs for top commit:
dergoegge:
ACK b685f60a08 - Less and cleaner code looks good. I tested this with `make deploy` and everything still works + the popup during DMG generation is gone.
Tree-SHA512: dcd38344e2dfcfa7ffbccf6226a71425c4d16b421a4881d5ee37b8e7ef393b3e8077262444c39b11912269d8cf688aba897e6518cba8361eb24a03fdd03b8caf
c23f6f84ef Add depends qt fix for ARM macs (Jonas Schnelli)
Pull request description:
With this, depends builds fine on macOS 11 on an Apple Silicon Mac (ARM64).
ACKs for top commit:
laanwj:
Code review ACK c23f6f84ef
Tree-SHA512: a8354cec99969cff9e7dab150c335050ddb4b3c93a9f12a4db5e8046f02b11ce692ac17c2b96cbbe7f380c1aa110b15b8d6d48d51bc9c560282c702e99fd8a8d
c82d15b6d1 depends: Do not force Precompiled Headers (PCH) for building Qt on Linux (Hennadii Stepanov)
Pull request description:
On CentOS 8 (Cirrus CI job) the forced `-pch` option breaks Qt build.
Removing `-pch` option does not affect build time for other systems:
- master (e2ff5e7b35):
```
$ time make -j 9 -C depends/ qt
...
Caching qt...
make: Leaving directory '/home/hebasto/guix/GitHub/bitcoin/depends'
real 4m22,359s
user 18m3,719s
sys 1m24,769s
```
- this PR:
```
$ time make -j 9 -C depends/ qt
...
Caching qt...
make: Leaving directory '/home/hebasto/guix/GitHub/bitcoin/depends'
real 4m14,862s
user 18m3,355s
sys 1m24,506s
```
Qt docs: https://doc.qt.io/qt-5/qmake-precompiledheaders.htmlFixes#20423
ACKs for top commit:
MarcoFalke:
review ACK c82d15b6d1
Tree-SHA512: 0f2a3712e90de881d00f8e56c363edde33dd4f5c117df5744ab4e51d0a8146331de7236bc8329d68ddd91535cd853e68ee80ef4cceb6a909786abfd8881b01e8
Rather than using OSX_QT_TRANSLATIONS which must be manually updated,
and we forget to update anyway, i.e: #19059, automatically find and copy
available translations from the translations directory.
2f5dfe4a7f depends: build qt in c++17 mode (fanquake)
104e859c97 builds: don't pass -silent to qt when building in debug mode (fanquake)
e2c500636c depends: build zeromq with -std=c++17 (fanquake)
2374f2fbef depends: build Boost with -std=c++17 (fanquake)
2dde55702d depends: build bdb with -std=c++17 (fanquake)
Pull request description:
In packages where we are passing `-std=c++11` switch to `-std=c++17`, or, `-std=c++1z` in the case of Qt.
This PR also contains a [commit](104e859c97) that improves debug output when building Qt for debugging (`DEBUG=1`).
Now we'll get output like this:
```bash
g++ -c -pipe -ffunction-sections -O2 -fPIC -std=c++11 -fno-exceptions <lots more> ../../corelib/kernel/qcoreapplication.cpp
```
rather than just:
```bash
compiling ../../corelib/kernel/qcoreapplication.cpp
```
Note that when you look at the DEBUG output for these changes when building Qt, you'll see objects being compiled with a mix of C++11 and C++17. The breakdown is roughly:
1. `qmake` built with `-std=c++11`:
```bash
Creating qmake...
make[1]: Entering directory '<trim>/qt/5.9.8-4110fa99945/qtbase/qmake'
g++ -c -o project.o -std=c++11 -ffunction-sections -O2 -g <trim> <trim>/qt/5.9.8-4110fa99945/qtbase/qmake/project.cpp
# when qmake, Qt also builds some of it's corelib, such as corelib/global/qmalloc.cpp
g++ -c -o qmalloc.o -std=c++11 -ffunction-sections -O2 -g <trim> <trim>/qt/5.9.8-4110fa99945/qtbase/src/corelib/global/qmalloc.cpp
```
2. `qmake` is run, and passed our build options, including `-c++std`:
```bash
make[1]: Entering directory '<trim>/qt/5.9.8-4110fa99945/qtbase'
<trim>qt/5.9.8-4110fa99945/qtbase/bin/qmake -o Makefile qtbase.pro -- -bindir <trim>/native/bin -c++std c++1z -confirm-license <trim>
```
3. After some cleaning and configuring, we actually start to build Qt, as well as it's tools and internal libs:
```bash
Building qt...
make[1]: Entering directory '<trim>/qt/5.9.8-4110fa99945/qtbase/src'
# build libpng, zlib etc
gcc -c -m64 -pipe -pipe -O1 <trim> -o .obj/png.o png.c
# build libQt5Bootstrap, using C++11, which again compiles qmalloc.cpp
make[2]: Entering directory '<trim>/qt/5.9.8-4110fa99945/qtbase/src/tools/bootstrap'
g++ -c -pipe -ffunction-sections -O2 -fPIC -std=c++11 <trim> -o .obj/qmalloc.o ../../corelib/global/qmalloc.cpp
# build a bunch of tools like moc, rcc, uic, qfloat16-tables, qdbuscpp2xml, using C++11
g++ -c -pipe -O2 -std=c++11 -fno-exceptions -Wall -W <trim> -o .obj/rcc.o rcc.cpp
# from here, Qt is compiled with -std=c++1z, including qmalloc.cpp, for the third and final time:
g++ -c -include .pch/Qt5Core <trim> -g -Og -fPIC -std=c++1z -fvisibility=hidden <trim> -o .obj/qmalloc.o global/qmalloc.cpp
```
4. Finally, build tools like `lrelease`, `lupdate`, etc, but back to using -std=c++11
```bash
make[1]: Entering directory '<trim>/qt/5.9.8-4110fa99945/qttools/src/linguist/lrelease'
g++ -c -pipe -O2 -std=c++11 -fno-exceptions -Wall -W <trim> -o .obj/translator.o ../shared/translator.cpp
```
If you dump the debug info from the built Qt libs, they should also tell you that they were compiled with `C++17`:
```bash
objdump -g bitcoin/depends/x86_64-pc-linux-gnu/lib/libQt5Core.a
GNU C++17 9.3.0 -m64 -mtune=generic -march=x86-64 -g -O1 -Og -std=c++17 -fPIC -fvisibility=hidden -fvisibility-inlines-hidden -fasynchronous-unwind-tables -fstack-protector-strong -fstack-clash-protection -fcf-protection
```
ACKs for top commit:
laanwj:
Code review ACK 2f5dfe4a7f
practicalswift:
cr ACK 2f5dfe4a7f: patch looks correct
fjahr:
Code review ACK 2f5dfe4a7f
hebasto:
ACK 2f5dfe4a7f, I have reviewed the code and it looks OK, I agree it can be merged.
Tree-SHA512: fc5e9d7c7518c68349c8228fb1aead829850373efc960c9b8c079096a83d1dad19c62a9730fce5802322bf07e320960fd47851420d429eda0a87c307f4e8b03a
8f7d1b39ef Fix QPainter non-determinism on macOS (Andrew Chow)
Pull request description:
Aplies a patch to Qt that fixes the non-determinism by modifying Qt. The source of the non-determinism is how LLVM 8 optimizes qt_intersect_spans when compiling. The particular optimization that seems to be causing the problems is that a temp variable is being added for spans->y. For some reason, when it does this, it chooses different instructions to use when making that variable. We bypass this problem by patching qt_intersect_spans to always make and use this local variable.
Potential alternative to #20436 and #20440
ACKs for top commit:
hebasto:
re-ACK 8f7d1b39ef ~for merging into the 0.21 branch, but [not into the master](https://github.com/bitcoin/bitcoin/pull/20454) branch.~
fanquake:
ACK 8f7d1b39ef
Tree-SHA512: b0d00a77643554021736524fb64611462ef2ec849a220543c12d99edb0f52f2e8128d2cc61fa82176b7e13b294574774a92d6b649badf8b7630c6d6a7e70ce10
This means we'll get build output like this when building with DEBUG=1:
g++ -c -pipe -ffunction-sections -O2 -fPIC -std=c++11 -fno-exceptions <lots more> ../../corelib/kernel/qcoreapplication.cpp
rather than just:
compiling ../../corelib/kernel/qcoreapplication.cpp
Aplies a patch to Qt that fixes the non-determinism by modifying Qt. The
source of the non-determinism is how LLVM 8 optimizes qt_intersect_spans
when compiling. The particular optimization that seems to be causing the
problems is that a temp variable is being added for spans->y. For some
reason, when it does this, it chooses different instructions to use when
making that variable. We bypass this problem by patching
qt_intersect_spans to always make and use this local variable.
a52ecc936a build: set minimum supported macOS to 10.14 (fanquake)
Pull request description:
This is a requirement for C++17 support. See my comments [here](https://github.com/bitcoin/bitcoin/issues/16684#issuecomment-643722538):
> You cannot use std::get with std::variant on macOS < 10.14, because Apples libc++ doesn't support the std::bad_variant_access exception. [Relevant comment](https://github.com/bitcoin/bitcoin/pull/19183#discussion_r439794318) in #19183.
> While we could work around this in our own code, using std::get_if, this would still be a problem for 3rd-party dependencies.
> I've been testing Qt 5.15LTS (we'll have to enable C++17 in qt, and may upgrade to a newer version at the same time), and you can't enable -std c++17, while targeting a macOS deployment version < 10.14, configuring will fail. They are making use of std::get with std::variant throughout their cocoa code.
We would have to had to have bumped to at least 10.13 in any case, as Qt 5.15 (#19716) [requires 10.13+](https://doc.qt.io/qt-5/supported-platforms.html).
ACKs for top commit:
hebasto:
ACK a52ecc936a, I have reviewed the code and it looks OK, I agree it can be merged.
Tree-SHA512: f669b2fc777aeea1e9afdbbc7bd9afe3997418211db6ba53c934cae0e62a9b999603da539518c229f34961d275c9e2f315c7b022cf5fb97bd201a69c85d470cc
e1f2553e11 build: remove global_init_link_order from mac qt qmake.conf (fanquake)
498fa16bea build: document preprocessing steps in qt package (fanquake)
bd5d9336d9 build: don't copy Info.plist.* into mkspec for macOS qt build (fanquake)
bfd7e33b4b build: remove plugin_no_soname from mac qt qmake.conf (fanquake)
fdde4c7ce6 build: pass XCODE_VERSION through to qt macOS cross compile conf (fanquake)
49473ef211 build: convert "echo" usage into a patch in qt package (fanquake)
Pull request description:
Follow up on removing `sed` usage in #19761. Also nice to revisit & cleanup before 5.15.x.
ACKs for top commit:
laanwj:
Code review ACK e1f2553e11
Tree-SHA512: 4e6489d877aaa300f69e091d7117136da49611bd80afd45adfbd7ddeb5b3c9c76fb0f87a3249cbe63ba93129df56281fd4a9389daadc852211325c5ca9ac6567
7087440894 depends: native_ds_store 1.3.0 (fanquake)
Pull request description:
`ds_store` [now takes advantage](36fb607940) of Pythons ability to decode binary [plists](https://docs.python.org/3/library/plistlib.html) (since 3.4), so we can drop its biplist dependency.
The call to `biplist.Data()` in `custom_dsstore.py` doesn't seem to do anything, and from what I can tell can just be removed. i.e:
```diff
diff --git a/contrib/macdeploy/custom_dsstore.py b/contrib/macdeploy/custom_dsstore.py
index dc1c1882d..e475bc6c3 100755
--- a/contrib/macdeploy/custom_dsstore.py
+++ b/contrib/macdeploy/custom_dsstore.py
@@ -47,6 +47,7 @@ alias.volume.disk_image_alias.target.filename = package_name_ns + '.temp.dmg'
alias.volume.disk_image_alias.target.carbon_path = 'Macintosh HD:Users:\x00bitcoinuser:\x00Documents:\x00bitcoin:\x00bitcoin:\x00' + package_name_ns + '.temp.dmg'
alias.volume.disk_image_alias.target.posix_path = 'Users/bitcoinuser/Documents/bitcoin/bitcoin/' + package_name_ns + '.temp.dmg'
alias.target.carbon_path = package_name_ns + ':.background:\x00background.tiff'
+assert(biplist.Data(alias.to_bytes()) == alias.to_bytes())
icvp['backgroundImageAlias'] = biplist.Data(alias.to_bytes())
ds['.']['icvp'] = icvp
```
ACKs for top commit:
laanwj:
ACK 7087440894
Tree-SHA512: 8ba3cf561937efe4a3daae8b0cb4de3bf9e425b3a9244161b09d94ee2b1bd4c3e21315fa70e495b19a052aabdc1731b3b6f346b63272d72d2762ced83237d02f
This has been around since the original import of Qt
(38be0d13830efd2d98281c645c3a60afe05ffece), however there
are now only two instatnces of it left in the qt codebase,
and from what I can gather, it's unused.
We generate our own Info.plist as part of make deploy, and as far as I
can tell, it doesn't seem to have an effect wether these are present
during qt's build.
I also can't find a single mention of the .app plist in the qt code,
whereas there are multiple instances of .lib.
plugin_no_soname was removed from Qt some time ago, see upstream commit
1d034244c261520d5e739534dc264c2500e02b5f. It was replaced with
plugin_with_soname, however that is currently only used (as of 5.15.x)
in the Android Clang mkspec.
This should mostly be a no-op, however it would seem to make more sense
that we pass through the XCODE_VERSION we now have in depends, rather
than leaving the version set to 4.3.
native_ds_store now takes advantage of Pythons ability to decode binary
plists (since 3.4), so we can drop its biplist dependency.
The call to biplist.Data() in custom_dsstore doesn't seem to do anything,
and from what I can tell can just be removed.
Previously, when running ./configure:
1. With CONFIG_SITE pointed to our depends config.site.in, and
2. PYTHONPATH was not set either in the environment or by the user
The configure would output something like:
PYTHONPATH='depends/x86_64-pc-linux-gnu/share/../native/lib/python3/dist-packages:'
When we really mean:
PYTHONPATH='depends/x86_64-pc-linux-gnu/share/../native/lib/python3/dist-packages'
...without the colon
This change makes sure that:
1. There's no trailing colon, and
2. We use the $PATH_SEPARATOR variable instead of a colon
Files like config.site.in are not referenced by any other script in our
tree, so we need to mark it manually with a "shellcheck shell="
directive and make sure that shellcheck is run on them.
Previously, if ./configure was invoked with:
```
$ env CONFIG_SITE=depends/x86_64-pc-linux-gnu/share/config.site ./configure
```
Where $CONFIG_SITE was a relative path, ./configure would fail with the
following misleading output:
```
checking for boostlib >= 1.58.0 (105800)... yes
checking whether the Boost::System library is available... yes
configure: error: Could not find a version of the Boost::System library!
```
Fully resolving depends_prefix in config.site.in fixes this. To make
sure that there are no other side effects I ran a diff on the
config.status generated by:
1. The scripts prior to this change with CONFIG_SITE set to a full path:
env CONFIG_SITE=$PWD/depends/x86_64-pc-linux-gnu/share/config.site ./configure
2. The scripts after this change with CONFIG_SITE set to a relative path:
env CONFIG_SITE=depends/x86_64-pc-linux-gnu/share/config.site ./configure
And it looks good!
Diff: https://paste.sr.ht/~dongcarl/95b469fbc555c128046e85723d87a9082a754f6b
d0a829e963 build: fix mutex detection when building bdb on macOS (fanquake)
Pull request description:
Starting with the Apple Clang shipped with Xcode 12, [Apple has enabled -Werror=implicit-function-declaration by default](https://developer.apple.com/documentation/xcode-release-notes/xcode-12-release-notes):
> Clang now reports an error when you use a function without an explicit declaration when building C or Objective-C code for macOS (-Werror=implicit-function-declaration flag is on). This additional error detection unifies Clang’s behavior for iOS/tvOS and macOS 64-bit targets for this diagnostic. (49917738)
This causes bdbs mutex detection to fail when building on macOS (not cross-compiling):
```bash
checking for mutexes... UNIX/fcntl
configure: WARNING: NO SHARED LATCH IMPLEMENTATION FOUND FOR THIS PLATFORM.
configure: error: Unable to find a mutex implementation
```
as previously emitted warnings are being turned into errors. i.e:
```bash
configure:18704: checking for mutexes
configure:18815: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -mmacosx-version-min=10.12 --sysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk -o conftest -pipe -O2 -I/Users/michael/github/fanquake-bitcoin/depends/x86_64-apple-darwin19.6.0/include -L/Users/michael/github/fanquake-bitcoin/depends/x86_64-apple-darwin19.6.0/lib conftest.c -lpthread >&5
conftest.c:46:1: warning: type specifier missing, defaults to 'int' [-Wimplicit-int]
main() {
^
conftest.c:51:2: error: implicitly declaring library function 'exit' with type 'void (int) __attribute__((noreturn))' [-Werror,-Wimplicit-function-declaration]
exit (
^
conftest.c:51:2: note: include the header <stdlib.h> or explicitly provide a declaration for 'exit'
1 warning and 1 error generated.
```
Append `-Wno-error=implicit-function-declaration` to `cflags` so that `-Wimplicit-function-declaration` [returns to being a warning](https://clang.llvm.org/docs/UsersManual.html#cmdoption-wno-error), and the configure checks succeed.
Fixes#19411.
ACKs for top commit:
laanwj:
Code review ACK d0a829e963
Tree-SHA512: 7813005b1fc0b370f843b6c0672acab32c999416e92c3f02b75d866e9c7aa41fe5822704fc74de6b65f0d7d94f2cdd05cc7c3ee83295ff1ecbc71d8492b9a2bf
Starting with the Clang shipped with Xcode 12, Apple has enabled
-Werror=implicit-function-declaration by default. This causes bdbs mutex
detection to fail when building on macOS (not cross-compiling):
checking for mutexes... UNIX/fcntl
configure: WARNING: NO SHARED LATCH IMPLEMENTATION FOUND FOR THIS PLATFORM.
configure: error: Unable to find a mutex implementation
as previously emitted warnings are being turned into errors. i.e:
error: implicitly declaring library function 'exit' with type 'void (int) __attribute__((noreturn))' [-Werror,-Wimplicit-function-declaration]
Append -Wno-error=implicit-function-declaration to cflags so that
-Wimplicit-function-declaration returns to being a warning, and the
configure checks will succeed.
Fixes#19411.
After #19685 started setting LDFLAGS, the INSTALL_RPATH_USE_LINK_PATH cmake
option used in the libmultiprocess build no longer works, so it is neccessary
to set CMAKE_INSTALL_RPATH as a fallback.
It's unclear currently whether the bad interaction between
INSTALL_RPATH_USE_LINK_PATH and LDFLAGS is a bug, but the issue is reported:
https://github.com/bitcoin/bitcoin/issues/19981#issuecomment-696680877https://discourse.cmake.org/t/install-rpath-use-link-path-not-working-when-cmake-exe-linker-flags-ldflags-is-set/1892
Commands useful for building / testing this change
make -C depends MULTIPROCESS=1 print-libmultiprocess_cmake
make -C depends MULTIPROCESS=1 print-native_libmultiprocess_cmake
make -C depends MULTIPROCESS=1 HOST=x86_64-apple-darwin16 print-libmultiprocess_cmake
rm -rvf depends/x86_64-pc-linux-gnu/native depends/work/staging depends/work/build
make -C depends MULTIPROCESS=1 V=1 native_libmultiprocess_staged
for f in `find -name mpgen`; do echo == $f ==; readelf -d $f | grep -i path; done
make -C depends MULTIPROCESS=1 V=1 native_libmultiprocess_built
find -name CMakeCache.txt
Fixes#19981
7a89f2e6c5 build: Fix target name (Hennadii Stepanov)
Pull request description:
It seems like a typo :)
This PR:
- fixes errors when building a package in depends for `HOST=x86_64-apple-darwin16` (fix#19799)
- is a correct alternative to d25e0e308f from #19764
ACKs for top commit:
icota:
tACK 7a89f2e6c5
dongcarl:
Code Review ACK 7a89f2e6c5
theuni:
ACK 7a89f2e6c5.
Tree-SHA512: a0bcbc6805d3450e201476ef1e22e0eb53903db1586c5515314c19afd337bded887e56de0fbe62feaf359b2de15dbccd49a44f1a8b566b4c64f5ae3d94a2ab6d
This is an alternative to #19751 that fixes the build without requiring
splitting out libpng. This patch can be dropped once we are building qt
5.12.0 or later.
b893688357 depends: Specify LDFLAGS to cmake as well (Carl Dong)
b3f541f618 depends: Prepend CPPFLAGS to C{,XX}FLAGS for CMake (Carl Dong)
8e121e5509 depends: Cleanup CMake invocation (Carl Dong)
8c7cd0c6d9 depends: More robust cmake invocation (Carl Dong)
3ecf0eca63 depends: Use $($(package)_cmake) instead of cmake (Carl Dong)
Pull request description:
- Use `$($(package)_cmake)` instead of invoking `cmake` directly
- Use well-known env vars instead of overriding CMake variables
ACKs for top commit:
ryanofsky:
Code review ACK b893688357. Only changes since last review are new commits adding whitespace, cppflags and ldflags to cmake invocation
Tree-SHA512: cfcd8cc9dcd0b336cf48b82fca9fe4bbc7930ed397cb7a68a07066680eb4c1906a6a9b5bd2589b4b4999e8f16232fa30ee9b376b60f4456d0fff931fbf9cc19a
The usage of pragmas within the macOS SDK requires LLVM Clang 8. This is
the version as our prebuilt Clang, however the minimum is worth noting here
as they may diverge and/or expert users might expect they could use an
earlier version.
If you compile using Clang 7 you'll see output like:
```bash
In file included from kernel/qcore_mac_objc.mm:44:
In file included from /bitcoin/depends/SDKs/Xcode-11.3.1-11C505-extracted-SDK-with-libcxx-headers/System/Library/Frameworks/AppKit.framework/Headers/NSText.h:9:
In file included from /bitcoin/depends/SDKs/Xcode-11.3.1-11C505-extracted-SDK-with-libcxx-headers/System/Library/Frameworks/AppKit.framework/Headers/NSView.h:19:
In file included from /bitcoin/depends/SDKs/Xcode-11.3.1-11C505-extracted-SDK-with-libcxx-headers/System/Library/Frameworks/AppKit.framework/Headers/NSResponder.h:10:
/bitcoin/depends/SDKs/Xcode-11.3.1-11C505-extracted-SDK-with-libcxx-headers/System/Library/Frameworks/AppKit.framework/Headers/NSEvent.h:19:1: error:
expected 'push' or 'pop' after '#pragma clang attribute'
/bitcoin/depends/SDKs/Xcode-11.3.1-11C505-extracted-SDK-with-libcxx-headers/usr/include/os/availability.h:104:273: note: expanded from macro
'API_UNAVAILABLE_BEGIN'
...__API_UNAVAILABLE_BEGIN5, __API_UNAVAILABLE_BEGIN4, __API_UNAVAILABLE_BEGIN3, __API_UNAVAILABLE_BEGIN2, __API_UNAVAILABLE_BEGIN1, 0)(__VA_A...
^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.
```
5962522fbc depends: bump native_cctools for fixed lto with external clang (Cory Fields)
00d1ba7aaa depends: enable lto support for Apple's ld64 (Cory Fields)
Pull request description:
This didn't work for a few reasons (various toolchain compatibility issues) the last time I tested it, but after the last round of bumps it works with no apparent issues.
Note that this does not _enable_ LTO by default in any way, only hooks up the machinery for ```-flto``` to work correctly when specified.
Lines were split for an easier rebase after #17919 is merged.
ACKs for top commit:
fanquake:
ACK 5962522fbc. The relevant option upstream is [here](https://github.com/tpoechtrager/cctools-port/blob/master/cctools/m4/llvm.m4#L4).
Tree-SHA512: df2775e74e7bc847e6cef94cb8457d503d6c9e2fdea861e51386fa6ed5a7ba688241db3685561ae1a32f66724c1b3801727252025f00c04b90a3bdc8a4f6f93b
2a701a1c42 build: pass -fcommon when building genisoimage (fanquake)
Pull request description:
Starting with the 10.1 release, GCC [defaults to -fno-common](https://gcc.gnu.org/gcc-10/porting_to.html). This causes
linking issues when building genisoimage:
```bash
[ 98%] Building C object genisoimage/CMakeFiles/genisoimage.dir/checksum.o
[100%] Linking C executable genisoimage
/usr/bin/ld: CMakeFiles/genisoimage.dir/apple.o:(.bss+0x0): multiple definition of `outfile'; CMakeFiles/genisoimage.dir/genisoimage.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/genisoimage.dir/boot.o:(.bss+0x0): multiple definition of `outfile'; CMakeFiles/genisoimage.dir/genisoimage.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/genisoimage.dir/desktop.o:(.bss+0x0): multiple definition of `outfile'; CMakeFiles/genisoimage.dir/genisoimage.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/genisoimage.dir/dvd_file.o:(.bss+0x0): multiple definition of `outfile'; CMakeFiles/genisoimage.dir/genisoimage.o:(.bss+0x0): first defined here
```
Rather than patching genisoimage further, just pass -fcommon to preserve
the legacy GCC behaviour.
Noticed [while testing #19530](https://github.com/bitcoin/bitcoin/pull/19530#issuecomment-659802512).
ACKs for top commit:
laanwj:
ACK 2a701a1c42
hebasto:
ACK 2a701a1c42, tested on Fedora 32 (x86_64, GCC 10.1.1):
Tree-SHA512: 0e6dd1b4c7c3f5b6997616da27a7ea54f757c81677766ee6813c776356527787a32b959f834e699fee9798553b13bcb142469f442c5e7a04f810f2c06e3a9505
Starting with the 10.1 release, GCC defaults to -fno-common. This causes
linking issues when building genisoimage:
```bash
[ 98%] Building C object genisoimage/CMakeFiles/genisoimage.dir/checksum.o
[100%] Linking C executable genisoimage
/usr/bin/ld: CMakeFiles/genisoimage.dir/apple.o:(.bss+0x0): multiple definition of `outfile'; CMakeFiles/genisoimage.dir/genisoimage.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/genisoimage.dir/boot.o:(.bss+0x0): multiple definition of `outfile'; CMakeFiles/genisoimage.dir/genisoimage.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/genisoimage.dir/desktop.o:(.bss+0x0): multiple definition of `outfile'; CMakeFiles/genisoimage.dir/genisoimage.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/genisoimage.dir/dvd_file.o:(.bss+0x0): multiple definition of `outfile'; CMakeFiles/genisoimage.dir/genisoimage.o:(.bss+0x0): first defined here
```
Rather than patching genisoimage further, pass -fcommon to preserve the
legacy GCC behaviour.
6457361e90 qt: Fix QFileDialog for static builds (Hennadii Stepanov)
Pull request description:
This change partially reverts 248e22bbc0 (#16386) and makes `QFileDialog`s work again for static builds.
Fixes https://github.com/bitcoin-core/gui/issues/32.
ACKs for top commit:
fanquake:
ACK 6457361e90. Although it would be good to know exactly _why_ this fixes the issue. At this stage I also don't think this should be a blocker for 0.20.1.
theuni:
ACK 6457361e90
Tree-SHA512: 8ad27e0bcae6debd02f73b7c374743e37d4edd806922b103a2fe494cf2d9930fe9ef3107b5a6c61f3c466cf7462de2641171880398954e7f2c4f417f5bb820d7
