332f373a9d [scripts] previous_release: improve failed download error message (Sebastian Falbesoner)
Pull request description:
Currently, if the earlier release build/fetch script `previous_release.sh` is invoked with the option `-b` (intending to fetch a binary package from `https://bitcoin.org`) and the download fails, the user sees the following confusing output:
```
$ contrib/devtools/previous_release.sh -r -b v0.9.5
[...]
gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Error is not recoverable: exiting now
```
This implies that the download worked, but the archive is corrupted, when in reality the HTML document containing the delivery fail reason (most likely 404 Not Found) is saved and tried to get unpacked. In contrast to wget, curl is a bit stubborn and needs explicit instructions to react to server errors via the flag `-f` (outputs error message and returns error code, ideal for scripts): https://curl.haxx.se/docs/manpage.html#-f
On the PR branch, the output on failed download looks now the following:
```
$ contrib/devtools/previous_release.sh -r -b v0.9.5
[...]
curl: (22) The requested URL returned error: 404 Not Found
Download failed.
```
ACKs for top commit:
fanquake:
ACK 332f373a9d
Tree-SHA512: 046c931ad9e78aeb2d13faa4866d46122ed325aa142483547c2b04032d03223ed2411783b00106fcab0cd91b2f78691531ac526ed7bb3ed7547b6e2adbfb2e93
This bump will includes a couple of commits which improve the
reproducibility of the mingw-w64 toolchain. Most of which came from
debian. They will be upstreamed as upstream Guix release timeline
allows.
- Add "--no-insert-timestamp" LDFLAG for x86_64-w64-mingw32 builds
"The option --no-insert-timestamp can be used to insert a zero value for
the timestamp, this ensuring that binaries produced from identical
sources will compare identically." - ld(1)
- Set "SetDateSave off" in NSIS script
From https://nsis.sourceforge.io/Docs/Chapter4.html#flags
"This command sets the file date/time saving flag which is used by the
File command to determine whether or not to save the last write date and
time of the file, so that it can be restored on installation. Valid
flags are 'on' and 'off'. 'on' is the default."
- Add commented out NSIS options for reproducibility debugging in NSIS
script
- Make ZIPs deterministic by reseting file modification times to
SOURCE_DATE_EPOCH using touch(1) (Reference:
https://reproducible-builds.org/docs/archives/)
3e0df92bf2 Update with new Windows code signing certificate (Andrew Chow)
Pull request description:
The current Windows code signing certificate is about expire (on March 26th 2020). As I have volunteered to take over the Windows code signing duties, I've purchased a new Windows code signing certificate with the same CA and under the same organization (Bitcoin Core Code Signing Association).
A signature by the old certificate over the new certificate has been provided to me. This signature can be verified using
```
openssl cms -verify -inform pem -purpose any -content path/to/new/win-codesign.cert -CAfile path/to/old/win-codesign.cert -certfile path/to/old/win-codesign.cert
```
The verification should succeed and the new certificate will be printed out. This can be compared to the contents of `win-codesign.cert`.
```
-----BEGIN PKCS7-----
MIIC3AYJKoZIhvcNAQcCoIICzTCCAskCAQExDzANBglghkgBZQMEAgEFADALBgkq
hkiG9w0BBwExggKkMIICoAIBATCBkTB8MQswCQYDVQQGEwJHQjEbMBkGA1UECBMS
R3JlYXRlciBNYW5jaGVzdGVyMRAwDgYDVQQHEwdTYWxmb3JkMRgwFgYDVQQKEw9T
ZWN0aWdvIExpbWl0ZWQxJDAiBgNVBAMTG1NlY3RpZ28gUlNBIENvZGUgU2lnbmlu
ZyBDQQIRALWcUnSOxv9FQW3xdaMDO6swDQYJYIZIAWUDBAIBBQCggeQwGAYJKoZI
hvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMjAwMzI0MjA0ODM3
WjAvBgkqhkiG9w0BCQQxIgQgtLkmnuSQyczDlJSnJeqbi61p3iJ/rpFABrY8JWBO
o74weQYJKoZIhvcNAQkPMWwwajALBglghkgBZQMEASowCwYJYIZIAWUDBAEWMAsG
CWCGSAFlAwQBAjAKBggqhkiG9w0DBzAOBggqhkiG9w0DAgICAIAwDQYIKoZIhvcN
AwICAUAwBwYFKw4DAgcwDQYIKoZIhvcNAwICASgwDQYJKoZIhvcNAQEBBQAEggEA
XaCl3Q8HwI9VpLCb9OY9eQh0QOPyl1KWEc3TP3UvwZwR4/gXkfPOKKf19UnS8eRB
48SgUKRMYWoDYfSVUJRMda9BLkbJbQlHG3LFXhSY2alajpPXEHcMto/XPhVAmqzL
w6aSNY0Gaorow696JHpetpKqAAlL1r2GjeaPYi2aZyIAifuhay/qwA+ig0SqzGOw
UdgFZWMyS5yanq8/WlLCCql6kKOzT4tEqUaleD7R1q8BTcG2+fmhWR8WwJLpIV6y
7GAqt0Cocu8sYpTNBNk8iKHxzZ2hMZKJpH9lHZuiJ/9vSercrvDy2R4/MG+KnBWb
OyiFAt2mC51+63RhLOMJfg==
-----END PKCS7-----
```
ACKs for top commit:
laanwj:
ACK 3e0df92bf2
theuni:
ACK 3e0df92bf2.
Tree-SHA512: 4210f4db1e805ab11231fbae49ea197257c6f7e44f1f6219685b63831704984d824ac2f9e0a3b1bd2655953af72636a474f077cb859fb35852551f5a9f8fbde3
e4d366788b build: Drop needless EXTRA_DIST content (Hennadii Stepanov)
6c4da59f5b build: Drop SOURCEDIST reordering (Hennadii Stepanov)
5e6b8b3912 build: Use git archive as source tarball (Hennadii Stepanov)
Pull request description:
This PR:
- is an alternative to #17104
- closes#16734
- closes#6753
The idea is clear described by some developers:
- [MarcoFalke](https://github.com/bitcoin/bitcoin/pull/17097#issuecomment-540691850):
> This whole concept of explicitly listing each and every file manually (or with a fragile wildcard) is an obvious sisyphean task. I'd say all we need to do is run git archive and be done with it forever, see #16734, #6753, #11530 ...
- [laanwj](https://github.com/bitcoin/bitcoin/pull/17097#issuecomment-540706025):
> I agree, I've never been a fan of it. I don't think we have any files in the git repository we don't want to ship in the source tarball.
---
The suggested changes have a downside which is pointed by [**luke-jr**](https://github.com/bitcoin/bitcoin/pull/17104#issuecomment-540828045):
> ... but the distfile needs to include autogen-generated files.
This means that a user is not able to run `./configure && make` right away. One must run `./autogen.sh` at first.
Here are opinions about mandatory use of `./autogen.sh`:
- [ryanofsky](https://github.com/bitcoin/bitcoin/issues/16734#issuecomment-534139356):
> It's probably ok to require autogen. I think historically configure scripts were supposed to work on obscure unix systems that would just have a generic shell + make tool + c compiler, and not necessarily need gnu packages like m4 which are needed for autogen.
- [laanwj](https://github.com/bitcoin/bitcoin/issues/16734#issuecomment-540729483):
> I also think it's fine to require autogen. What is one dependency more, if you're building from source.
---
~Also this PR provides Windows users with ZIP archives of the sources. Additionally the commit ID is stored in these ZIP files as a file comment:~
---
Note for reviewers: please verify is `git archive` output deterministic?
ACKs for top commit:
MarcoFalke:
re-ACK e4d366788b, only change is adding two dots in a the path 🛳
laanwj:
ACK e4d366788b
Tree-SHA512: d1153d3ca4a580696019b92be3555ab004d197d9a2146aacff9d3150eb7093b7d40eebd6eea12d861d93ff62d62b68706e04e64dbe5ea796ff6757486e462193
before:
------------------------------------------------------------
$ contrib/devtools/previous_release.sh -r -b v0.9.5
[...]
gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Error is not recoverable: exiting now
------------------------------------------------------------
now:
------------------------------------------------------------
$ contrib/devtools/previous_release.sh -r -b v0.9.5
[...]
curl: (22) The requested URL returned error: 404 Not Found
Download failed.
------------------------------------------------------------
Previously, Guix would produce a gcc which did not know to use the SSP
function from glibc, and required a gcc make flag for it to do so, in my
attempt to fix it upstream I realized that this is no longer the case.
This can be verified by performing a Guix build and doing
readelf -s ... | grep __stack_chk
to check that symbols are coming from glibc, and doing
readelf -d ... | grep NEEDED | grep ssp
to see that libssp.so is not being depended on
fae9084ac5 build: Skip i686 build by default in guix and gitian (MarcoFalke)
fa55a2554c depends: Remove reference to win32 (MarcoFalke)
Pull request description:
Closes#17504
Now that we no longer provide downloads for i686 on our website (https://bitcoincore.org/en/download/), there is no need to build them by default.
i686 can still be built in depends (tested by ci/travis) and in guix/gitian by setting the appropriate `HOSTS`.
ACKs for top commit:
practicalswift:
ACK fae9084ac5 -- patch looks correct
dongcarl:
ACK fae9084ac5 patch looks correct
laanwj:
Code review ACK fae9084ac5
hebasto:
ACK fae9084ac5, I have reviewed the code and it looks OK, I agree it can be merged.
Tree-SHA512: b000c19a2cd2a596a52028fa298c4022c24cfdfc1bdb3795a90916d0a00a32e4dd22278db93790b6a11724e08ea8451f4f05c77bc40d1664518e11a8c82d6e29
677fb8e923 test: Add ubsan surpression for crc32c (Wladimir J. van der Laan)
8e68bb1dde build: Disable msvc warning 4722 for leveldb build (Aaron Clauson)
be23949765 build: MSVC changes for leveldb update (Aaron Clauson)
9ebdf04757 build: CRC32C build system integration (Wladimir J. van der Laan)
402252a808 build: Add LCOV exception for crc32c (Wladimir J. van der Laan)
3a037d0067 test: Add crc32c exception to various linters and generation scripts (Wladimir J. van der Laan)
84ff1b2076 test: Add crc32c to subtree check linter (Wladimir J. van der Laan)
7cf13a5134 doc: Add crc32c subtree to developer notes (Wladimir J. van der Laan)
24d02a9ac0 build: Update build system for new leveldb (Wladimir J. van der Laan)
2e1819311a Squashed 'src/crc32c/' content from commit 224988680f7673cd7c769963d4035cb315aa3388 (Wladimir J. van der Laan)
66480821b3 Squashed 'src/leveldb/' changes from f545dfabff4c2e9836efed094dba99a34fbc6b88..f8ae182c1e5176d12e816fb2217ae33a5472fdd7 (Wladimir J. van der Laan)
Pull request description:
This updates leveldb to currently newest upstream commit 0c40829872:
- CRC32C hardware acceleration is now an external library [crc32c](https://github.com/google/crc32c). This adds acceleration on ARM, and should be faster on x86 because of using prefetch. It also makes it easy to support similar instruction sets on other platforms in the future.
- Thread handling uses C++11, instead of platform specific code.
- Native windows environment was added. No need to maintain our own hacky one, anymore.
- Upstream now builds using CMake. This doesn't mean we need to use that (phew), but internal configuration changed to a a series of checks, instead of OS profiles. This means the blanket error "Cannot build leveldb for $host. Please file a bug report' is removed.
All changes: a53934a3ae...0c40829872
Pretty much all our changes have been subsumed by upstream, so we figured it was cleaner to start over with a new branch from upstream with the still-relevant patches applied: https://github.com/bitcoin-core/leveldb/tree/bitcoin-fork-new
There's quite some testing to be done (see below). See https://github.com/bitcoin-core/leveldb/issues/25 and https://github.com/bitcoin-core/leveldb/pull/26 for more history and context.
TODO:
- [x] Subtree `crc32c`
- [x] Make linters happy about crc32 subtree
- [x] Integrate `crc32c` library into build system
- [x] MSVC build system
ACKs for top commit:
sipa:
ACK 677fb8e923
Tree-SHA512: 37ee92a750e053e924bc4626b12bb3fd81faa9f8c5ebaa343931fee810c45ba05aa6051fdea82535fa351bf2be7297801b98af9469865fc5ead771650a5d6240
317fb96de9 Add search for first blk file with pruned node (Rjected)
Pull request description:
<!--
*** Please remove the following help text before submitting: ***
Pull requests without a rationale and clear improvement may be closed
immediately.
-->
<!--
Please provide clear motivation for your patch and explain how it improves
Bitcoin Core user experience or Bitcoin Core developer experience
significantly:
* Any test improvements or new tests that improve coverage are always welcome.
* All other changes should have accompanying unit tests (see `src/test/`) or
functional tests (see `test/`). Contributors should note which tests cover
modified code. If no tests exist for a region of modified code, new tests
should accompany the change.
* Bug fixes are most welcome when they come with steps to reproduce or an
explanation of the potential issue as well as reasoning for the way the bug
was fixed.
* Features are welcome, but might be rejected due to design or scope issues.
If a feature is based on a lot of dependencies, contributors should first
consider building the system outside of Bitcoin Core, if possible.
* Refactoring changes are only accepted if they are required for a feature or
bug fix or otherwise improve developer experience significantly. For example,
most "code style" refactoring changes require a thorough explanation why they
are useful, what downsides they have and why they *significantly* improve
developer experience or avoid serious programming bugs. Note that code style
is often a subjective matter. Unless they are explicitly mentioned to be
preferred in the [developer notes](/doc/developer-notes.md), stylistic code
changes are usually rejected.
-->
When bitcoind is running in pruned mode, producing a hashlist with `./linearize-hashes.py linearize.cfg > hashlist.txt` and then executing `linearize-data.py linearize.cfg` will produce:
```
Read 313001 hashes
Input file /home/dan/.bitcoin/blocks/blk00000.dat
Premature end of block data
```
This happens because `linearize-data` starts by attempting to process `blk00000.dat` regardless of whether or not `blk00000.dat` actually exists - this may not be the case if working with a pruned node.
This PR adds a function which finds the first block file that does exist, and calls that function when the `BlockDataCopier` is initialized.
This is a refactor of #16431.
<!--
Bitcoin Core has a thorough review process and even the most trivial change
needs to pass a lot of eyes and requires non-zero or even substantial time
effort to review. There is a huge lack of active reviewers on the project, so
patches often sit for a long time.
-->
ACKs for top commit:
darosior:
ACK 317fb96de9
laanwj:
Code review ACK 317fb96de9
theStack:
Code review ACK 317fb96de9
Tree-SHA512: fc8014282df6cfe7b267e64db8ce7d82b86b758c302fbfea4a3c39b62d93512f5c2e31a0de4e9c5ec18fc0268c917f011257d37b45afaef6033eec90e4aa585f
b3c4d9bac6 test: rename test suite name "tx_validationcache_tests" to match filename (Sebastian Falbesoner)
Pull request description:
Quoting `src/test/README.md`, '`Adding test cases`':
> "The file naming convention is `<source_filename>_tests.cpp`
> and such files should wrap their tests in a test suite
> called `<source_filename>_tests`."
Currently the unit test source file `txvalidationcache_tests.cpp` contains a unit test suite with the name `tx_validationcache_tests`, which is fixed by this PR. The following shell script shows that this is the only mismatch and for all other unit test source files the test suite names are correct:
```
#!/bin/bash
shopt -s globstar
for test_full_filename in **/*_tests.cpp; do
test_name_file=`basename $test_full_filename .cpp`
test_name_suite=`sed -n "s/^.*TEST_SUITE(\(.*_tests\).*$/\1/p" $test_full_filename`
if [ $test_name_file != $test_name_suite ]; then
echo "TestFilename: $test_name_file != TestSuitname: $test_name_suite"
fi
done
```
ACKs for top commit:
practicalswift:
ACK b3c4d9bac6 -- expected naming is better than unexpected naming :)
kristapsk:
ACK b3c4d9bac6
Tree-SHA512: 29d409b1eb22057ee2cc407508e2580d2bc03f412401df11b8ecf77be5ada6bda8f7d2cb5338c5e079490fa12242c1fd6230a09e47252c1b0d9fe535a828ca4c
88c83636d5 guix: Update documentation for time-machine (Carl Dong)
e6050884fd guix: Pin Guix using `guix time-machine` (Carl Dong)
Pull request description:
An alternative to #16519, pinning our version of Guix and eliminating a `guix pull` and changing the default Guix profile of builders.
I think this method might be superior, as it:
- Eliminates the possibility of future changes to the `guix environment` command line interface breaking our builds
- Eliminates the need to set up a separate channel repo
It is a more general pinning solution than #16519.
-----
The reason why I didn't originally propose this is because `guix time-machine` is a recent addition to Guix, only available since `f675f8dec73d02e319e607559ed2316c299ae8c7`
ACKs for top commit:
fanquake:
ACK 88c83636d5
Tree-SHA512: 85e03b0987ffa86da73e02801e1cd8b7622698d70c4ba4e60561611be1e9717d661c2811a59b3e137b1b8eef2d0ba37c313867d035ebc89c3bd06a23a078064a
Quoting src/test/README.md, 'Adding test cases':
"The file naming convention is `<source_filename>_tests.cpp`
and such files should wrap their tests in a test suite
called `<source_filename>_tests`."
Currently the unit test source file txvalidationcache_tests.cpp contains a unit
test suite with the name tx_validationcache_tests, which is fixed by this commit.
The following shell script shows that this is the only mismatch and for all other
unit test source files the test suite names are correct:
#!/bin/bash
shopt -s globstar
for test_full_filename in **/*_tests.cpp; do
test_name_file=`basename $test_full_filename .cpp`
test_name_suite=`sed -n "s/^.*TEST_SUITE(\(.*_tests\).*$/\1/p" $test_full_filename`
if [ $test_name_file != $test_name_suite ]; then
echo "TestFilename: $test_name_file != TestSuitname: $test_name_suite"
fi
done
2d23082cbe bump test timeouts so that functional tests run in valgrind (Micky Yun Chan)
Pull request description:
ci/tests: Bump timeouts so all functional tests run on travis in valgrind #17763
Top commit has no ACKs.
Tree-SHA512: 5a8c6e2ea02b715facfcb58c761577be15ae58c45a61654beb98c2c2653361196c2eec521bcae4a9a1bab8e409d6807de771ef4c46d3d05996ae47a22d499d54
c491368d8c scripts: add MACHO dylib checking to symbol-check.py (fanquake)
76bf97213f scripts: fix check-symbols & check-security argument passing (fanquake)
Pull request description:
Based on #17857.
This adds dynamic library checks for MACHO executables to symbol-check.py. The script has been modified to function more like `security-check.py`. The error output is now also slightly different. i.e:
```bash
# Linux x86
bitcoin-cli: symbol operator new[](unsigned long) from unsupported version GLIBCXX_3.4
bitcoin-cli: export of symbol vtable for std::basic_ios<char, std::char_traits<char> > not allowed
bitcoin-cli: NEEDED library libstdc++.so.6 is not allowed
bitcoin-cli: failed IMPORTED_SYMBOLS EXPORTED_SYMBOLS LIBRARY_DEPENDENCIES
# RISCV (skips exported symbols checks)
bitcoin-tx: symbol operator new[](unsigned long) from unsupported version GLIBCXX_3.4
bitcoin-tx: NEEDED library libstdc++.so.6 is not allowed
bitcoin-tx: failed IMPORTED_SYMBOLS LIBRARY_DEPENDENCIES
# macOS
Checking macOS dynamic libraries...
libboost_filesystem.dylib is not in ALLOWED_LIBRARIES!
bitcoind: failed DYNAMIC_LIBRARIES
```
Compared to `v0.19.0.1` the macOS allowed dylibs has been slimmed down somewhat:
```diff
src/qt/bitcoin-qt:
/usr/lib/libSystem.B.dylib
-/System/Library/Frameworks/DiskArbitration.framework/Versions/A/DiskArbitration
/System/Library/Frameworks/IOKit.framework/Versions/A/IOKit
/System/Library/Frameworks/Foundation.framework/Versions/C/Foundation
/System/Library/Frameworks/CoreServices.framework/Versions/A/CoreServices
/System/Library/Frameworks/AppKit.framework/Versions/C/AppKit
/System/Library/Frameworks/ApplicationServices.framework/Versions/A/ApplicationServices
/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation
-/System/Library/Frameworks/Security.framework/Versions/A/Security
-/System/Library/Frameworks/SystemConfiguration.framework/Versions/A/SystemConfiguration
/System/Library/Frameworks/CoreGraphics.framework/Versions/A/CoreGraphics
-/System/Library/Frameworks/OpenGL.framework/Versions/A/OpenGL
-/System/Library/Frameworks/AGL.framework/Versions/A/AGL
/System/Library/Frameworks/Carbon.framework/Versions/A/Carbon
/usr/lib/libc++.1.dylib
-/System/Library/Frameworks/CFNetwork.framework/Versions/A/CFNetwork
/System/Library/Frameworks/CoreText.framework/Versions/A/CoreText
/System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO
/usr/lib/libobjc.A.dylib
```
ACKs for top commit:
laanwj:
ACK c491368d8c
Tree-SHA512: f8624e4964e80b3e0d34e8d3cc33f3107938f3ef7a01c07828f09b902b5ea31a53c50f9be03576e1896ed832cf2c399e03a7943a4f537a1e1c705f3804aed979
fae75306ba scripted-diff: Set gitian arch back to amd64 (MarcoFalke)
Pull request description:
This was required to allow gitian builds on non-amd64 architecture, however, it seems to break the current builds (with lxc), see https://github.com/bitcoin/bitcoin/pull/17409#issuecomment-554099626
Also, the gititan builds wouldn't be deterministic across arches anyway, see #17468
So instead of wasting more time on this, revert the change and hope that guix allows to compile on non-amd64 architectures.
Top commit has no ACKs.
Tree-SHA512: 801e9a30ae1b0882ef45d5eb3a3cf80f3ace3b99db046069dbd95b6162119e977e3cf3134287d1ac5d09483906206acc71e1ac34d6b74dbc533d46aaf73f5cc2
e1c582cbaa contrib: makeseeds: Read suspicious hosts from a file instead of hardcoding (Sanjay K)
Pull request description:
referring to: https://github.com/bitcoin/bitcoin/issues/17020
good first issue: reading SUSPICIOUS_HOSTS from a file.
I haven't changed the base hosts that were included in the original source, just made it readable from a file.
ACKs for top commit:
practicalswift:
ACK e1c582cbaa -- diff looks correct
Tree-SHA512: 18684abc1c02cf52d63f6f6ecd98df01a9574a7c470524c37e152296504e2e3ffbabd6f3208214b62031512aeb809a6d37446af82c9f480ff14ce4c42c98e7c2
fac86ac7b3 scripted-diff: Add missed copyright headers (Hennadii Stepanov)
6fde9d5e47 script: Update EXLUDE list in copyright_header.py (Hennadii Stepanov)
1998152f15 script: Add empty line after C++ copyright (Hennadii Stepanov)
071f2fc204 script: Add ability to insert copyright to *.sh (Hennadii Stepanov)
Pull request description:
This PR improves `contrib/devtools/copyright_header.py` script and adds copyright headers to the files in `src` and `test` directories with two exceptions:
- [`src/reverse_iterator.h`](https://github.com/bitcoin/bitcoin/blob/master/src/reverse_iterator.h) (added to exceptions)
- [`src/test/fuzz/FuzzedDataProvider.h`](https://github.com/bitcoin/bitcoin/blob/master/src/test/fuzz/FuzzedDataProvider.h) (added to exceptions)
On master 5622d8f315:
```
$ ./contrib/devtools/copyright_header.py report . | grep zero
25 with zero copyrights
```
With this PR:
```
$ ./contrib/devtools/copyright_header.py report . | grep zero
2 with zero copyrights
```
~I am uncertain about our copyright policy with `build_msvc` and `contrib` directories content, so they are out of scope of this PR.~
ACKs for top commit:
MarcoFalke:
ACK fac86ac7b3
Tree-SHA512: d7832c4a7a1a3b7806119775b40ec35d7982f49ff0e6199b8cee4c0e0a36e68d51728b6ee9924b1c161df4bc6105bd93391b79d42914357fa522f499cb113fa8
582e66b6e7 doc: Added regtest config for linearize script (Gr0kchain)
Pull request description:
Updated the example-linearize.cfg file to include support for the regtest chain network config which is used by the ./linearize-data.py
Problem:
Without the regtest magic, genesis hash and path config, the `linearize-data.py` script cannot generate a bootstrap.dat file.
Example:
./linearize-data.py ./linearize.cfg
Read 102 hashes
Genesis block not found in hashlist
Solution:
Added netmagic, genesis and input example parameters to file.
Resolution
1. Starting bitcoind in regtest mode
2. bitcoin-cli generatetoaddress 101 $(bitcoin-cli getnewaddress)
3. ./linearize-hashes.py ./linearize.cfg > ./hashlist.txt
4. ./linearize-data.py ./linearize.cfg
```
$ ./linearize-data.py ./linearize.cfg
Read 102 hashes
Input file /Users/gr0kchain/.bitcoin/regtest/blocks/blk00000.dat
Output file /Users/gr0kchain/Downloads/bootstrap.dat
Done (102 blocks written)
```
ACKs for top commit:
fanquake:
ACK 582e66b6e7
Tree-SHA512: 699e92e740e68e2e5190ba37538efbbe3e4d4e725ebd6af704a0cf5517683b691754f7ea097bf840845d2b53b793c63258d406e9bd37922db810cf58bed053c3
-BEGIN VERIFY SCRIPT-
s() { contrib/devtools/copyright_header.py insert "$1"; }
s build_msvc/bitcoin_config.h
s build_msvc/msvc-autogen.py
s build_msvc/testconsensus/testconsensus.cpp
s contrib/devtools/circular-dependencies.py
s contrib/devtools/gen-manpages.sh
s contrib/filter-lcov.py
s contrib/gitian-build.py
s contrib/install_db4.sh
s src/crypto/sha256_avx2.cpp
s src/crypto/sha256_sse41.cpp
s src/fs.cpp
s src/qt/test/addressbooktests.cpp
s src/qt/test/addressbooktests.h
s src/qt/test/util.cpp
s src/qt/test/util.h
s src/qt/test/wallettests.cpp
s src/qt/test/wallettests.h
s src/test/blockchain_tests.cpp
s test/functional/combine_logs.py
s test/lint/lint-locale-dependence.sh
sed -i '1G' test/lint/lint-shebang.sh
s test/lint/lint-shebang.sh
-END VERIFY SCRIPT-
c966ff14c7 gitian: fixed SC2001 regex (willyk)
Pull request description:
Currently the gitian-win-signer.yml produces OUTFILE names without `-unsigned` stripped out
This is due to regex having an`%` in front of it
```
$ INFILE="bitcoin-0.19.0-win64-setup-unsigned.exe"
$ echo "${INFILE/%-unsigned}"
bitcoin-0.19.0-win64-setup-unsigned.exe
$ echo "${INFILE/-unsigned}"
bitcoin-0.19.0-win64-setup.exe
```
Fixes#17361
ACKs for top commit:
laanwj:
ACK c966ff14c7
hebasto:
ACK c966ff14c7
Tree-SHA512: 954547f9dfa4cab4def5f284d4837c21f0e6fed7454a04e83e6b1b7d3fd3f9661ea657047f0d8162f6591909d32ef2f72e801b2f3a44cbb1131ac344cb913a69
Debian 8 (Jessie) has:
- g++ version 4.9.2
- libc version 2.19
Ubuntu 16.04.4 (Xenial) has:
- g++ version 5.3.1
- libc version 2.23.0
CentOS 7 has:
- g++ version 4.8.5
- libc version 2.17
Taking the minimum of these as our target.
According to the GNU ABI document this corresponds to:
- GCC 4.8.5: GCC_4.8.0
- (glibc) GLIBC_2_17
Co-Authored-By: fanquake <fanquake@gmail.com>
7d7bf2ff4a build: set minimum supported macOS to 10.12 (fanquake)
Pull request description:
Extracted from #16392 as this doesn't need to wait for the other build changes.
Reasoning:
* `10.10` has been unsupported since July 2017 (~3 years at `v0.20.0` release)
* `10.11` has been unsupported since July 2018 (~2 years at `v0.20.0` release)
* macOS users are consistent at upgrading to new releases.
* Qt 5.12 LTS only supports [macOS > 10.12](https://doc-snapshots.qt.io/qt5-5.12/supported-platforms.html). As long as we're supporting macOS < 10.12 we would not be able to bump Qt in depends to 5.12 for the `v0.20.0` release.
Once we drop support for 10.12 and start using the 10.15 SDK there are some other follow ups:
* Enabling support for [`thread_local`](https://github.com/bitcoin/bitcoin/blob/master/configure.ac#L901).
* Removing some of the macOS notification code
* Removing macOS [startup item code](https://github.com/bitcoin/bitcoin/blob/master/src/qt/guiutil.cpp#L695).
There was also some related discussion in the `#bitcoin-builds` channel yesterday arvo.
ACKs for top commit:
laanwj:
ACK 7d7bf2ff4a
dongcarl:
post-IRC-grilling-ACK 7d7bf2ff4a
Tree-SHA512: 30b0ed75e3b4df2d3f94db2091b7982c1ea7be8fb3a2732d8efd1a53ef1e492d7265a47e90c38e92f66e638d10e9400a6ecb56e5093688bb26d1621645b453e8
feb5075777 build: use osslsigncode 2.0 in gitian (fanquake)
Pull request description:
The original osslsigncode project, https://sourceforge.net/projects/osslsigncode, has been marked as abandonware:
> This is now - and has been for a long while - abandonware. Feel free to create your own forks etc.".
However, a fork has emerged, https://github.com/mtrojnar/osslsigncode, that has incorporated
theuni's patches ([add the -pem option in extract-signature mode ](36715c1183) & [add the attach-signature command](3be7eb1676)) as well as updated the tool to work with OpenSSL 1.1 and other improvements.
This commit switches the windows signer descriptor to use this new version of `osslsigncode`.
I've tested using this new version of `osslsigncode` while doing a 0.18.1 gitian build, and it "seems" to work. However this needs a look over from Cory, to check if the tool is still compatible with his usage in the [`detached-sig-create.sh`](https://github.com/bitcoin/bitcoin/blob/master/contrib/windeploy/detached-sig-create.sh) script, as well as some review of the changes to `osslsigncode` itself. Hence WIP and chasing Concept ACKs / NACKs.
ACKs for top commit:
MarcoFalke:
Concept ACK feb5075777 given that this upstream is now used in Ubuntu and Debian
laanwj:
ACK feb5075777
Tree-SHA512: c48de6dc32751d96dd04b920bfacca40af47a2883330ba0700371d56c580a7e45cedd8d8a913709d56be036762b63cb1825a98cff7aa77b6d7804fab11220850
0ec967164f Add TheCharlatan's pgp key (TheCharlatan)
Pull request description:
Since I have submitted my pair of signed gitian assertions, I am now adding my key's fingerprint to the gitian keys list.
ACKs for top commit:
jonasschnelli:
ACK 0ec967164f - confirm key A8FC55F3B04BA3146F3492E79303B33A305224CB
Tree-SHA512: bbd5e637186ed1659432e4fcc96bdc57fdbdb608325805701e06a51689726e722d7abeb11b5c9de723d051976d9d7ac23602316403fa74029dceb3cf1e837aea
Updated the example-linearize.cfg file to include support for the regtest chain network config which is used by the ./linearize-data.py
Problem:
Without the regtest magic, genesis hash and path config, the `linearize-data.py` script cannot generate a bootstrap.dat file.
Example of error:
./linearize-data.py ./linearize.cfg
Read 102 hashes
Genesis block not found in hashlist
Solution:
Added netmagic, genesis and input example parameters to file.
Resolution
1. Starting bitcoind in regtest mode
2. bitcoin-cli generatetoaddress 101 $(bitcoin-cli getnewaddress)
3. ./linearize-hashes.py ./linearize.cfg > ./hashlist.txt
4. ./linearize-data.py ./linearize.cfg
Example after fix:
$ ./linearize-data.py ./linearize.cfg
Read 102 hashes
Input file /Users/gr0kchain/.bitcoin/regtest/blocks/blk00000.dat
Output file /Users/gr0kchain/Downloads/bootstrap.dat
Done (102 blocks written)
92b2f5306b test: add dumptxoutset RPC test (James O'Beirne)
c1ccbc3dde devtools: add utxo_snapshot.sh (James O'Beirne)
57cf74c991 rpc: add dumptxoutset (James O'Beirne)
92fafb3a7d coinstats: add coins_count (James O'Beirne)
707fde7b9b add unused SnapshotMetadata class (James O'Beirne)
Pull request description:
This is part of the [assumeutxo project](https://github.com/bitcoin/bitcoin/projects/11):
Parent PR: #15606
Issue: #15605
Specification: https://github.com/jamesob/assumeutxo-docs/tree/master/proposal
---
This changeset defines the serialization format for UTXO snapshots and adds an RPC command for creating them, `dumptxoutset`. It also adds a convenience script for generating and verifying snapshots at a certain height, since that requires doing a hacky rewind of the chain via `invalidateblock`.
All of this is unused at the moment.
ACKs for top commit:
laanwj:
ACK 92b2f5306b
Tree-SHA512: 200dff87767f157d627e99506ec543465d9329860a6cd49363081619c437163a640a46d008faa92b1f44fd403bfc7a7c9e851c658b5a4849efa9a34ca976bf31
3b3b93174a nsis: Write to correct filename in first place (Carl Dong)
Pull request description:
Per MarcoFalke's suggestion here https://github.com/bitcoin/bitcoin/pull/17029#discussion_r333216722
ACKs for top commit:
MarcoFalke:
unsigned ACK 3b3b93174a, makes sense to name it that way because it will raise the "unsinged" error in Windows
Tree-SHA512: da72aae438505e162d0b3cd27d873b7ad8176178bb459a738e61b6e2ad0fa739d905b3109fab641bb1a3950fe59ad526c5568d12cf48a305166cdb7db6686543
The original osslsigncode project (https://sourceforge.net/projects/osslsigncode/) has been marked as abandonware,
"This is now - and has been for a long while - abandonware. Feel free to create your own forks etc.".
However, a fork at https://github.com/mtrojnar/osslsigncode has emerged that has incorporated
theuni's patches, updated the tool to work with OpenSSL 1.1 and made other improvements.
This commit switches the windows signer descriptor to use this new version of osslsigncode.
bd3f5a90ec build: remove mingw linker workaround from win gitian descriptor (fanquake)
Pull request description:
This workaround was added as part of the switch to gitian building using Ubuntu 14.04 (#6900).
However, it should no longer be required, as we have switched to Bionic (#13171) and that
has a far newer version of binutils.
Original discussion: https://github.com/bitcoin/bitcoin/pull/6900
binutils patch: https://sourceware.org/bugzilla/show_bug.cgi?id=16192
ACKs for top commit:
MarcoFalke:
ACK bd3f5a90ec
theuni:
ACK bd3f5a90ec
laanwj:
ACK bd3f5a90ec
Tree-SHA512: 01a5789994decf8cdedf7aaa0a449d2100a77e2e6b422d6b9dd5a4ac3e2e0b538c3d43aae4a1c3713614782f3c6b09d8d8bb21c20e86ce3c1734183dedd02d0c
9d1f971c75 gitian: Put things in the right place to begin with (Carl Dong)
71949a97a7 gitian: Eliminate rename dependency (Carl Dong)
999a9a5f5b gitian: Smaller diff with gitian-linux.yml (Carl Dong)
c4a3c25ba1 gitian: Fix README inclusion in archives (Carl Dong)
93cb974980 gitian: Use split-debug.sh for Win builds (Carl Dong)
Pull request description:
It would seem that our `gitian-win.yml` has not been keeping up with `gitian-linux.yml`, this PR:
1. Minimizes the diff size between `gitian-{win,linux}.yml`
2. Eliminates the `rename` dependency
ACKs for top commit:
laanwj:
ACK 9d1f971c75
Tree-SHA512: 84ed47c685e12d0064c02811907ae3d0fd3c47db8773d497dcc38f0defbfb3040fd82899fb026cf355f229b906d05a1c8038a95642bb90d044afbc2e0b239af2
Linux:
The README was originally added in 8550f1fb2, but included the README
under the docs directory, which has a bunch of internal links that won't
make sense in a release tarball. In this patch, we include the root
level README instead, which makes more sense.
Windows:
.md files are inconvenient to open on windows and the line endings
differ, so we use README_windows.txt instead.
