Commit Graph

1686 Commits

Author SHA1 Message Date
Hennadii Stepanov
5202bd1dc0
test: Bump shellcheck version to 0.8.0 2021-11-30 21:15:40 +02:00
W. J. van der Laan
c5712d172e
Merge bitcoin/bitcoin#23506: test: Make more shell scripts verifiable by the shellcheck tool
a3f61676e8 test: Make more shell scripts verifiable by the `shellcheck` tool (Hennadii Stepanov)

Pull request description:

  Some shell scripts from `contrib/guix` and `contrib/shell` are not verifiable by the `shellcheck` tool for the following reasons:
  - they have no extension (see 4eccf063b2 from bitcoin/bitcoin#21375)
  - they have the `.bash` extension while `.sh` is expected

  This PR adds these scripts to the input for the `shellcheck` tool, and it fixes discovered `shellcheck` warnings.

ACKs for top commit:
  dongcarl:
    Code Review ACK a3f61676e8, this is a good robustness improvement for our shell scripts.
  jamesob:
    crACK a3f61676e8

Tree-SHA512: 6703f5369d9c04c1a174491f381afa5ec2cc4d37321c1b93615abcdde4dfd3caae82868b699c25b72132d8c8c6f2e9cf24d38eb180ed4d0f0584d8c282e58935
2021-11-30 20:05:44 +01:00
Hennadii Stepanov
a3f61676e8
test: Make more shell scripts verifiable by the shellcheck tool 2021-11-28 14:08:54 +02:00
Arnab Sen
2bc51c5c32 [tracing] tracepoints to utxocache add, spent and uncache
Signed-off-by: Arnab Sen <arnabsen1729@gmail.com>
2021-11-28 11:51:21 +05:30
Arnab Sen
a26e8eef43 [tracing] tracepoint for utxocache flushes
Signed-off-by: Arnab Sen <arnabsen1729@gmail.com>
2021-11-28 11:34:44 +05:30
João Barbosa
655d52a0cd contrib: Specify wb mode when creating mac sdk
Fix the warning:
```
./contrib/macdeploy/gen-sdk:84: FutureWarning: GzipFile was opened for writing, but this will change in future Python releases.  Specify the mode argument for opening it for writing.
```
2021-11-25 21:38:34 +00:00
Nathan Garabedian
7bb8eb0bc3 script install_db4.sh added check for patch command 2021-11-23 07:24:35 -08:00
fanquake
f31d4bd214
scripts: test for PE control flow instrumentation 2021-11-17 16:12:47 +08:00
fanquake
dbfca4a815
build: require glibc 2.18+ for release builds
From what I can see the only platform this drops support for is CentOS
7. CentOS 7 reached the end of it's "full update" support at the end of
2020. It does receive maintenance updates until 2024, however I don't
think supporting glibc 2.17 until 2024 is realistic. Note that anyone
wanting to self-compile and target a glibc 2.17 runtime could build with
--disable-threadlocal.

glibc 2.18 was released in August 2013.
https://sourceware.org/legacy-ml/libc-alpha/2013-08/msg00160.html
2021-11-16 07:51:24 +08:00
W. J. van der Laan
aec631bccc
Merge bitcoin/bitcoin#23462: test: Enable SC2046 and SC2086 shellcheck rules
fe0ff569ea test: Enable SC2046 shellcheck rule (Hennadii Stepanov)
9a1ad7bc0d test: Enable SC2086 shellcheck rule (Hennadii Stepanov)

Pull request description:

  Closes #20879.
  Replaces #22695.

  **Note for reviewers**. Some touched shell scripts are not being run in CI, therefore they require more thorough reviewing:
  - `contrib/devtools/gen-manpages.sh`
  - `contrib/macdeploy/detached-sig-apply.sh`
  - `contrib/windeploy/detached-sig-create.sh`
  - `src/qt/res/animation/makespinner.sh`

ACKs for top commit:
  laanwj:
    Code review re-ACK fe0ff569ea

Tree-SHA512: 73619b9a7bcb6cf0dfc4189a753ef550d40c82a3432bb9d8d8a994310d42594576038daac7e0c2fc004d716976bb1413b9a77848ecf088b25b69ed0773b77e8e
2021-11-15 16:22:52 +01:00
W. J. van der Laan
a85442f62b
Merge bitcoin/bitcoin#22768: script: Add commits signed with sipas expired key to allow-revsig-commits
365f35481d script: Add commits signed with sipas expired key to allow-revsig-commits (nthumann)

Pull request description:

  Fixes https://github.com/bitcoin/bitcoin/issues/22737.
  While investigating the issue above, I noticed that there are 141 commits that are signed with sipas expired key.
  To allow `./contrib/verify-commits/verify-commits.py` to succeed, this PR adds them to `allow-revsig-commits`.
  Feel free to confirm that they're indeed signed with an expired key using e.g. `git show --show-signature d8cd7b137fb075616f31d2b43b85fa2e27ea7477` :)

ACKs for top commit:
  laanwj:
    Code review ACK 365f35481d

Tree-SHA512: 860e372c5314714c6c205cd234ebec89756c9ade43a2ed65ed25575ae0a0d4d8dd7cf43692a5b267abe742f87e5cba0a3f1fb76a5fed7b1615ea2859902dfcdf
2021-11-14 15:25:39 +01:00
Hennadii Stepanov
fe0ff569ea
test: Enable SC2046 shellcheck rule 2021-11-13 18:05:26 +02:00
Hennadii Stepanov
9a1ad7bc0d
test: Enable SC2086 shellcheck rule 2021-11-13 16:54:56 +02:00
fanquake
c1fb30633b
Merge bitcoin/bitcoin#23114: Add minisketch subtree and integrate into build/test
29173d6c6c ubsan: add minisketch exceptions (Cory Fields)
54b5e1aeab Add thin Minisketch wrapper to pick best implementation (Pieter Wuille)
ee9dc71c1b Add basic minisketch tests (Pieter Wuille)
0659f12b13 Add minisketch dependency (Gleb Naumenko)
0eb7928ab8 Add MSVC build configuration for libminisketch (Pieter Wuille)
8bc166d5b1 build: add minisketch build file and include it (Cory Fields)
b2904ceb85 build: add configure checks for minisketch (Cory Fields)
b6487dc4ef Squashed 'src/minisketch/' content from commit 89629eb2c7 (fanquake)

Pull request description:

  This takes over #21859, which has [recently switched](https://github.com/bitcoin/bitcoin/pull/21859#issuecomment-921899200) to my integration branch. A few more build issues came up (and have been fixed) since, and after discussing with sipa it was decided I would open a PR to shepherd any final changes through.

  > This adds a `src/minisketch` subtree, taken from the master branch of https://github.com/sipa/minisketch, to prepare for Erlay implementation (see #21515). It gets configured for just supporting 32-bit fields (the only ones we're interested in in the context of Erlay), and some code on top is added:
  > * A very basic unit test (just to make sure compilation & running works; actual correctness checking is done through minisketch's own tests).
  > * A wrapper in `minisketchwrapper.{cpp,h}` that runs a benchmark to determine which field implementation to use.

  Only changes since my last update to the branch in the previous PR have been rebasing on master and fixing an issue with a header in an introduced file.

ACKs for top commit:
  naumenkogs:
    ACK 29173d6c6c

Tree-SHA512: 1217d3228db1dd0de12c2919314e1c3626c18a416cf6291fec99d37e34fb6eec8e28d9e9fb935f8590273b8836cbadac313a15f05b4fd9f9d3024c8ce2c80d02
2021-11-12 10:00:49 +08:00
Vasil Dimov
420695c193
contrib: recognize CJDNS seeds as such
An IPv6 address from fc00::/8 could be either from the CJDNS network or
from a private-unroutable-reserved segment of IPv6. A seed node with
such an address must be from the CJDNS network, otherwise other peers
will not be able to connect to it.
2021-11-03 14:58:55 +01:00
Gleb Naumenko
0659f12b13
Add minisketch dependency 2021-10-21 09:38:55 +08:00
fanquake
a7f28af437
Merge bitcoin/bitcoin#22646: build: tighter Univalue integration, remove --with-system-univalue
0f95247246 Integrate univalue into our buildsystem (Cory Fields)
9b49ed656f Squashed 'src/univalue/' changes from 98fadc0909..a44caf65fe (fanquake)

Pull request description:

  This PR more tightly integrates building Univalue into our build system. This follows the same approach we use for [LevelDB](https://github.com/bitcoin-core/leveldb/), ([`Makefile.leveldb.include`](https://github.com/bitcoin/bitcoin/blob/master/src/Makefile.leveldb.include)), and [CRC32C](https://github.com/bitcoin-core/crc32c) ([`Makefile.crc32c.include`](https://github.com/bitcoin/bitcoin/blob/master/src/Makefile.crc32c.include)), and will be the same approach we use for [minisketch](https://github.com/sipa/minisketch); see #23114.

  This approach yields a number of benefits, including:
  * Faster configuration due to one less subconfigure being run during `./configure` i.e 22s with this PR vs 26s
  * Faster autoconf i.e 13s with this PR vs 17s
  * Improved caching
  * No more issues with compiler flags i.e https://github.com/bitcoin/bitcoin/pull/12467
  * More direct control means we can build exactly the objects we want

  There might be one argument against making this change, which is that builders should have the option to use "proper shared/system libraries". However, I think that falls down for a few reasons. The first being that we already don't support building with a number of system libraries (secp256k1, leveldb, crc32c); some for good reason. Univalue is really the odd one out at the moment.

  Note that the only fork of Core I'm aware of, that actively patches in support for using system libs, also explicitly marks them as ["DANGEROUS"](a886811721/configure.ac (L1430)) and ["NOT SUPPORTED"](a886811721/configure.ac (L1312)). So it would seem they exist more to satisfy a distro requirement, as opposed to something that anyone should, or would actually use in practice.

  PRs like #22412 highlight the "issue" with us operating with our own Univalue fork, where we actively fix bugs, and make improvements, when upstream (https://github.com/jgarzik/univalue) may not be taking those improvements, and by all accounts, is not currently actively maintained. Bitcoin Core should not be hamstrung into not being able to fix bugs in a library, and/or have to litter our source with "workarounds", i.e #22412, for bugs we've already fixed, based on the fact that an upstream project is not actively being maintained. Allowing builders to use system libs is really only exacerbating this problem, with little benefit to our project. Bitcoin Core is not quite like your average piece of distro packaged software.

  There is the potential for us to give the same treatment to libsecp256k1, however it seems doing that is currently less straightforward.

ACKs for top commit:
  dongcarl:
    ACK 0f95247246 less my comment above, always nice to have an include-able `sources.mk` which makes integration easier.
  theuni:
    ACK 0f95247246. Thanks fanquake for keeping this going.

Tree-SHA512: a7f2e41ee7cba06ae72388638e86b264eca1b9a8b81c15d1d7b45df960c88c3b91578b4ade020f8cc61d75cf8d16914575f9a78fa4cef9c12be63504ed804b99
2021-10-20 11:01:38 +08:00
0xb10c
53c9fa9e62
tracing: drop block_connected hash.toString() arg
The tracepoint `validation:block_connected` was introduced in #22006.
The first argument was the hash of the connected block as a pointer
to a C-like String. The last argument passed the hash of the
connected block as a pointer to 32 bytes. The hash was only passed as
string to allow `bpftrace` scripts to print the hash. It was
(incorrectly) assumed that `bpftrace` cannot hex-format and print the
block hash given only the hash as bytes.

The block hash can be printed in `bpftrace` by calling
`printf("%02x")` for each byte of the hash in an `unroll () {...}`.
By starting from the last byte of the hash, it can be printed in
big-endian (the block-explorer format).

```C
  $p = $hash + 31;
  unroll(32) {
      $b = *(uint8*)$p;
      printf("%02x", $b);
      $p -= 1;
  }
```

See also: https://github.com/bitcoin/bitcoin/pull/22902#discussion_r705176691

This is a breaking change to the block_connected tracepoint API, however
this tracepoint has not yet been included in a release.
2021-10-18 14:35:25 +02:00
josibake
a46f71bb70
lint: enable mypy checking for missing imports
Achieve this by adding some ignore, and making data/ importable.

Co-authored-by: João Barbosa <joao.paulo.barbosa@gmail.com>
2021-10-16 09:14:37 +08:00
Carl Dong
1527b7e8a1 symbol-check: Check requested ELF interpreter
It is important that binaries request a standard interpreter location
where most distros would place the linker-loader. Otherwise, the user
would be met with a very confusing message:

    bash: <path>/<to>/bitcoind: No such file or directory

When really it's the interpreter that's not found.
2021-10-13 08:39:48 -04:00
Carl Dong
b96adcbfae guix: Fix powerpc64(le) dynamic linker name
I used Guix's values for the powerpc64(le) dynamic linkers, and the
/lib-prefix seems to be a Guix-ism rather than standard. The standard
path for the linker-loaders start with /lib64.

I've taken the new loader values from SYSDEP_KNOWN_INTERPRETER_NAMES in
glibc's sysdeps/unix/sysv/linux/powerpc/ldconfig.h file.

For future reference, loader path values can also be found on glibc's
website: https://sourceware.org/glibc/wiki/ABIList?action=recall&rev=16
2021-10-13 08:38:40 -04:00
fanquake
ce69e18947
scripts: remove pixie.py 2021-10-12 08:36:21 +08:00
fanquake
00b85d0b13
scripts: only parse the binary once in security-check.py 2021-10-12 08:36:21 +08:00
fanquake
cad40a5b16
scripts: use LIEF for ELF checks in security-check.py 2021-10-12 08:36:21 +08:00
fanquake
8242ae230e
scripts: only parse the binary once in symbol-check.py 2021-10-12 08:36:21 +08:00
fanquake
309eac9019
scripts: use LIEF for ELF checks in symbol-check.py
Co-authored-by: Carl Dong <contact@carldong.me>
2021-10-12 08:36:15 +08:00
Carl Dong
610a8a8e39
test-*-check: Pass in *FLAGS and compile with them
These test-*-check scripts should compile "test" binaries in a way that
is as close to what autotools would do, since the goal is to make sure
that if we run the *-check script, they can correctly detect flaws in
binaries which are compiled by our autotools-based system.

Therefore, we should emulate what happens when the binary is linked in
autotools, meaning that for C binaries, we need to supply the CFLAGS,
CPPFLAGS, and LDFLAGS flags in that order.

Note to future developers: perhaps it'd be nice to have these
test-*-check scripts be part of configure.ac to avoid having to manually
replicate autoconf-like behaviour every time we find a discrepancy. Of
course, that would also mean you'd have to write more m4...
2021-10-12 08:28:25 +08:00
Cory Fields
0f95247246
Integrate univalue into our buildsystem
This addresses issues like the one in #12467, where some of our compiler flags
end up being dropped during the subconfigure of Univalue. Specifically, we're
still using the compiler-default c++ version rather than forcing c++17.

We can drop the need subconfigure completely in favor of a tighter build
integration, where the sources are listed separately from the build recipes,
so that they may be included directly by upstream projects. This is
similar to the way leveldb build integration works in Core.

Core benefits of this approach include:
- Better caching (for ex. ccache and autoconf)
- No need for a slow subconfigure
- Faster autoconf
- No more missing compile flags
- Compile only the objects needed

There are no benefits to Univalue itself that I can think of. These changes
should be a no-op there, and to downstreams as well until they take advantage
of the new sources.mk.

This also removes the option to use an external univalue to avoid similar ABI
issues with mystery binaries.

Co-authored-by: fanquake <fanquake@gmail.com>
2021-10-11 20:46:25 +08:00
W. J. van der Laan
a6d9675d60 contrib: Ask for captcha in signet getcoins.py
As the faucet will always ask for a captcha now, the current script is
no longer usable.

Change the script to print the captcha in dot-matrix to the terminal,
using unicode Braille characters.
2021-10-02 05:39:58 +02:00
fanquake
8f022a59b8
Merge bitcoin/bitcoin#22993: build: set OSX_MIN_VERSION to 10.15
a43b8e9555 build: set OSX_MIN_VERSION to 10.15 (fanquake)

Pull request description:

  Taken out of #20744, as splitting up some of the build changes was mentioned [here](https://github.com/bitcoin/bitcoin/pull/22937#discussion_r707303172).

  This is required to use `std::filesystem` on macOS, as support for it only landed in the libc++.dylib shipped with 10.15. So if we want to move to using `std::filesystem` for `23.0`, this bump is required.

  See also: https://developer.apple.com/documentation/xcode-release-notes/xcode-11-release-notes

  > Clang now supports the C++17 \<filesystem\> library for iOS 13, macOS 10.15, watchOS 6, and tvOS 13.

  macOS 10.15 was released in October 2019. macOS OS's seem to have a life of about 3 years, so it's possible that 10.14 will become officially unsupported by the end of 2021 and prior to the release of 23.0.

  Guix builds:
  ```bash
  bash-5.1# find guix-build-$(git rev-parse --short=12 HEAD)/output/ -type f -print0 | env LC_ALL=C sort -z | xargs -r0 sha256sum
  abc8b749be65f1339dcdf44bd1ed6ade2533b8e3b5030ad1dde0ae0cede78136  guix-build-a43b8e955558/output/dist-archive/bitcoin-a43b8e955558.tar.gz
  1edcc301eb4c02f3baa379beb8d4c78e661abc24a293813bc9d900cf7255b790  guix-build-a43b8e955558/output/x86_64-apple-darwin19/SHA256SUMS.part
  e9dbb5594a664519da778dde9ed861c3f0f631525672e17a67eeda599f16ff44  guix-build-a43b8e955558/output/x86_64-apple-darwin19/bitcoin-a43b8e955558-osx-unsigned.dmg
  11b23a17c630dddc7594c25625eea3de42db50f355733b9ce9ade2d8eba3a8f3  guix-build-a43b8e955558/output/x86_64-apple-darwin19/bitcoin-a43b8e955558-osx-unsigned.tar.gz
  257ba64a327927f94d9aa0a68da3a2695cf880b3ed1a0113c5a966dcc426eb5e  guix-build-a43b8e955558/output/x86_64-apple-darwin19/bitcoin-a43b8e955558-osx64.tar.gz
  ```

ACKs for top commit:
  hebasto:
    ACK a43b8e9555
  jarolrod:
    ACK a43b8e9

Tree-SHA512: 9ac77be7cb56c068578860a3b2b8b7487c9e18b71b14aedd77a9c663f5d4bb19756d551770c02ddd12f1797beea5757b261588e7b67fb53509bb998ee8022369
2021-09-21 15:37:12 +08:00
Jeremy Rand
9bdda50151
Enable TLS in links in documentation 2021-09-16 22:00:20 +00:00
fanquake
a43b8e9555
build: set OSX_MIN_VERSION to 10.15
This is required to use std::filesystem on macOS as support for it only
landed in the libc++ dylib shipped with 10.15.

See also: https://developer.apple.com/documentation/xcode-release-notes/xcode-11-release-notes

Clang now supports the C++17 <filesystem> library for iOS 13, macOS 10.15, watchOS 6, and tvOS 13.
2021-09-16 17:50:19 +08:00
MarcoFalke
245462b66c
Merge bitcoin/bitcoin#22418: release: Remove gitian
ab9c34237a release: remove gitian (fanquake)

Pull request description:

  Note that this doesn't yet touch any glibc back compat related code.

ACKs for top commit:
  laanwj:
    Code review ACK ab9c34237a

Tree-SHA512: 8e2fe3ec1097f54bb11ab9136b43818d90eab5dbb0a663ad6a552966ada4bdb49cc12ff4e66f0ec0ec5400bda5c81f3a3ce70a9ebb6fe1e0db612da9f00a51a7
2021-09-02 10:09:53 +02:00
fanquake
9487b68f06
Merge bitcoin/bitcoin#22847: guix/prelude: Override VERSION with FORCE_VERSION
96cc6bb04f guix/prelude: Override VERSION with FORCE_VERSION (Carl Dong)

Pull request description:

  ```
  Previously, if the builder exported $VERSION in their environment (as
  past Gitian-building docs told them to), but their HEAD does not
  actually point to v$VERSION, their build outputs will differ from those
  of other builders.

  This is because the contrib/guix/guix-* scripts only ever act on the
  current git worktree, and does not try to check out $VERSION if $VERSION
  is set in the environment.

  Setting $VERSION only makes the scripts pretend like the current
  worktree is $VERSION.

  This problem was seen in jonatack's attestation for all.SHA256SUMS,
  where only his bitcoin-22.0rc3-osx-signed.dmg differed from everyone
  else's.

  Here is my deduced sequence of events:

  1. Aug 27th: He guix-builds 22.0rc3 and uploads his attestations up to
     guix.sigs

  2. Aug 30th, sometime after POSIX time 1630310848: he pulls the latest
     changes from master in the same worktree where he guix-built 22.0rc3
     and ends up at 7be143a960

  3. Aug 30th, sometime before POSIX time 1630315907: With his worktree
     still on 7be143a960, he guix-codesigns. Normally, this would result
     in outputs going in guix-build-7be143a960e2, but he had
     VERSION=22.0rc3 in his environment, so the guix-* scripts pretended
     like he was building 22.0rc3, and used 22.0rc3's guix-build directory
     to locate un-codesigned outputs and dump codesigned ones.

     However, our SOURCE_DATE_EPOCH defaults to the POSIX time of HEAD
     (7be143a960), which made all timestamps in the resulting codesigned
     DMG 1630310848, 7be143a960e2's POSIX timestamp. This differs from the
     POSIX timestamp of 22.0rc3, which is 1630348517. Note that the
     windows codesigning procedure does not consider SOURCE_DATE_EPOCH.

  We resolve this by only allowing VERSION overrides via the FORCE_VERSION
  environment variable.
  ```

  Please ignore the branch name, it's not relevant to the change.

ACKs for top commit:
  fanquake:
    ACK 96cc6bb04f  - Also makes sense given there are Guix build guides recommending to set `VERSION` as part of the process. i.e https://gist.github.com/hebasto/7293726cbfcd0b58e1cfd5418316cee3.

Tree-SHA512: 9dca3fc637ce11049286a3ebee3cd61cce2125fc51d31cf472fbed7f659e1846fc44062753e0e71bfaec9e7fbab6f040bb88d9d4bc4f8acb28c6890563584acf
2021-09-02 09:35:15 +08:00
Carl Dong
96cc6bb04f guix/prelude: Override VERSION with FORCE_VERSION
Previously, if the builder exported $VERSION in their environment (as
past Gitian-building docs told them to), but their HEAD does not
actually point to v$VERSION, their build outputs will differ from those
of other builders.

This is because the contrib/guix/guix-* scripts only ever act on the
current git worktree, and does not try to check out $VERSION if $VERSION
is set in the environment.

Setting $VERSION only makes the scripts pretend like the current
worktree is $VERSION.

This problem was seen in jonatack's attestation for all.SHA256SUMS,
where only his bitcoin-22.0rc3-osx-signed.dmg differed from everyone
else's.

Here is my deduced sequence of events:

1. Aug 27th: He guix-builds 22.0rc3 and uploads his attestations up to
   guix.sigs

2. Aug 30th, sometime after POSIX time 1630310848: he pulls the latest
   changes from master in the same worktree where he guix-built 22.0rc3
   and ends up at 7be143a960

3. Aug 30th, sometime before POSIX time 1630315907: With his worktree
   still on 7be143a960, he guix-codesigns. Normally, this would result
   in outputs going in guix-build-7be143a960e2, but he had
   VERSION=22.0rc3 in his environment, so the guix-* scripts pretended
   like he was building 22.0rc3, and used 22.0rc3's guix-build directory
   to locate un-codesigned outputs and dump codesigned ones.

   However, our SOURCE_DATE_EPOCH defaults to the POSIX time of HEAD
   (7be143a960), which made all timestamps in the resulting codesigned
   DMG 1630310848, 7be143a960e2's POSIX timestamp. This differs from the
   POSIX timestamp of 22.0rc3, which is 1630348517. Note that the
   windows codesigning procedure does not consider SOURCE_DATE_EPOCH.

We resolve this by only allowing VERSION overrides via the FORCE_VERSION
environment variable.
2021-08-31 10:57:04 -04:00
Kristaps Kaupe
9d43045b74
builder-keys: add kristapsk
https://keys.openpgp.org/search?q=70A1D47DD44F59DF8B22244333E472FE870C7E5D
https://github.com/JoinMarket-Org/joinmarket-clientserver/blob/master/pubkeys/KristapsKaupe.asc
2021-08-31 13:41:32 +03:00
fanquake
ab9c34237a
release: remove gitian 2021-08-31 09:37:23 +08:00
MarcoFalke
718d9f2f77
Merge bitcoin/bitcoin#22660: contrib: catch bitcoin-cli RPC call errors in getcoins.py
42dbd9025a contrib: return non-zero status if getcoins.py errors (Sebastian Falbesoner)
8c203cf0e1 contrib: catch bitcoin-cli RPC call errors in getcoins.py (Sebastian Falbesoner)
0eca5ebace contrib: refactor: introduce bitcoin-cli RPC call helper in getcoins.py (Sebastian Falbesoner)

Pull request description:

  This PR is based on #22565 ("[script] signet's getcoins.py improvements"), which should be reviewed first.

  The signet faucet script `contrib/signet/getcoins.py` currently issues bitcoin-cli RPC calls without catching errors -- the only case tackled is if there is no `bitcoin-cli` file found. Instead of crashing with a stack-trace on a failed RPC call, the changes in this PR aim to produce a more user-friendly output (see also https://github.com/bitcoin/bitcoin/pull/22565#discussion_r683754875). Additionally, in case of any error, a non-zero status is now returned (instead of 0, indicating success), which could be useful for other scripts taking use of signet faucet script.

  The most straight-forward way to test this is invoking the script without a `bitcoind` running on signet:

  PR22565 branch:
  ```
  $ ./contrib/signet/getcoins.py
  error: Could not connect to the server 127.0.0.1:8332

  Make sure the bitcoind server is running and that you are connecting to the correct RPC port.
  Traceback (most recent call last):
    File "./contrib/signet/getcoins.py", line 26, in <module>
      curr_signet_hash = subprocess.check_output([args.cmd] + args.bitcoin_cli_args + ['getblockhash', '1']).strip().decode()
    File "/usr/local/lib/python3.8/subprocess.py", line 415, in check_output
      return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
    File "/usr/local/lib/python3.8/subprocess.py", line 516, in run
      raise CalledProcessError(retcode, process.args,
  subprocess.CalledProcessError: Command '['bitcoin-cli', 'getblockhash', '1']' returned non-zero exit status 1.
  ```

  this PR branch:
  ```
  $ ./contrib/signet/getcoins.py
  error: Could not connect to the server 127.0.0.1:38332

  Make sure the bitcoind server is running and that you are connecting to the correct RPC port.
  -----
  Error while calling "bitcoin-cli -signet getblockhash 1" (see output above).
  ```

ACKs for top commit:
  kallewoof:
    Code ACK 42dbd9025a
  Zero-1729:
    tACK 42dbd90 🧪

Tree-SHA512: 912240a4ed03c87035e370602f4095c7ffe26806421bbbd6cf86588126f2310a01a6a61606e9e2918fb2c1a0debdd0ce768c69ba2e4b8e7750fa3474a56d01a0
2021-08-26 10:04:57 +02:00
nthumann
365f35481d
script: Add commits signed with sipas expired key to allow-revsig-commits 2021-08-22 11:52:21 +02:00
W. J. van der Laan
1bbe289f20
Merge bitcoin/bitcoin#22565: [script] signet's getcoins.py improvements
b0c8246cac Add cleaner errors for unsuccessful faucet transactions (NikhilBartwal)
1c612b274b [script] Update signet getcoins.py for custom network (NikhilBartwal)

Pull request description:

  Currently, using the getcoins.py with a custom signet executes successfully and shows the transfer of 0.001 testBTC as complete, however for obvious reasons, it should not. In fact, upon verification it does not actually execute the transaction, but rather gives the output that it did, as shown below which can be misleading:

  ```
  [nikhilb@nikhil-PC bitcoin]$ echo $datadir
  /home/nikhilb/signet-custom
  [nikhilb@nikhil-PC bitcoin]$ contrib/signet/getcoins.py -- -datadir=$datadir
  Payment of 0.00100000 BTC sent with txid dd22c7d996e95f3e5baf20f73140d517ff48f1b26d0e4fefd61e3c37991b8f86
  [nikhilb@nikhil-PC bitcoin]$ bitcoin-cli -datadir=$datadir getrawtransaction dd22c7d996e95f3e5baf20f73140d517ff48f1b26d0e4fefd61e3c37991b8f86
  error code: -5
  error message:
  No such mempool or blockchain transaction. Use gettransaction for wallet transactions.
  [nikhilb@nikhil-PC bitcoin]$ bitcoin-cli -datadir=$datadir gettransaction dd22c7d996e95f3e5baf20f73140d517ff48f1b26d0e4fefd61e3c37991b8f86
  error code: -5
  error message:
  Invalid or non-wallet transaction id

  ```
  This PR adds a sanity check for custom signet by comparing the current network's first block hash (the block after the genesis block) with global signet's respective block hash (since all signet networks share the same genesis block) and if a custom network is detected, the user is prompted to either work on the global signet or setup their own faucet.

  The PR was checked to be working successfully, giving the output as below:

  ```
  [nikhilb@nikhil-PC bitcoin]$ git checkout update_signet_getcoins
  Switched to branch 'update_signet_getcoins'
  Your branch is ahead of 'upstream/master' by 1 commit.
    (use "git push" to publish your local commits)
  [nikhilb@nikhil-PC bitcoin]$ contrib/signet/getcoins.py -- -datadir=$datadir
  The global faucet cannot be used with a custom Signet network. Please use the global signet or setup your custom faucet for the same.

  You can have a look here for setting up your own faucet: https://en.bitcoin.it/wiki/Signet

  ```

ACKs for top commit:
  prayank23:
    utACK b0c8246cac
  kallewoof:
    ACK b0c8246cac
  arnabsen1729:
    utACK b0c8246
  prakash1512:
    utACK b0c8246
  0xB10C:
    Tested ACK b0c8246cac
  theStack:
    Tested ACK b0c8246cac
  Zero-1729:
    crACK b0c8246 🧉

Tree-SHA512: 144b47a83008521a5cda13f4c1b12809a125a744f865a8e0f792132d52fdb88926d4f4f4d7230452c2e129b5879892cdbeda981b8af10b789e9fc0cda2905a5d
2021-08-23 19:19:00 +02:00
fanquake
d3169349a9
Merge bitcoin/bitcoin#22654: guix: Don't include directory name in SHA256SUMS
132cae44f2 doc: Mention the flat directory structure for uploads (Andrew Chow)
fb17c99e35 guix: Don't include directory name in SHA256SUMS (Andrew Chow)

Pull request description:

  The SHA256SUMS file can be used in a sha256sum -c command to verify downloaded binaries. However users are likely to download just a single file and not place this file in the correct directory relative to the SHA256SUMS file for the simple verification command to work. By not including the directory name in the SHA256SUMS file, it will be easier for users to verify downloaded binaries.

ACKs for top commit:
  Zero-1729:
    re-ACK 132cae44f2
  fanquake:
    ACK 132cae44f2

Tree-SHA512: c9ff416b8dfb2f3ceaf4d63afb84aac9fcaefbbf9092f9e095061b472884ec92c7a809e6530c7132a82cfe3ab115a7328e47994a412072e1d4feb26fc502c8c5
2021-08-20 15:10:26 +08:00
Andrew Chow
fb17c99e35 guix: Don't include directory name in SHA256SUMS
The SHA256SUMS file can be used in a sha256sum -c command to verify
downloaded binaries. However users are likely to download just a single
file and not place this file in the correct directory relative to the
SHA256SUMS file for the simple verification command to work. By not
including the directory name in the SHA256SUMS file, it will be easier
for users to verify downloaded binaries.

Co-authored-by: Carl Dong <contact@carldong.me>
2021-08-18 20:07:32 -04:00
MarcoFalke
f5a406f003
Merge bitcoin/bitcoin#22633: refactor: Replace remaining binascii method calls
021daedfa1 refactor: replace remaining binascii method calls (Zero-1729)

Pull request description:

  This PR removes the remaining `binascii` method calls outside `test/functional` and `test_framework`, as pointed out here  https://github.com/bitcoin/bitcoin/pull/22619#pullrequestreview-722153458.

  Follow-up to #22593 and #22619
  Closes #22605

ACKs for top commit:
  josibake:
    re-ACK 021daedfa1
  theStack:
    re-ACK 021daedfa1

Tree-SHA512: 2ae9fee8917112c91a5406f219ca70f24cd8902b903db5a61fc2de85ad640d669a772f5c05970be0fcee6ef1cdd32fae2ca5d1ec6dc9798b43352c8160ddde6f
2021-08-21 16:29:55 +02:00
W. J. van der Laan
90499358e9
Merge bitcoin/bitcoin#22645: scripts: prevent GCC optimising test symbols in test-symbol-check
5449d44e37 scripts: prevent GCC optimising test symbols in test-symbol-check (fanquake)

Pull request description:

  I noticed in #22381 that when the test-symbol-check target was being built with Clang and run in the CI it would fail due to using a too-new version of `pow` (used [here](d67330d112/contrib/devtools/test-symbol-check.py (L85))). Our CIs use Focal (glibc 2.31) and the version of `pow` was the optimized version introduced in [glibc 2.29](https://lwn.net/Articles/778286/):
  ```bash
  * Optimized generic exp, exp2, log, log2, pow, sinf, cosf, sincosf and tanf.
  ```
  This made sense, except for that if it was failing when built using Clang, why hadn't it also been failing when being built with GCC?

  Turns out GCC is optimizing away that call to `pow` at all optimization levels, including `-O0`, see: https://godbolt.org/z/53MhzMxT7, and this has been the case forever, or at least since GCC 5.x. Clang on the other hand, will only optimize away the `pow` call at `-O1` and `-O2`, not `-O0`: https://godbolt.org/z/Wbnqj3q6c. Thus when this test was built with Clang (we don't pass `-O` so we default to `-O0`) it was failing in the CI environment, because it would actually have a call to the "new" `pow`.

  Avoid this issue by using a symbol that won't be optimized away, or that we are unlikely to ever have versioning issues with.

ACKs for top commit:
  laanwj:
    ACK 5449d44e37

Tree-SHA512: 3a26c5c3a5f2905fd0dd90892470e241ba625c0af3be2629d06d5da3a97534c1d6a55b796bbdd41e2e6a26a8fab7d981b98c45d4238565b0eb7edf3c5da02007
2021-08-18 17:16:11 +02:00
Zero-1729
021daedfa1
refactor: replace remaining binascii method calls 2021-08-16 19:30:56 +01:00
fanquake
4c43b7d41d
contrib: use hkps://keys.openpgp.org to retrieve builder keys
hkps://hkps.pool.sks-keyservers.net is essentially no-longer functional,
and a number of distributions and GPG tools have since switched to using
this key server as their default.

See this Debian patch for additional context:
https://salsa.debian.org/debian/gnupg2/-/blob/debian/main/debian/patches/Use-hkps-keys.openpgp.org-as-the-default-keyserver.patch

Switch to using keys.openpgp.org in the CI as well.
2021-08-17 08:59:04 +08:00
fanquake
11c7d001a9
Merge bitcoin/bitcoin#22700: builder-keys: Add dongcarl
7d95777417 builder-keys: Add dongcarl (Carl Dong)

Pull request description:

  https://keys.openpgp.org/search?q=04017A2A6D9A0CCDC81D8EC296AB007F1A7ED999

  This is my master key, will be bumping the expiration of subkeys or rotating when necessary.

ACKs for top commit:
  fanquake:
    ACK 7d95777417 - matches what I've got.

Tree-SHA512: 3a76b8eda81821b3221402501cf8191bce73118624b932aa80a7fc1a32a91e3825aeb2b03ed261bbf284b088e927c384f92e08eadddf7f94ed4de579d9f6d2b7
2021-08-16 10:58:55 +08:00
Carl Dong
7d95777417 builder-keys: Add dongcarl 2021-08-13 15:43:53 -04:00
James O'Beirne
45babb2788
builder-keys: add jamesob
http://keyserver.ubuntu.com/pks/lookup?search=0x25F27A38A47AD566&fingerprint=on&hash=on&op=vindex

This is also the key I sign GitHub commits with.
2021-08-12 16:16:56 -04:00
fanquake
8193294cab
Merge bitcoin/bitcoin#22642: release: Release with separate SHA256SUMS and sig files
90b3e482e9 release: Release with separate SHA256SUMS and sig files (Carl Dong)

Pull request description:

  This allows us to:
  - remove the rfc4880 EOL hacks, and
  - release with a SHA256SUMS.asc file that's a combination of all signer signatures

ACKs for top commit:
  achow101:
    ACK 90b3e482e9
  laanwj:
    Concept and code review ACK 90b3e482e9

Tree-SHA512: 5d5086063d303aa0cbd590e5fdf2ae8f555e25f4e43bf67545e33384449b990e94834c711622530ad0eb3dcc83f52746884a5081dadb0acff8dd799cfadafac7
2021-08-09 16:19:49 +08:00