8746600eec Merge bitcoin-core/secp256k1#1093: hash: Make code agnostic of endianness
37d36927df tests: Add tests for _read_be32 and _write_be32
912b7ccc44 Merge bitcoin-core/secp256k1#1094: doc: Clarify configure flags for optional modules
55512d30b7 doc: clean up module help text in configure.ac
d9d94a9969 doc: mention optional modules in README
616b43dd3b util: Remove endianness detection
8d89b9e6e5 hash: Make code agnostic of endianness
d0ad5814a5 Merge bitcoin-core/secp256k1#995: build: stop treating schnorrsig, extrakeys modules as experimental
1ac7e31c5b Merge bitcoin-core/secp256k1#1089: Schnorrsig API improvements
587239dbe3 Merge bitcoin-core/secp256k1#731: Change SHA256 byte counter from size_t to uint64_t
f8d9174357 Add SHA256 bit counter tests
7f09d0f311 README: mention that ARM assembly is experimental
b8f8b99f0f docs: Fix return value for functions that don't have invalid inputs
f813bb0df3 schnorrsig: Adapt example to new API
99e6568fc6 schnorrsig: Rename schnorrsig_sign to schnorsig_sign32 and deprecate
fc94a2da44 Use SECP256K1_DEPRECATED for existing deprecated API functions
3db0560606 Add SECP256K1_DEPRECATED attribute for marking API parts as deprecated
80cf4eea5f build: stop treating schnorrsig, extrakeys modules as experimental
e0508ee9db Merge bitcoin-core/secp256k1#1090: configure: Remove redundant pkg-config code
21b2ebaf74 configure: Remove redundant pkg-config code
0e5cbd01b3 Merge bitcoin-core/secp256k1#1088: configure: Use modern way to set AR
0d253d52e8 configure: Use modern way to set AR
9b514ce1d2 Add test vector for very long SHA256 messages
8e3dde1137 Simplify struct initializer for SHA256 padding
eb28464a8b Change SHA256 byte counter from size_t to uint64_t
ac83be33d0 Merge bitcoin-core/secp256k1#1079: configure: Add hidden --enable-dev-mode to enable all the stuff
e0838d663d configure: Add hidden --enable-dev-mode to enable all the stuff
fabd579dfa configure: Remove redundant code that sets _enable variables
0d4226c051 configure: Use canonical variable prefix _enable consistently
64b34979ed Merge bitcoin-core/secp256k1#748: Add usage examples
7c9502cece Add a copy of the CC0 license to the examples
42e03432e6 Add usage examples to the readme
517644eab1 Optionally compile the examples in autotools, compile+run in travis
422a7cc86a Add a ecdh shared secret example
b0cfbcc143 Add a Schnorr signing and verifying example
fee7d4bf9e Add an ECDSA signing and verifying example
1253a27756 Merge bitcoin-core/secp256k1#1033: Add _fe_half and use in _gej_add_ge and _gej_double
3ef94aa5ba Merge bitcoin-core/secp256k1#1026: ecdh: Add test computing shared_secret=basepoint with random inputs
3531a43b5b ecdh: Make generator_basepoint test depend on global iteration count
c881dd49bd ecdh: Add test computing shared_secret=basepoint with random inputs
077528317d Merge bitcoin-core/secp256k1#1074: ci: Retry brew update a few times to avoid random failures
e51ad3b737 ci: Retry `brew update` a few times to avoid random failures
b1cb969e8a ci: Revert "Attempt to make macOS builds more reliable"
5dcc6f8dbd Merge bitcoin-core/secp256k1#1069: build: Replace use of deprecated autoconf macro AC_PROG_CC_C89
59547943d6 Merge bitcoin-core/secp256k1#1072: ci: Attempt to make macOS builds more reliable
85b00a1c65 Merge bitcoin-core/secp256k1#1068: sage: Fix incompatibility with sage 9.4
ebb1beea78 sage: Ensure that constraints are always fastfracs
d8d54859ed ci: Run sage prover on CI
77cfa98dbc sage: Normalize sign of polynomial factors in prover
eae75869cf sage: Exit with non-zero status in case of failures
d9396a56da ci: Attempt to make macOS builds more reliable
e0db3f8a25 build: Replace use of deprecated autoconf macro AC_PROG_CC_C89
e848c3799c Update sage files for new formulae
d64bb5d4f3 Add fe_half tests for worst-case inputs
b54d843eac sage: Fix printing of errors
4eb8b932ff Further improve doubling formula using fe_half
557b31fac3 Doubling formula using fe_half
2cbb4b1a42 Run more iterations of run_field_misc
9cc5c257ed Add test for secp256k1_fe_half
925f78d55e Add _fe_half and use in _gej_add_ge
e108d0039c sage: Fix incompatibility with sage 9.4
d8a2463246 Merge bitcoin-core/secp256k1#899: Reduce stratch space needed by ecmult_strauss_wnaf.
0a40a4861a Merge bitcoin-core/secp256k1#1049: Faster fixed-input ecmult tests
070e772211 Faster fixed-input ecmult tests
c8aa516b57 Merge bitcoin-core/secp256k1#1064: Modulo-reduce msg32 inside RFC6979 nonce fn to match spec. Fixes#1063
b797a500ec Create a SECP256K1_ECMULT_TABLE_VERIFY macro.
a731200cc3 Replace ECMULT_TABLE_GET_GE_STORAGE macro with a function.
fe34d9f341 Eliminate input_pos state field from ecmult_strauss_wnaf.
0397d00ba0 Eliminate na_1 and na_lam state fields from ecmult_strauss_wnaf.
7ba3ffcca0 Remove the unused pre_a_lam allocations.
b3b57ad6ee Eliminate the pre_a_lam array from ecmult_strauss_wnaf.
ae7ba0f922 Remove the unused prej allocations.
e5c18892db Eliminate the prej array from ecmult_strauss_wnaf.
c9da1baad1 Move secp256k1_fe_one to field.h
45f37b6506 Modulo-reduce msg32 inside RFC6979 nonce fn to match spec. Fixes#1063.
a1102b1219 Merge bitcoin-core/secp256k1#1029: Simpler and faster ecdh skew fixup
e82144edfb Fixup skew before global Z fixup
40b624c90b Add tests for _gej_cmov
8c13a9bfe1 ECDH skews by 0 or 1
1515099433 Simpler and faster ecdh skew fixup
39a36db94a Merge bitcoin-core/secp256k1#1054: tests: Fix test whose result is implementation-defined
a310e79ee5 Merge bitcoin-core/secp256k1#1052: Use xoshiro256++ instead of RFC6979 for tests
423b6d19d3 Merge bitcoin-core/secp256k1#964: Add release-process.md
9281c9f4e1 Merge bitcoin-core/secp256k1#1053: ecmult: move `_ecmult_odd_multiples_table_globalz_windowa`
77a19750b4 Use xoshiro256++ PRNG instead of RFC6979 in tests
5f2efe684e secp256k1_testrand_int(2**N) -> secp256k1_testrand_bits(N)
05e049b73c ecmult: move `_ecmult_odd_multiples_table_globalz_windowa`
3d7cbafb5f tests: Fix test whose result is implementation-defined
3ed0d02bf7 doc: add CHANGELOG template
6f42dc16c8 doc: add release_process.md
0bd3e4243c build: set library version to 0.0.0 explicitly
b4b02fd8c4 build: change libsecp version from 0.1 to 0.1.0-pre
09971a3ffd Merge bitcoin-core/secp256k1#1047: ci: Various improvements
0b83b203e1 Merge bitcoin-core/secp256k1#1030: doc: Fix upper bounds + cleanup in field_5x52_impl.h comment
1287786c7a doc: Add comment to top of field_10x26_impl.h
58da5bd589 doc: Fix upper bounds + cleanup in field_5x52_impl.h comment
b39d431aed Merge bitcoin-core/secp256k1#1044: Add another ecmult_multi test
b4ac1a1d5f ci: Run valgrind/memcheck tasks with 2 CPUs
e70acab601 ci: Use Cirrus "greedy" flag to use idle CPU time when available
d07e30176e ci: Update brew on macOS
22382f0ea0 ci: Test different ecmult window sizes
a69df3ad24 Merge bitcoin-core/secp256k1#816: Improve checks at top of _fe_negate methods
22d25c8e0a Add another ecmult_multi test
515e7953ca Improve checks at top of _fe_negate methods
26a022a3a0 ci: Remove STATICPRECOMPUTATION
10461d8bd3 precompute_ecmult: Always compute all tables up to default WINDOW_G
be6944ade9 Merge bitcoin-core/secp256k1#1042: Follow-ups to making all tables fully static
e05da9e480 Fix c++ build
c45386d994 Cleanup preprocessor indentation in precompute{,d}_ecmult{,_gen}
19d96e15f9 Split off .c file from precomputed_ecmult.h
1a6691adae Split off .c file from precomputed_ecmult_gen.h
bb36331412 Simplify precompute_ecmult_print_*
38cd84a0cb Compute ecmult tables at runtime for tests_exhaustive
e458ec26d6 Move ecmult table computation code to separate file
fc1bf9f15f Split ecmult table computation and printing
31feab053b Rename function secp256k1_ecmult_gen_{create_prec -> compute}_table
725370c3f2 Rename ecmult_gen_prec -> ecmult_gen_compute_table
075252c1b7 Rename ecmult_static_pre_g -> precomputed_ecmult
7cf47f72bc Rename ecmult_gen_static_prec_table -> precomputed_ecmult_gen
f95b8106d0 Rename gen_ecmult_static_pre_g -> precompute_ecmult
bae77685eb Rename gen_ecmult_gen_static_prec_table -> precompute_ecmult_gen
git-subtree-dir: src/secp256k1
git-subtree-split: 8746600eec5e7fcd35dabd480839a3a4bdfee87b
ffffb7a25a doc: Convert remaining comments to clang-tidy format (MarcoFalke)
Pull request description:
This is a follow-up to commit 0da559e02e, hopefully the last one.
ACKs for top commit:
Empact:
Code review ACK ffffb7a25a
vincenzopalazzo:
ACK ffffb7a25a
Tree-SHA512: 541f5e69aeee53815e77e63b9012d3ed24856eec5f7e28cc764b3ce29f897563e56ade07db9891324f2809dcf064ce10333b1a1e4a7490af9bd97dcf6ff4e4f7
4105a54381 lint: remove boost::bind linter (fanquake)
Pull request description:
I don't think we need to maintain a linter for reintroducing boost::bind at this point.
ACKs for top commit:
hebasto:
ACK 4105a54381, I have reviewed the code and it looks OK, I agree it can be merged.
Tree-SHA512: 86bda91ee7ed11f0aa7ac95e9e7b62dbba626dcea75444d2851a3d40e794ab16bef09a1f0c956a716d43602b23c1cf67e1ff3a51184ea1ee7d686fbb76316cb0
faa7ae8242 ci: Build all optional tools in tidy task (MarcoFalke)
Pull request description:
Ideally the whole source code is run through clang-tidy, but it can only run if the code is compiled. So install all optional deps for the targets.
Hopefully this doesn't increase the run time too much.
ACKs for top commit:
fanquake:
ACK faa7ae8242 - runtime is still ~13 minutes.
Tree-SHA512: dc74e673a2998829f1e7069c89d7bc11bd6705b53d7a639a42a88c38d2e64c529e535e4e65c1cad9047c11d700ec4fa4433571048c89ed76e0486647c21652a2
65c49ac750 test: throw `ValueError` for invalid base58 checksum (Sebastian Falbesoner)
219d2c7ee1 contrib: testgen: use base58 methods from test framework (Sebastian Falbesoner)
605fecfb66 scripted-diff: rename `chars` to `b58chars` in test_framework.address (Sebastian Falbesoner)
11c63e374d contrib: testgen: import OP_* constants from test framework (Sebastian Falbesoner)
7d755bb31c contrib: testgen: avoid need for manually setting PYTHONPATH (Sebastian Falbesoner)
Pull request description:
This PR removes the redundant base58 implementation [contrib/testgen/base58.py](https://github.com/bitcoin/bitcoin/blob/master/contrib/testgen/base58.py) for the test generation script `gen_key_io_test_vectors.py` and uses the one from the test framework instead. Additionally, three other cleanups/improvements are done:
- import script operator constants `OP_*` from test framework instead of manually defining them
- add Python path to test framework directly in the script (via `sys.path.append(...)`) instead of needing the caller to specify `PYTHONPATH=...` on the command line (the same approach is done for the signet miner and the message capture scripts)
- rename `chars` to `b58chars` in the test_framework.address module (is more explicit and makes the diff for the base58 replacement smaller)
ACKs for top commit:
laanwj:
Code review ACK 65c49ac750
Tree-SHA512: 92e1534cc320cd56262bf455de7231c6ec821bfcd0ed58aa5718271ecec1a89df7951bf31527a2306db6398e7f2664d2ff8508200c28163c0b164d3f5aaf8b0e
9d65ad365c Clear vTxHashes when mapTx is cleared (Peter Bushnell)
Pull request description:
vTxHashes is a vector of all entries in mapTx, if you clear one you should clear the other, lest someone try to use the txiter in vTxHashes which would result in a segfault.
ACKs for top commit:
laanwj:
Code review ACK 9d65ad365c
Tree-SHA512: 6832755e43ab7f528b46817aeadcb6ffdc965b97f59ab96bb053dedbb7e68155ba3db52286355dca33b509237f80eda249760b26db493762bc50d8e2cef16d8f
76c60d7b31 test: validation:block_connected tracepoint test (0xb10c)
260e28ece8 test: utxocache:* tracepoint tests (0xb10c)
34b27bac68 test: net:in/out_message tracepoint tests (0xb10c)
c934087b62 test: checks for tracepoint tests (0xb10c)
Pull request description:
This adds functional tests for the USDT tracepoints added in https://github.com/bitcoin/bitcoin/pull/22006 and https://github.com/bitcoin/bitcoin/pull/22902. This partially fixes#23296. The tests **are probably skipped** on most systems as these tests require:
- a Linux system with a kernel that supports BPF (and available kernel headers)
- that Bitcoin Core is compiled with tracepoints for USDT support (default when compiled with depends)
- [bcc](https://github.com/iovisor/bcc) installed
- the tests are run with a privileged user that is able to e.g. do BPF syscalls and load BPF maps
The tests are not yet run in our CI as the CirrusCI containers lack the required permissions (see https://github.com/bitcoin/bitcoin/issues/23296#issuecomment-1024920845). Running the tests in a VM in the CI could work, but I haven't experimented with this yet. The priority was to get the actual tests done first to ensure the tracepoints work as intended for the v23.0 release. Running the tracepoint tests in the CI is planned as the next step to finish #23296.
The tests can, however, be run against e.g. release candidates by hand. Additionally, they provide a starting point for tests for future tracepoints. PRs adding new tracepoint should include tests. This makes reviewing these PRs easier.
The tests require privileges to execute BPF sycalls (`CAP_SYS_ADMIN` before Linux kernel 5.8 and `CAP_BPF` and `CAP_PERFMON` on 5.8+) and permissions to `/sys/kernel/debug/tracing/`. It's currently recommended to run the tests in a virtual machine (or on a VPS) where it's sensible to use the `root` user to gain these privileges. Never run python scripts you haven't carefully reviewed with `root` permissions! It's unclear if a non-root user can even gain the required privileges. This needs more experimenting.
The goal here is to test the tracepoint interface to make sure the [documented interface](https://github.com/bitcoin/bitcoin/blob/master/doc/tracing.md#tracepoint-documentation) does not break by accident. The tracepoints expose implementation details. This means we also need to rely on implementation details of Bitcoin Core in these functional tests to trigger the tracepoints. An example is the test of the `utxocache:flush` tracepoint: On Bitcoin Core shutdown, the UTXO cache is flushed twice. The corresponding tracepoint test expects two flushes, too - if not, the test fails. Changing implementation details could cause these tests to fail and the tracepoint API to break. However, we purposefully treat the tracepoints only as [**semi-stable**](https://github.com/bitcoin/bitcoin/blob/master/doc/tracing.md#semi-stable-api). The tracepoints should not block refactors or changes to other internals.
ACKs for top commit:
jb55:
tACK 76c60d7b31
laanwj:
Tested ACK 76c60d7b31
Tree-SHA512: 9a63d945c68102e59d751bd8d2805ddd7b37185408fa831d28a9cb6641b701961389b55f216c475df7d4771154e735625067ee957fc74f454ad7a7921255364c
af74e061c0 guix: make it possible to override gpg binary (Pavol Rusnak)
Pull request description:
For example on Qubes OS one might want to use qubes-gpg-client-wrapper instead
Fixes https://github.com/bitcoin/bitcoin/issues/24346
ACKs for top commit:
laanwj:
Concept and code review ACK af74e061c0
Tree-SHA512: 9e56b5fab231f8908fff15c88fe5b356ac4a31a14a27ae2dd3b6e876f32628910a666a4e2da5bf7c5d159de66cf57652c94c81cdc3b1c3d39a23c23e2c77dd03
3d41521569 build: perform /Applications symlink generation in macdeployqtplus (fanquake)
dac6936719 build: perform all .tiff copying in macdeployqtplus (fanquake)
Pull request description:
Rather than maintaining 2 different versions of the same code (`.tiff` copying and symlink generation), consolidate to just the Python code, and use it on macOS and Linux. Previously Linux would perform the 2 actions in the makefile, and then would still be running the `macdeployqtplus` script, so it makes sense to further consolidate deployment operations into the script.
Guix Build (on x86_64):
```bash
23343f04c426c7ff078afae4e600a7028970d4d86eed8b7834696d9e4d684151 guix-build-3d415215699e/output/arm64-apple-darwin/SHA256SUMS.part
c28b2a2e4888bf84369aa25804e2576347d5ab09416354ec8b95c76a9d38ff96 guix-build-3d415215699e/output/arm64-apple-darwin/bitcoin-3d415215699e-arm64-apple-darwin-unsigned.dmg
9a57077b2bd722a7d85d26b66cbce5abdb791985fe9d9d37e884c79ba8751e24 guix-build-3d415215699e/output/arm64-apple-darwin/bitcoin-3d415215699e-arm64-apple-darwin-unsigned.tar.gz
d2b06dc5b86541798ace41dab569849f7403e7ff9ec329bda671ec84e6fad549 guix-build-3d415215699e/output/arm64-apple-darwin/bitcoin-3d415215699e-arm64-apple-darwin.tar.gz
608e7d51a44ab9c5b28eb3703a0f4fe98b4adff22c77a5502786b84bd96cc188 guix-build-3d415215699e/output/dist-archive/bitcoin-3d415215699e.tar.gz
3e483705b1f9f1fb8f6afedc8ad0214a6cb00e77f766c0b03c42d56f410d4362 guix-build-3d415215699e/output/x86_64-apple-darwin/SHA256SUMS.part
9370e3e3b7d47b5a44e64554cf3b6d7e0671b072c08cd251eacc7ec72ce2b53f guix-build-3d415215699e/output/x86_64-apple-darwin/bitcoin-3d415215699e-x86_64-apple-darwin-unsigned.dmg
ad0f68682d78c311497669fc3d627138be37510215d259b5f0b686d93e7d83b7 guix-build-3d415215699e/output/x86_64-apple-darwin/bitcoin-3d415215699e-x86_64-apple-darwin-unsigned.tar.gz
e09dce4ff692ef66d1f4818083c1880bcf3a79c53112561d9e929bb6e5ffc011 guix-build-3d415215699e/output/x86_64-apple-darwin/bitcoin-3d415215699e-x86_64-apple-darwin.tar.gz
```
ACKs for top commit:
laanwj:
Re-ACK 3d41521569
Tree-SHA512: 80dd66a6e94c5b3e8823ccb57dcb08a8851a1e70a154b62385443f8d2d5ed5af900a0ac5003143959863586f1c7b90002fe6bff3ca5e37697253e051f69d7629
fabdf9f870 Remove gui-only syscalls (MarcoFalke)
fa0c2aa826 init: Disable syscall sandbox in the bitcoin-qt process (MarcoFalke)
Pull request description:
It is basically impossible (and a bit out of scope) for us to maintain a sandbox for the qt library. I am not sure if it is possible to only sandbox a few threads in a process, but I doubt this will add no practical benefit anyway, so I am disabling the sandbox for the whole bitcoin-qt process.
See also https://github.com/bitcoin/bitcoin/pull/24690#issuecomment-1084372400
ACKs for top commit:
laanwj:
Code review ACK fabdf9f870
Tree-SHA512: 944ded03ee25f7dfd0bfeea9c3f97f575f2d470aa03b387b07f3e3bec5cb886e4aaa17e4a9fb359d3e670e6da69adc9111673d13e6561ec55b3161bb67dfe760
a2b56dcd1f doc: update OpenBSD build docs for 7.0 (fanquake)
Pull request description:
Removes redundant notes for setting `CC` &`CXX` now that Clang is well and truly the base compiler. See: https://www.openbsd.org/70.html
> Disabled base-gcc on amd64.
Cleans up the wallet docs, i.e #23446.
Make the notes more similar to the FreeBSD notes.
ACKs for top commit:
shaavan:
ACK a2b56dcd1f
theStack:
ACK a2b56dcd1f
Tree-SHA512: a0494de3b168e5c35f541edf62dcb42529b23387febbe4c004eb82ef9aff6f97def43b6cd5c91e13612c5247767d79553efcd21b9792ccb6a9608302c5d082f1
cccc4e879a Remove nHeightEnd and nHeight in generateBlocks helper (MarcoFalke)
fa38b1c8bd Remove buggy and confusing IncrementExtraNonce (MarcoFalke)
Pull request description:
IncrementExtraNonce has many issues:
* It is test-only code, but part of bitcoind
* It is using the block height of the tip, as opposed to the block's previous block as reference for the new height. See https://github.com/bitcoin/bitcoin/issues/24730#issuecomment-1085586193
* It has no use case in regtest testing. With a low difficulty the extra nonce won't be incremented. With a high difficulty the test-only functions are clumsy to handle anyway. For example, the generate* RPCs will return an empty array once they reached `maxtries`, as opposed to an error. Also the calls can't be aborted early unless the node shuts down completely. So I think it is fine to just remove the extra nonce functionality and leave it to the outside to implement, if needed. For example, a wrapper script can call the `generate*` RPCs once every second, to use the timestamp as extra nonce.
ACKs for top commit:
ajtowns:
ACK cccc4e879a
Tree-SHA512: d8a3989ad280ebd4b1b574159b3a396b8a42134347e6be3c88445162d86624d221c416456f45ae75aea62ed8c8a1a9bb3a2532924abca2ef7a879cb8e6b15654
54b39cfb34 Add release notes (stickies-v)
f959fc0397 Update /<count>/ endpoints to use a '?count=' query parameter instead (stickies-v)
a09497614e Add GetQueryParameter helper function (stickies-v)
fff771ee86 Handle query string when parsing data format (stickies-v)
c1aad1b3b9 scripted-diff: rename RetFormat to RESTResponseFormat (stickies-v)
9f1c54787c Refactoring: move declarations to rest.h (stickies-v)
Pull request description:
In RESTful APIs, [typically](https://rapidapi.com/blog/api-glossary/parameters/query/) path parameters (e.g. `/some/unique/resource/`) are used to represent resources, and query parameters (e.g. `?sort=asc`) are used to control how these resources are being loaded through e.g. sorting, pagination, filtering, ...
As first [discussed in #17631](https://github.com/bitcoin/bitcoin/pull/17631#discussion_r733031180), the [current REST api](https://github.com/bitcoin/bitcoin/blob/master/doc/REST-interface.md) contains two endpoints `/headers/` and `/blockfilterheaders/` that rather unexpectedly use path parameters to control how many (filter) headers are returned in the response. While this is no critical issue, it is unintuitive and we are still early enough to easily phase this behaviour out and ensure new endpoints (if any) do not have to stick to non-standard behaviour just for internal consistency.
In this PR, a new `HTTPRequest::GetQueryParameter` method is introduced to easily parse query parameters, as well as two new `/headers/` and `/blockfilterheaders/` endpoints that use a count query parameter are introduced. The old path parameter-based endpoints are kept without too much overhead, but the documentation now points to the new query parameter-based endpoints as the default interface to encourage standardness.
## Behaviour change
### New endpoints and default values
`/headers/` and `/blockfilterheaders/` now have 2 new endpoints that contain query parameters (`?count=<count>`) instead of path parameters (`/<count>/`), as described in REST-interface.md. Since query parameters can easily have default values, I have set this at 5 for both endpoints.
**headers**
`GET /rest/headers/<BLOCK-HASH>.<bin|hex|json>?count=<COUNT=5>`
should now be used instead of
`GET /rest/headers/<COUNT>/<BLOCK-HASH>.<bin|hex|json>`
**blockfilterheaders**
`GET /rest/blockfilterheaders/<FILTERTYPE>/<BLOCK-HASH>.<bin|hex|json>?count=<COUNT=5>`
should now be used instead of
`GET /rest/blockfilterheaders/<FILTERTYPE>/<COUNT>/<BLOCK-HASH>.<bin|hex|json>`
### Some previously invalid API calls are now valid
API calls that contained query strings in the URI could not be parsed prior to this PR. This PR changes behaviour in that previously invalid calls (e.g. `GET /rest/headers/5/somehash.json?someunusedparam=foo`) would now become valid, as the query parameters are properly parsed, and discarded if unused.
For example, prior to this PR, adding an irrelevant `someparam` parameter would be illegal:
```
GET /rest/headers/5/0000004c6aad0c89c1c060e8e116dcd849e0554935cd78ff9c6a398abeac6eda.json?someparam=true
->
Invalid hash: 0000004c6aad0c89c1c060e8e116dcd849e0554935cd78ff9c6a398abeac6eda.json?someparam=true
```
**This behaviour change affects all rest endpoints, not just the 2 new ones introduced here.**
*(Note: I'd be open to implementing additional logic to refuse requests containing unrecognized query parameters to minimize behaviour change, but for the endpoints that we currently have I don't really see the point for that added complexity. E.g. I don't see any scenarios where misspelling a parameter could lead to harmful outcomes)*
## Using the REST API
To run the API HTTP server, start a bitcoind instance with the `-rest` flag enabled. To use the
`blockfilterheaders` endpoint, you'll also need to set `-blockfilterindex=1`:
```
./bitcoind -signet -rest -blockfilterindex=1
```
As soon as bitcoind is fully up and running, you should be able to query the API, for example by
using curl on the command line: ```curl "127.0.0.1:38332/rest/chaininfo.json"```.
To more easily parse the JSON output, you can also use tools like 'jq' or `json_pp`, e.g.:
```
curl -s "localhost:38332/rest/blockfilterheaders/basic/0000004c6aad0c89c1c060e8e116dcd849e0554935cd78ff9c6a398abeac6eda.json?count=2" | json_pp .
```
## To do
- [x] update `doc/release-notes`
## Feedback
This is my first PR (hooray!). Please don't hold back on any feedback/comments/nits/... you may have, big or small, whether they are code, process, language, ... related. I welcome private messages too if there's anything you don't want to clutter the PR with. I'm here to learn and am grateful for everyone's input.
ACKs for top commit:
stickies-v:
I've had to push a tiny doc update to `REST-interface.md` (`git range-diff 219d728 9aac438 54b39cf`) since this was not merged for v23, but since there are no significant changes beyond theStack and jnewbery's ACKs I think this PR is now ready to be considered for merging? @MarcoFalke
jnewbery:
ACK 54b39cfb34
theStack:
re-ACK 54b39cfb34
Tree-SHA512: 3b393ffde34f25605ca12c0b1300799a19684b816a1d03aed38b0f5439df47bfe6a589ffbcd7b83fd2def6c9d00a1bae5e45b1d18df4ae998c617c709990f83f
4685463301 doc: Update lint test docs (Fabian Jahr)
77f98df41f lint: convert spell check lint test to python (Fabian Jahr)
Pull request description:
The new python version should produce the exact same output as the bash version but be easier to maintain.
ACKs for top commit:
MarcoFalke:
cr ACK 4685463301
Tree-SHA512: 242b802b750b42b299b93d1de4bcf17d92ad0a633d31894145d8590782a1db1041de59a283f133a4f75898d95444eb3c842005a6aa5cb919543625addad596d8
076cd6835f lint: Convert Python dead code linter to Python (Fabian Jahr)
Pull request description:
The new python version should produce the exact same output as the bash version but be easier to maintain.
ACKs for top commit:
MarcoFalke:
review ACK 076cd6835f
Tree-SHA512: 6d71a5230d612a981958fb6e178214240b09e842ffe35e207cbbda870ca35476626bf832f55d96f2fc7323a2875935edfee30cacef659a8e6ec4bbb28ff6e36a
ba0bf79a22 build: Do not modify `common.init.vcxproj` directly (Hennadii Stepanov)
2391fb7850 build, refactor: Add set_properties() to msvc-autogen.py (Hennadii Stepanov)
Pull request description:
When building with MSVC, and using a non-default toolset, the following command
```
>python build_msvc\msvc-autogen.py -toolset v143
```
actually modifies the source tree:
```diff
>git diff
warning: LF will be replaced by CRLF in build_msvc/common.init.vcxproj.
The file will have its original line endings in your working directory
diff --git a/build_msvc/common.init.vcxproj b/build_msvc/common.init.vcxproj
index 0cbe2effd..44b7efff3 100644
--- a/build_msvc/common.init.vcxproj
+++ b/build_msvc/common.init.vcxproj
@@ -39,7 +39,7 @@
<PropertyGroup Condition="'$(Configuration)'=='Release'" Label="Configuration">
<LinkIncremental>false</LinkIncremental>
<UseDebugLibraries>false</UseDebugLibraries>
- <PlatformToolset>v142</PlatformToolset>
+ <PlatformToolset>v143</PlatformToolset>
<CharacterSet>Unicode</CharacterSet>
<GenerateManifest>No</GenerateManifest>
<OutDir>$(SolutionDir)$(Platform)\$(Configuration)\$(ProjectName)\</OutDir>
@@ -49,7 +49,7 @@
<PropertyGroup Condition="'$(Configuration)'=='Debug'" Label="Configuration">
<LinkIncremental>true</LinkIncremental>
<UseDebugLibraries>true</UseDebugLibraries>
- <PlatformToolset>v142</PlatformToolset>
+ <PlatformToolset>v143</PlatformToolset>
<CharacterSet>Unicode</CharacterSet>
<OutDir>$(SolutionDir)$(Platform)\$(Configuration)\$(ProjectName)\</OutDir>
<IntDir>$(Platform)\$(Configuration)\$(ProjectName)\</IntDir>
```
This PR fixes this bug.
ACKs for top commit:
sipsorcery:
tACK ba0bf79a22.
Tree-SHA512: 614662fdbce6aea7a09fac5bba1e3632dd921b0d8f89c448566f527bf5df049ae6bacd951662825188b88324aec7e3416b58237e0a227d7520e2a75223b0edd0
In most RESTful APIs, path parameters are used to represent resources, and
query parameters are used to control how these resources are being filtered/sorted/...
The old /<count>/ functionality is kept alive to maintain backwards compatibility,
but new paths with query parameters are introduced and documented as the default
interface so future API methods don't break consistency by using query parameters.
6a02355ae9 Add and improve informational links in doc/cjdns.md (Jon Atack)
19538dd41e Add concrete steps in doc/cjdns.md to easily find a friend (Jon Atack)
Pull request description:
and improve the informational links. CJDNS functions with a friend-of-a-friend topology and a key hurdle to getting started is to find a public peer and set up an outbound connection to it. This update makes doing it much easier for people getting started.
Credit to Vasil Dimov for an [IRC suggestion in October 2021](https://www.erisian.com.au/bitcoin-core-dev/log-2021-10-04.html#l-469) and to stickies-v for IRC discussions this week and the [testing guide](https://github.com/bitcoin-core/bitcoin-devwiki/wiki/23.0-Release-Candidate-Testing-Guide) that led me to redo these steps, provide feedback at https://github.com/bitcoin/bitcoin/issues/24706 and refine the added documentation here.
ACKs for top commit:
dunxen:
ACK 6a02355
stickies-v:
re-ACK [6a02355](6a02355ae9) even though I wasn't opposed to the "friend" terminology since it's the language CJDNS seems to use to denominate the peers you connect to directly in general. Not worth bikeshedding over though.
lsilva01:
Strong ACK 6a02355
Tree-SHA512: b2fa2a200a6a55a709486f7ed2d3830cabffbbffa61a0d211fcb666a918b5754d4e99a58c32909fe58540598066e6ff67bf2fa2fcd56b1b5dcff3c2162f6d962
fa9112aac0 Remove utxo db upgrade code (MarcoFalke)
Pull request description:
It is not possible to upgrade Bitcoin Core pre-segwit (pre-0.13.1) to a recent version without a full IBD from scratch after commit 19a56d1519 (released in version 22.0).
Any Bitcoin Core version with the new database format after commit 1088b02f0c (released in version 0.15), can upgrade to any version that is supported as of today.
This leaves the versions 0.13.1-0.14.x. Even though those versions are unsupported, some users with an existing datadir may want to upgrade to a recent version. However, it seems reasonable to simply ask them to `-reindex` to run a full IBD from scratch. This allows us to remove the utxo db upgrade code.
ACKs for top commit:
Sjors:
re-ACK fa9112aac0
laanwj:
Code review ACK fa9112aac0
Tree-SHA512: 4243bb35df9ac4892f9fad30fe486d338745952bcff4160bcb0937c772d57b13b800647da14695e21e3655e85ee0d95fa3dc7789ee309d59ad84f422297fecb8
d2ba43fec8 test: use MiniWallet for mempool_unbroadcast.py (Ayush Sharma)
Pull request description:
This PR enables one of the non-wallet functional tests (mempool_unbroadcast.py) to be run even with the Bitcoin Core wallet disabled by using the MiniWallet instead, as proposed in #20078 .
Top commit has no ACKs.
Tree-SHA512: e4c577899b66855dafca9dab875fa9b9c68b762a8cdb14f3a7547841c4f001e79d62641e6ae202fb56a3f28aeea1779143164c872507ff8da0bd9930a8ed182e
2da94a4c6f fuzz: add a fuzz target for Miniscript decoding from Script (Antoine Poinsot)
f8369996e7 Miniscript: ops limit and stack size computation (Pieter Wuille)
2e55e88f86 Miniscript: conversion from script (Pieter Wuille)
1ddaa66eae Miniscript: type system, script creation, text notation, tests (Pieter Wuille)
4fe29368c0 script: expose getter for CScriptNum, add a BuildScript helper (Antoine Poinsot)
f4e289f384 script: move CheckMinimalPush from interpreter to script.h (Antoine Poinsot)
31ec6ae92a script: make IsPushdataOp non-static (Antoine Poinsot)
Pull request description:
Miniscript is a language for writing (a subset of) Bitcoin Scripts in a structured way.
Miniscript permits:
- To safely extend the Output Descriptor language to many more scripting features thanks to the typing system (composition).
- Statical analysis of spending conditions, maximum spending cost of each branch, security properties, third-party malleability.
- General satisfaction of any correctly typed ("valid" [0]) Miniscript. The satisfaction itself is also analyzable.
- To extend the possibilities of external signers, because of all of the above and since it carries enough metadata.
Miniscript guarantees:
- That for any statically-analyzed as "safe" [0] Script, a witness can be constructed in the bounds of the consensus and standardness rules (standardness complete).
- That unless the conditions of the Miniscript are met, no witness can be created for the Script (consensus sound).
- Third-party malleability protection for the satisfaction of a sane Miniscript, which is too complex to summarize here.
For more details around Miniscript (including the specifications), please refer to the [website](https://bitcoin.sipa.be/miniscript/).
Miniscript was designed by Pieter Wuille, Andrew Poelstra and Sanket Kanjalkar.
This PR is an updated and rebased version of #16800. See [the commit history of the Miniscript repository](https://github.com/sipa/miniscript/commits/master) for details about the changes made since September 2019 (TL;DR: bugfixes, introduction of timelock conflicts in the type system, `pk()` and `pkh()` aliases, `thresh_m` renamed to `multi`, all recursive algorithms were made non-recursive).
This PR is also the first in a series of 3:
- The first one (here) integrates the backbone of Miniscript.
- The second one (#24148) introduces support for Miniscript in Output Descriptors, allowing for watch-only support of Miniscript Descriptors in the wallet.
- The third one (#24149) implements signing for these Miniscript Descriptors, using Miniscript's satisfaction algorithm.
Note to reviewers:
- Miniscript is currently defined only for P2WSH. No Taproot yet.
- Miniscript is different from the policy language (a high-level logical representation of a spending policy). A policy->Miniscript compiler is not included here.
- The fuzz target included here is more interestingly extended in the 3rd PR to check a script's satisfaction against `VerifyScript`. I think it could be further improved by having custom mutators as we now have for multisig (see https://github.com/bitcoin/bitcoin/issues/23105). A minified corpus of Miniscript Scripts is available at https://github.com/bitcoin-core/qa-assets/pull/85.
[0] We call "valid" any correctly-typed Miniscript. And "safe" any sane Miniscript, ie one whose satisfaction isn't malleable, which requires a key for any spending path, etc..
ACKs for top commit:
jb55:
ACK 2da94a4c6f
laanwj:
Light code review ACK 2da94a4c6f (mostly reviewed the changes to the existing code and build system)
Tree-SHA512: d3ef558436cfcc699a50ad13caf1e776f7d0addddb433ee28ef38f66ea5c3e581382d8c748ccac9b51768e4b95712ed7a6112b0e3281a6551e0f325331de9167
7b00595d33 build: stop overriding user CXXFLAGS (fanquake)
3e2ef23c3e build: stop overriding user LDFLAGS (fanquake)
35c3fd43c3 build: stop overriding user CPPFLAGS (fanquake)
bc7cc57607 doc: explain why we clear CXXFLAGS with enable-debug (fanquake)
Pull request description:
Historically our build system has hijacked `CXXFLAGS` and friends, and this has always been a source of complaints from users and developers. With this PR, we move away from using `CXXFLAGS`, `CPPFLAGS` and `LDFLAGS`, and instead use `CORE_*FLAGS` variables for our flags / options, leaving autoconfs `FLAG` vars to the user.
Note that there are currently two cases where we will at least clear `CXXFLAGS` (if not alreaddy overridden by the user), when doing debugging or when coverage is enabled, to avoid Autoconfs `-g -O2` CXXFLAG default.
ACKs for top commit:
hebasto:
ACK 7b00595d33
Tree-SHA512: bda936a7aa8f98a1bf1552306845cb4bbab54e19a7a0b9ce3210e10fef70db146e9fe42a0cc8c50b2908506771b5b96f39c334e41323b70ec878e4010373096c
7c218dacd0 build: specify cmake build dir for multiprocess depends build (fanquake)
Pull request description:
When no build dir is specified, cmake will warn:
```bash
Preprocessing libmultiprocess...
Configuring libmultiprocess...
CMake Warning:
No source or binary directory provided. Both will be assumed to be the
same as the current working directory, but note that this warning will
become a fatal error in future CMake releases.
```
It's unclear if this will actually ever become an error, but it's also easy
enough to just supply the directory, and save this maybe breaking in
future.
ACKs for top commit:
ryanofsky:
Code review ACK 7c218dacd0. I guess the purpose of the warning is to encourage people not to build in the source directory, but reasons for encouraging this don't really apply to the depends build system, so it is appropriate to disable the warning.
hebasto:
ACK 7c218dacd0, tested on Ubuntu 22.04.
Tree-SHA512: 6904f2095fe62cead4abc644ec888c5d836e54a3c0b2a84c467029116e5d14eba35190570acaa23c6831aed9a4a65898134480cc46cdb141279ec0dc6f534d5f
fa4943e8df doc: Add template for empty release notes (MarcoFalke)
Pull request description:
* Move release process notes from the release notes to the release process documentation
* Clarify that wallet RPC or Settings related release notes snippets should not be duplicated. I think it should be sufficient to only mention them in the wallet section and leave them out from the general RPC section.
* Create an empty template to ensure the release notes can be cleared with a single `cp` command. Also, this ensures that the "no duplication" note isn't deleted again. (We used to have it in at least the 22.0 and 21.0 release notes: https://github.com/bitcoin/bitcoin/blob/master/doc/release-notes/release-notes-0.21.0.md#updated-settings , but it was lost in the 23.0 notes)
ACKs for top commit:
laanwj:
ACK fa4943e8df
Tree-SHA512: 808e100ee1e371f7746a479ddfb237c6895935cffefc0e49033505492a03288013d5c20386af30f2a7dca8ad0c0628bdb6673dcb5cc4fcf4d0183b0ec65ce941
fae211c0ae lint: Start to use py lint scripts (MarcoFalke)
fa82e890e7 Move lint script and data file to avoid lint- prefix (MarcoFalke)
Pull request description:
ACKs for top commit:
fjahr:
tACK fae211c0ae
Tree-SHA512: f8272a1bab9efb8203cac121710baae68f01f79e520ad71ff15aa516d19763d61c088b411b019de105a6a30e7ee3c274814d59963f6ac22ba1084560fb601f45
