Commit Graph

506 Commits

Author SHA1 Message Date
Rusty Russell
81b2092cfc CCAN: add base64 module.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-22 11:56:30 +02:00
Rusty Russell
947cc4730c Makefile: be sure to build generated sources before tests.
In particular, they are allowed to include .c files!

Here's `make check-units` on a maintainer-clean tree:

```
onchaind/test/run-onchainstress.c:4:10: fatal error: ../../hsmd/hsmd_wiregen.c: No such file or directory
    4 | #include "../../hsmd/hsmd_wiregen.c"
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make: *** [Makefile:276: onchaind/test/run-onchainstress.o] Error 1
make: *** Waiting for unfinished jobs....
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-08 13:47:30 +02:00
Christian Decker
71740283e4 gci: Add missing tooling 2021-09-28 18:34:43 +02:00
Rusty Russell
3dbf3b057a Makefile: remove NO_PYTHON flag in anticipation of making it compulsory.
And note the EXPERIMENTAL_DUAL_FUND env var in HACKING.md.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-22 15:25:14 +02:00
Rusty Russell
2736e997f6 ccan: remove autodata.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-21 18:04:43 +02:00
niftynei
23a6c00496 Makefile: import latest spec, which includes quickclose.
It also gets rid of the requirement that close negotiation fee maximum
is the old commitment transaction.  We still do that, however, to
avoid surprising old peers.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-09 12:04:48 +09:30
Rusty Russell
fb4edc2938 Makefile: update bolt version to include option_anchors_zero_fee_htlc_tx.
This touches a lot of text, mainly to change "if `option_anchor_outputs`"
to "if `option_anchors`"

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-09 12:04:48 +09:30
Rusty Russell
1b2c6964fd Makefile: update spec version
This includes the new bolt11 test vectors, and also removes the
requirement that HTLCs be less than 2^32 msat.  We keep that for now
because Electrum enforced it on receive: in two releases we will stop
that too.

So no longer warn about needing mpp in that case either.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: Protocol: No longer restrict HTLCs to
2021-09-09 12:04:48 +09:30
Rusty Russell
d637680476 Makefile: Update BOLT version to cover bolt11 test vectors with payment secret.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-09 12:04:48 +09:30
niftynei
8654c817da sendcustommsg: promote to non-dev
Enable non-dev builds to send custom messages.

Preserves 'dev-' for compat-enabled builds.

Changelog-Changed: JSON-RPC: moved dev-sendcustommsg to sendcustommsg
2021-07-14 14:39:44 -05:00
Rusty Russell
3aafe8cacf Makefile: pytest target should depend on ALL_TEST_PROGRAMS.
In particular, this includes the test plugins!

Not sure why this breaks *now* though.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-11 13:54:20 +09:30
Rusty Russell
bca8427317 Makefile: allow postfixes to SHA256STAMP.
For markdown, there's no simple comment prefix: we need a postfix too.
We also need to use "" since we want to use ' in some of the Makefiles
in future when V=1.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-27 20:28:49 +09:30
Rusty Russell
40b264cf95 Makefile: fix $(FORCE) in sub-Makefiles
It needs to be defined before we include them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-27 20:28:49 +09:30
Rusty Russell
d3f370944e Makefile: update to latest spec.
This includes anysegwit and the updated HTLC tiebreak test vector.  It
also adds explicit wording for invalid per_commitment_secret (which
nicely matches our code already!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-26 20:01:03 +09:30
Rusty Russell
d868d4470a Makefile: don't let update-mocks run unless DEVELOPER and EXPERIMENTAL_FEATURES.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Rusty Russell
25b5e1e099 update-mocks: make sure we cover all test programs.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Rusty Russell
cc6f1fd120 update-mocks: don't try to regenerate files.
If you do update-mocks in a dirty tree, the recursive make that it
uses will try to rebuild things!  Suppress that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Christian Decker
0f2009c7a3 make: Do not scan contrib for discouraged functions 2021-05-15 09:31:12 +09:30
Christian Decker
6c67afeb03 make: Add CPPCHECK_OPTS to allow excluding files from the check
With swig we now have C files that are generated with tools that are
not under our control, so provide an escape hatch for them.
2021-05-15 09:31:12 +09:30
Christian Decker
7f15f86718 libhsmd: First working build script for libhsmd-python
Changelog-Added: libhsmd: Added python bindings for `libhsmd`
2021-05-15 09:31:12 +09:30
Christian Decker
e02e972729 cleanup: Make lnprototest run only with DEVELOPER=1 2021-05-03 11:20:15 +09:30
Christian Decker
1bebdfdd5f make: Setup the PYTHONPATH to include the pyln-spec packages
They are currently not installable due to circular dependencies and
stuff, so we just add their source to the path and lnprototest will
pick them up from there.
2021-04-08 10:34:14 +09:30
niftynei
d2477721b9 make: add lnprototests to check!!
We've got great protocolo levle tests written, let's make sure we
put them to good use!
2021-04-08 10:34:14 +09:30
niftynei
e3ad60150d make: include lnprototest in pythonpath 2021-04-08 10:34:14 +09:30
Rusty Russell
3c5ea7fd8e Makefile: make it easier to check bolt text.
Instead of "only check suffix quotes when EXPERIMENTAL_FEATURES", make
it so we only check suffix quotes if you override BOLTVERSION on the
cmdline.

Before this, "make check-source-bolt" was effectively a NOOP with
--enable-experimental-features.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-04-07 14:34:39 +09:30
Rusty Russell
3ccb6d6e7a Makefile: update to latest BOLT versions.
The main change which affects us is that 2016 blocks to forget a channel
is a fixed number in the spec; we make this clear by renaming the
(developer-only) max_funding_unconfirmed to dev_max_funding_unconfirmed
and making it compile DEVELOPER only.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-04-07 14:34:39 +09:30
niftynei
3e8f575f9e dual-funding: convert to runtime flag, --experimental-dual-fund
You can now activate dual-funded channels using the
`--experimental-dual-fund` flag

Changelog-Changed: Config: `--experimental-dual-fund` runtime flag will enable dual-funded protocol on this node
2021-03-25 20:05:11 +10:30
Rusty Russell
9224d9a500 gossipd: use modern 'sync_complete' field.
We assume if they set this to 0 (which nobody did previously), they're
using it as a modern flag and use it to indicate when they're
finished.  Otherwise, we count how many blocks they've sent and use
that to determine whether they've finished.

See: https://github.com/lightningnetwork/lightning-rfc/pull/826

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: we use `sync_complete` for gossip range query replies, with detection for older spec nodes.
2021-02-25 13:23:16 +10:30
Rusty Russell
45ce7380af doc: update BOLT quotes (to include links for anchor outputs).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-02-25 13:23:16 +10:30
Rusty Russell
1a859ff583 doc: update BOLT quotes (to include same-transmission-order rule).
We implement this, but now we have an appropriate quote!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-02-25 13:23:16 +10:30
Rusty Russell
36bf037ca7 doc: update BOLT quotes (to include preimage leakage).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-02-25 13:23:16 +10:30
Rusty Russell
93d3b0d126 doc: update BOLT quotes (to include LOW-S-standard rule).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-02-25 13:23:16 +10:30
Christian Decker
927eda065b make: Add target to check generated/derived files were checked in
A new target `check-gen-updated` to verify that all derived/generated
files that were modified were also checked in. This is used on CI to
check, and is not added to `check` since it'll complain on dirty
trees, i.e., before the devs check in their changes.

Suggested-by: Rusty Russell <@rustyrussell>
2021-02-11 09:48:55 +01:00
Christian Decker
8c94d1a358 make: Remove hardcoded timeout to pytest
This should really be set by the environment by creating either a
pytest.ini or setting PYTEST_OPTS envvar.
2021-01-29 10:29:09 +10:30
Rusty Russell
25fcc4ddbf Makefile: separate check-python targets.
There's a 60 second delay in one of the contrib tests, and I just want
to run flake8 on my alterations.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-02 10:38:04 +10:30
Rusty Russell
497e71ca13 Makefile: fix bashism in NO_PYTHON
== is a bash extension; in shell it's a single =:

```
/bin/sh: 1: [: unexpected operator
sql-rewrite wallet/db_postgres_sqlgen.c
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-26 21:28:36 -05:00
Rusty Russell
375040a3d9 Makefile: fix dependencies for check-pyln-%
$(BIN_PROGRAMS) is defined afterwards, leading to the problem that
lightningd doens't get rebuilt (in fact, it was running my installed
lightningd instead!).

It also needs $(PLUGINS) and the subdaemons, so hoist them all.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-22 12:14:34 +10:30
Rusty Russell
a8177e9013 Makefile: make check-includes check all the non-generated files.
Note that check-whitespace and check-bolt already do this, so we
can eliminate redundant lines in common/Makefile and bitcoin/Makefile.

We also include the plugin headers in ALL_C_HEADERS so they get
checked.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-22 12:14:34 +10:30
Rusty Russell
5e865ce42b Makefile: unify generated files definition.
We change gen_ to _gen, because filtering on gen_% doesn't work if they're
in subdirectories :(

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-22 12:14:34 +10:30
Antoine Poinsot
b3594690d9 configure: allow to run with UB sanitizer
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot
62b54d0125 build: introduce a fuzzing mode
This adds a new configuration, --enable-fuzzing (which is more than
welcome to be coupled with --enable-address-sanitizer), to pass the
fuzzer sanitizer argument when compiling objects. This allows libfuzzer
to actually be able "to fuzz" by detecting coverage and be smart when
mutating inputs.

As libfuzzer brings its own ~~fees~~ main(), we compile objects with
fsanitize=fuzzer-no-link, and special-case the linkage of the fuzz
targets.

A "lib" is added to abstract out the interface to the fuzzing tool used.
This allow us to use the same targets to fuzz using AFL, hongfuzz or w/e
by adding their entrypoints into libfuzz. (h/t to practicalswift who
introduced this for bitcoin-core, which i mimiced)

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Michael Schmoock
db19f19eb3 chore: adds env NO_PYTHON check to Makefiles
This adds an environment variable $NO_PYTHON check to Makefiles so that:
 - It checks and runs into an defined error instead of some python hickup:
   `ModuleNotFoundError: No module named 'mako'`
 - makes it possible to manually export this environment variable NO_PYTHON=1
   to run the same testcase that travis is running on job 1 which causes
   so much pain ;)

Changelog-None
2020-10-19 13:17:03 +02:00
Christian Decker
30ecb17204 make: Include the $PYTEST_OPTS when running pytest 2020-10-13 14:30:45 +02:00
Christian Decker
2a176d2ef7 make: Enable mypy checks on pyln-testing 2020-09-28 09:19:46 +09:30
Christian Decker
e13c435aca make: Set $PYTHONPATH once
This is used a number of times all over the place, and there is no advantage
in setting it over and over again.
2020-09-28 09:19:46 +09:30
Christian Decker
2740f2eccb pyln: Use pyln-client Makefile for checks
Kinda uses an antipattern of `cd`ing into the directory and calling `make`
there, but this keeps the contrib dirs self-contained so we can split them out
eventually.
2020-09-28 09:19:46 +09:30
Rusty Russell
7260d9ea3d plugins: generate list of plugins more atomically, respect V=1 and --quiet.
I got a corrupt file, which looked like multiple concurrent attempts
to build it.  So instead, build it in one command, but also use
VERBOSE so we print correctly with V=1 (and --quiet).

Also move into plugins/ where it logically belongs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-24 13:19:59 +02:00
Rusty Russell
4e67b4fd03 Makefile: work with shasum -a as well as sha256sum.
At least on my Ubuntu box, they're compatible. If they're not, we need
to disable regeneration altogether.

Fixes: #4075
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-23 14:34:42 +02:00
Matt Whitlock
ec29f04b5f Makefile: sort input filenames to SHA256STAMP
Make's 'wildcard' function returns matching file paths in an
unpredictable order, thereby causing false mismatches in the produced
SHA256STAMPs from system to system. This commit sorts the file paths
given to 'cat' to make the stamps deterministic.

Changelog-Fixed: The build system no longer spuriously regenerates generated sources due to differences in `readdir`(3) sort order.
2020-09-17 10:23:40 +09:30
Christian Decker
274540f556 repro: Allow dashes in the version number 2020-09-16 06:27:12 +09:30