Commit Graph

612 Commits

Author SHA1 Message Date
Rusty Russell
1093b43ccf lnprototest: remove.
At this point it needs a complete rewrite to be useful, and it's just
constraining development.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-31 14:10:51 +10:30
Rusty Russell
94a539ee3d Makefile: add CLN_NEXT_VERSION, functions encoding deprecation schedule.
Each feature has a name, and says when deprecation begins and ends.

There's an API coming to allow you to re-enable on a per-feature basis
even if it's ended (as long as it's not been removed from the code ofc!).

Default end is 6 months after deprecation, i.e. we complain about it
at that point, if we can detect its use.

e.g, a standard deprecation in v24.05:

v24.02: allowed
v24.02 with mods: allowed

master after v24.02: allowed unless deprecated APIs disabled.
v24.05: allowed unless deprecated APIs disabled.
v24.08: allowed unless deprecated APIs disabled.

v24.11: allowed unless deprecated APIs disabled, but logs at BROKEN level.

v25.02: allowed only if --i-promise-to-fix-broken-api-user=FEATURE.
v25.05: code is actually removed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-26 10:30:22 +10:30
Jon Griffiths
1b2d81f7ce wally: remove now-redundant BUILD_ELEMENTS define
As of wally v1.0.0, Elements/Liquid support is enabled and part of the
library ABI by default.

Changelog-None

Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
2024-01-16 15:47:03 +01:00
Peter Neuroth
8f8202daa7 meta: Update release checklist
Add a version bump for clnrest and the main poetry.lock to the makefile
and update the release checklist to include the new target.

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2023-11-16 15:20:33 +01:00
Rusty Russell
2ea3c6155f Makefile: fix update-mocks dependencies.
It definitely needs generated source files.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-23 15:48:50 +10:30
Matt Whitlock
15b1ccb149 Makefile: use xargs -0 consistently
In its default mode, `xargs` interprets any whitespace as a delimiter,
and it interprets single and double quotes and backslashes specially.
This can play havoc when fed by tools that don't escape these special
characters in their output.

Fortunately, `xargs` (at least the GNU version of it) has a `-0` option
that changes the behavior so that only NUL characters act as delimiters,
and no other characters have any special meaning. Use this option
consistently to avoid any nasty surprises.

Note that `git-ls-files` has a `-z` option that causes it to terminate
filenames on output with a NUL character instead of a newline, and
`grep` has a `-z` option that causes it to operate on NUL-terminated
records rather than on lines. Use these options together with
`xargs -0`.

Note: This commit corrects an oversight introduced in
9d94687b0d.

Changelog-None
2023-09-27 13:01:39 +09:30
Rusty Russell
9d94687b0d Makefile: don't fail find -delete if we run in parallel.
Use xargs rm -f which doesn't care if it's already deleted.

```
find contrib/pyln-grpc-proto/pyln/ -type f -name "*.py.bak" -delete
find contrib/pyln-grpc-proto/pyln/ -type f -name "*.py.bak" -delete
find: cannot delete ‘contrib/pyln-grpc-proto/pyln/grpc/node_pb2_grpc.py.bak’: No such file or directory
make: *** [Makefile:390: contrib/pyln-grpc-proto/pyln/grpc/node_pb2.py] Error 1
make: *** Waiting for unfinished jobs....
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-21 20:08:24 +09:30
Rusty Russell
3d3bf87b34 build: remove all trace of DEVELOPER.
If you previously configured with `--enable-developer` we turn that into `--enable-debugbuild`.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: build: `--enable-developer` arg to configure (and DEVELOPER variables): use `./configure --enable-debugbuild` and `developer` setting at runtime.
2023-09-21 20:08:24 +09:30
Rusty Russell
da498c55bc doc: update to BOLT 6e85df448bfee7d10f26aabb06b8eba3d7505888 "BOLT2: change "reject" to "fail" in channel opening requirements (#1104)"
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-20 13:56:46 +09:30
Rusty Russell
4dfbaa3ebe bolt11: update to BOLT 7dda8f84edb8655766d83619cb41110c1bf43f85 "Bolt11: min_final_cltv_expiry_delta is optional, not required (#1100)"
Since we changed the default, it used to be required to set it.  That was a while ago, though, so we can make it optional again.

Changelog-Changed: Protocol: `invoice` no longer explicitly encodes `c` if it's the default (18)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-20 13:56:46 +09:30
Rusty Russell
98d2dc2d4c wire: update to BOLT 17ceba42dc6607b637a713a9bc051f63c1357edd "BOLT 4: onion message support."
We don't need to patch it in anymore, now it's merged.  However, we do
move the message itself from onion_wire.csv to peer_wire.csv (we
should get more sophisticated with our parsing, but this works for
now!).

The resulting peer_wire.csv is identical, the onion_wire.csv file is
slightly reordered.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-20 13:56:46 +09:30
Rusty Russell
b22202dccd doc: update to BOLT cded2df1fd0095c947af02a52e28b16c66bdfc5c "Fix undeclared reference in onion-routing"
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-20 13:56:46 +09:30
Rusty Russell
68a6084bab lightningd: update to BOLT 7d3ef5a6b20eb84982ea2bfc029497082adf20d8 "Allow unset onion_hash in invalid_onion_blinding (#1093)"
Explicitly allow all-zero in the onion_hash: we didn't do anything except log if it was unexpected anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-20 13:56:46 +09:30
Rusty Russell
6a2632d50b doc: update to BOLT 7f53a3e46ed59f19463447d5fa63b3e484c708a5 "BOLT 8: make it clear that there are *two* chaining keys."
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-20 13:56:46 +09:30
Rusty Russell
4413bfa6be wire: update to BOLT aad959a297ff66946effb165518143be15777dd6 "tools: remove option handling now last optional field was removed."
No effect on us, we didn't use this field anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-20 13:56:46 +09:30
Rusty Russell
df80a2c056 doc: update to BOLT 3747ba83022cd385093df2696ed342f1e41e31b3 "Remove requirements to disconnect on warnings"
Now we don't do that anymore (at least, for sending) we can update bolt quotes to match.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-20 13:56:46 +09:30
Rusty Russell
821d5f48ea doc: update to BOLT 50b2df24a27879e8329712c275db78876fd022fe "Update onion errors since we allow overpaying or under-CLTVing"
We already do this behaviour, we just didn't adjust comments.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-20 13:56:46 +09:30
Rusty Russell
0d70daa5ee doc: update to BOLT 29c14c6e12cbdf33f6b724094c81658a614d2e02 "fix typos"
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-20 13:56:46 +09:30
Rusty Russell
91cb4835d3 Makefile: don't delete python-generated files in distclean.
We do not, in fact, require Python to build, so we should still work
after `make distclean`.

Fixes:  #6536
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-15 20:27:24 +02:00
Rusty Russell
4431e4199f Makefile: regnerate config.vars if configure changes (recently PYTHON var).
We didn't want a rule to generate config.vars, since user should run configure manually
with their desired options.  However, we do want to be able to *refresh* it, because
otherwise, if we need a new configuration var for our Makefile, it won't get refreshed:

```
$ make
CC: cc -DBINTOPKGLIBEXECDIR="../libexec/c-lightning" -Wall -Wundef -Wmissing-prototypes -Wmissing-declarations -Wstrict-prototypes -Wold-style-definition -Werror -Wshadow=local -std=gnu11 -g -fstack-protector-strong  -I ccan -I external/libwally-core/include/ -I external/libwally-core/src/secp256k1/include/ -I external/jsmn/ -I external/libbacktrace/ -I external/gheap/ -I external/build-x86_64-linux-gnu/libbacktrace-build -I external/libsodium/src/libsodium/include -I external/libsodium/src/libsodium/include/sodium -I external/build-x86_64-linux-gnu/libsodium-build/src/libsodium/include -I . -I/usr/local/include  -I/usr/include/postgresql   -DCCAN_TAKE_DEBUG=1 -DCCAN_TAL_DEBUG=1 -DCCAN_JSON_OUT_DEBUG=1 -DSHACHAIN_BITS=48 -DJSMN_PARENT_LINKS  -DCOMPAT_V052=1 -DCOMPAT_V060=1 -DCOMPAT_V061=1 -DCOMPAT_V062=1 -DCOMPAT_V070=1 -DCOMPAT_V072=1 -DCOMPAT_V073=1 -DCOMPAT_V080=1 -DCOMPAT_V081=1 -DCOMPAT_V082=1 -DCOMPAT_V090=1 -DCOMPAT_V0100=1 -DCOMPAT_V0121=1  -DBUILD_ELEMENTS=1 -c -o
PYTHONPATH=contrib/msggen  contrib/msggen/msggen/__main__.py
LD: cc     config.vars  -Lexternal/build-x86_64-linux-gnu -lwallycore -lsecp256k1 -ljsmn -lbacktrace -lsodium -L/usr/local/include -lm -lsqlite3  -lz  -L/usr/lib/x86_64-linux-gnu -lpq -o
/bin/sh: 1: contrib/msggen/msggen/__main__.py: Permission denied
make: *** [Makefile:371: cln-grpc/proto/node.proto] Error 126
make: *** Waiting for unfinished jobs....
```

Here, PYTHON is the new var (here, unset).  If we were building a binary, we'd depend on config.h,
and the Makefile says how to refresh that.  But the Makefile includes config.vars (causing an
implicit dependency) so we really should have a rule to make that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-13 14:36:29 +09:30
Rusty Russell
e3b1549b64 configure: don't assume python3, create $PYTHON var.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-28 12:18:06 +02:00
Erik De Smedt
0a4ac04c00 Use python3 instead of python in Makefile
Debian likes this more
2023-08-27 15:16:00 +02:00
ShahanaFarooqui
620a61ee09 makefile: py plugins installed twice with make install
Steps to Reproduce: Run `sudo make install` then again `sudo make install` without uninstalling plugins in between.
First command will install the plugin at `/usr/local/libexec/c-lightning/plugins/clnrest` as expected but the
second command will install it inside the first folder like `/usr/local/libexec/c-lightning/plugins/clnrest/clnrest`.

Fix: Check and delete if the folder already exists in `install-program`.

Changelog-None
2023-08-05 17:12:17 +09:30
Lagrang3
c02f175a75 Import lqueue from CCAN
[ Split into separate commit --RR ]
Signed-off-by: Lagrang3 <eduardo.quintana@pm.me>
2023-07-31 12:58:56 +09:30
Shahana Farooqui
94205c094f plugins/clnrest: Update clnrest dependencies in scripts and Dockerfiles
Adding python3-json5 python3-flask python3-gunicorn flask_restx pyln-client
Also run `pip3 install -r plugins/clnrest/requirements.txt`
2023-07-25 19:20:16 +09:30
Shahana Farooqui
abcfda133f plugin: On-boarding default python plugins
plugin: On-boarding default python plugins
2023-07-25 19:20:16 +09:30
niftynei
f5bac83232 makefile: use recursive delete to target extraneous .baks
Make this a bit less difficult to see what's going on; use common
variables for the paths etc.

Also make the final `rm` step recursive, same as the above find step
which patches the files.

Files are in `contrib/pyln-grpc-proto/pyln/grpc`, but the upper level
find command acts at a level above.
2023-07-25 10:49:34 +09:30
Rusty Russell
e91475c3de Makefile: check-source-no-cppcheck.
Recent cppcheck doesn't like our code; until we fix that, make it easy
to run every other source check.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-07-22 11:08:04 +09:30
Rusty Russell
5132fea44e docs: fix up internal references now files have moved.
And tools/check-markdown.sh didn't do anything: remove it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-07-16 12:59:40 +09:30
niftynei
3a18d8b355 mac-fix: sed doesn't work the same on macos
https://stackoverflow.com/a/14813278

Found-by: @ddustin
2023-07-13 18:59:20 +09:30
Vincenzo Palazzo
da91e70d3e build: Regenerate the generated file as the default target
When modifying the schema, regenerating the file can be challenging.
This change sets the autogenerate file as the default target, m
aking the process more convenient and simplifying our workflow.

Fixes https://github.com/ElementsProject/lightning/issues/6365
Report-by: Dusty Daemon
Changelog-None
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2023-07-11 11:41:15 +09:30
Rusty Russell
0d1c46a5e5 Makefile: rebuild ccan when ccan_compat.h changes.
As it did recently:

```
/usr/bin/ld: bitcoin/pubkey.o: in function `pubkey_to_hash160':
/home/rusty/devel/cvs/lightning/bitcoin/pubkey.c:101: undefined reference to `ccan_ripemd160'
/usr/bin/ld: bitcoin/script.o: in function `hash160':
/home/rusty/devel/cvs/lightning/bitcoin/script.c:22: undefined reference to `ccan_ripemd160'
/usr/bin/ld: bitcoin/script.o: in function `bitcoin_wscript_htlc_offer':
/home/rusty/devel/cvs/lightning/bitcoin/script.c:662: undefined reference to `ccan_ripemd160'
/usr/bin/ld: bitcoin/script.o: in function `bitcoin_wscript_htlc_receive':
/home/rusty/devel/cvs/lightning/bitcoin/script.c:780: undefined reference to `ccan_ripemd160'
collect2: error: ld returned 1 exit status
make: *** [Makefile:660: devtools/create-gossipstore] Error 1
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-07-10 13:37:22 +02:00
Christian Decker
6583f0dead pyln: Create a dedicated pyln-grpc-proto package
Having the grpc bindings in the pyln-testing package was always a bit
strange, however it came with additional issues. Due to the way that
protos are handled by protobuf, any name clash, independently of where
we import the protos, would cause an import error. This usually
happens in diamond-pattern dependencies, and so pull out the generated
files into their own package that everyone else can rely on.

Changelog-None
2023-07-10 13:47:38 +09:30
Rusty Russell
fa596a9446 configure: --enable-debugbuild flag for debug builds.
This controls debug flags for the build, rather than --developer,
which is going away.

I thought about making this flag control the RUST_PROFILE too, but
it seems that we want that set to "release" for CI, whereas for the
C code we want --enable-debugbuild.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-07-09 16:49:48 +09:30
Bitkarrot
910630c497
add m1 native lib paths to Makefile
add m1 native lib paths to Makefile (also need PKG_CONFIG_PATH)

[ Squashed into a single commit --RR ]
2023-06-26 08:25:21 +09:30
Rusty Russell
69fc9c28bc Makefile: correctly erase generated contrib/pyln-testing/pyln/testing/ in distclean, and rebuild by default.
And add contrib/pyln-testing/pyln/testing/grpc2py.py since we didn't previously know
how to build it!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-23 13:58:31 +09:30
Matt Whitlock
8d737cc4bf Makefile: use grouped targets for recipes with multiple fixed outputs
See the section headed "Rules with Grouped Targets" on the Texinfo page
`(make)Multiple Targets`.

Without this fix, Make does not know that these recipes unconditionally
make *all* of their named targets regardless of which target triggers
their execution, and Make will blissfully execute multiple instances of
any such recipe in parallel, not only wasting CPU cycles but potentially
producing incorrect results if the recipe is not atomic in its effects
on the file system.  With this fix, Make understands that it need only
execute such a recipe once to make all of its targets.

In pursuit of the above, move and combine two redundant msggen recipes
into the top-level Makefile, and populate its grouped targets from the
subordinate Makefiles.

Changelog-None
2023-06-23 13:58:31 +09:30
Matt Morehouse
aa9356b961 make: allow Clang coverage instrumentation
Clang's coverage instrumentation [1] is the best I've seen, with
precision down to the expressions within a line of code. Add an option
to use this instrumentation for better coverage reports.

[1] https://clang.llvm.org/docs/SourceBasedCodeCoverage.html
2023-06-15 10:47:56 +09:30
Jon Griffiths
8b4136bad5 build: remove gmp
Core and secp have not used gmp for a very long time now (core disabled
it in 2015).

Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
2023-05-24 18:21:43 +09:30
Rusty Russell
e7d4c3175a build: remove --enable-experimental-features / EXPERIMENTAL_FEATURES
Changelog-EXPERIMENTAL: Build: all experimental features are now runtime-enabled; no more ./configure --enable-experimental-features
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-05-23 09:34:08 +09:30
Rusty Russell
4deb552fe9 build: don't generate experimental variants of wire files.
We no longer have any experimental-only wire definitions.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-05-23 09:34:08 +09:30
Christian Decker
0687fecf0d make: Use the CLN_PLUGIN_EXAMPLES variable for testbin
Listing things multiple times is error-prone, so use the variable we
already have :-)
2023-04-13 18:13:28 -07:00
Matt Morehouse
14afa6efe7 make: add check-fuzz target
The target builds and runs each fuzz target on its seed corpus.
2023-04-09 12:41:27 +09:30
Rusty Russell
d4ffc75691 Makefile: update to latest BOLT text.
In particular:
	- Bolt 4: add route blinding construction
	- Bolt 4: add blinded payments

And this means it's not experimental, so we can turn it on
by default!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Protocol: blinded payments are now supported by default (not just with `--experimental-onion-messages`)
2023-04-07 21:22:56 +09:30
Rusty Russell
458a85042b Makefile: update to BOLT 20066dc2aba906f37f3be5a810ae67040f265377
"BOLT 03: fix static-remote same amt and pre-image test vector"

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-07 21:22:56 +09:30
Rusty Russell
f26b1166b7 Makefile: update bolts a0bbe47b0278b4f152dbaa4f5fab2562413a217c
"BOLT 04: remove associated data from test vector"

(We actually use merge point).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-07 21:22:56 +09:30
Rusty Russell
dfa6c0ca52 Makefile: bolt version b38156b9510c0562cf50f8758a64602cc0315c19
"Allow nodes to overshoot final htlc amount and expiry (#1032)"

Note that this also renamed `min_final_cltv_expiry` to the more-correct
`min_final_cltv_expiry_delta`.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-07 21:22:56 +09:30
Rusty Russell
fdf9b13bdb Makefile: update bolts fc40879995ebc61cc50dfd729512f17afb15b355.
"Allow nodes to overshoot the MPP `total_msat` when paying (#1031)"

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: Allow slight overpaying, even with MPP, as spec now recommends.
2023-04-07 21:22:56 +09:30
Rusty Russell
15f8e1e63c Makefile: update bolts to 60cfb5972ad4bec4c49ee0f9e729fb3352fcdc6a.
"BOLT 4: Remove legacy format, make var_onion_optin compulsory."

This also renamed the redundant "tlv_payload" to "payload", so we
replace "tlv_tlv_payload" with "tlv_payload" everyhere!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-07 21:22:56 +09:30
Rusty Russell
1e24d4a0a0 Makefile: fix check-gen-update to diff *all* files.
This would have caught the missing man page change!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-06 09:10:53 +09:30