Commit Graph

622 Commits

Author SHA1 Message Date
Rusty Russell
2cb6da14fb BOLT: catch up now Dual-funding is included.
Mainly we don't need to patch the spec anywhere near as much.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-05-09 16:14:23 -05:00
Rusty Russell
cca784c9d0 BOLT: update to include 2016 for max_htlc_cltv.
And deprecate the --max-locktime-blocks which allows them to set it.

Hilariously, the spec misspells CLTV as CTLV at one point, so we work around it for now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-05-09 16:14:23 -05:00
Rusty Russell
bbb6fd4413 BOLTs: catch up with minor clarifications.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-05-09 16:14:23 -05:00
Rusty Russell
b22095b275 ccan: update tal so we can annote allcators as never returning NULL.
This helps with -O3 warnings on these commits.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-05-06 20:51:19 -05:00
ShahanaFarooqui
720af34d43 plugins/wss-proxy: Plugin included in core lightning 2024-04-22 09:12:09 +09:30
Rusty Russell
037eb08fd6 Makefile: update CLN_NEXT_VERSION so deprecations tick over.
Indeed, we now need to adjust our tests to use --i-promise-to-fix-broken-api-user
for the specific APIs past end-of-life.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-03-25 15:02:35 +10:30
Christian Decker
6e980c45eb make: The protobuf files are a grouped target 2024-02-21 11:59:41 +01:00
Christian Decker
cfea45d827 make: Add missing dependency between schemas and schema bundle
Reported-by: microsatoshi
2024-02-09 12:20:45 +01:00
Dusty Daemon
2cbf426296 make: always generate proto files sans rust
Generating the msggen proto files doesn’t require rust (even though it generates rust files).

Changelog-None
2024-02-08 15:05:18 +01:00
Christian Decker
00fbd5977f msggen: Start making the msggen library a standalone tool
There are quite some things we want to generate from the schema
definitions, both inside and outside of CLN itself. By bundling the
schemas into the library we can make use of the tooling without
running in the CLN source tree. This is in part used to generate some
of the interfaces in Greenlight.

Changelog-None
2024-02-08 15:03:34 +01:00
Rusty Russell
1093b43ccf lnprototest: remove.
At this point it needs a complete rewrite to be useful, and it's just
constraining development.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-31 14:10:51 +10:30
Rusty Russell
94a539ee3d Makefile: add CLN_NEXT_VERSION, functions encoding deprecation schedule.
Each feature has a name, and says when deprecation begins and ends.

There's an API coming to allow you to re-enable on a per-feature basis
even if it's ended (as long as it's not been removed from the code ofc!).

Default end is 6 months after deprecation, i.e. we complain about it
at that point, if we can detect its use.

e.g, a standard deprecation in v24.05:

v24.02: allowed
v24.02 with mods: allowed

master after v24.02: allowed unless deprecated APIs disabled.
v24.05: allowed unless deprecated APIs disabled.
v24.08: allowed unless deprecated APIs disabled.

v24.11: allowed unless deprecated APIs disabled, but logs at BROKEN level.

v25.02: allowed only if --i-promise-to-fix-broken-api-user=FEATURE.
v25.05: code is actually removed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-26 10:30:22 +10:30
Jon Griffiths
1b2d81f7ce wally: remove now-redundant BUILD_ELEMENTS define
As of wally v1.0.0, Elements/Liquid support is enabled and part of the
library ABI by default.

Changelog-None

Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
2024-01-16 15:47:03 +01:00
Peter Neuroth
8f8202daa7 meta: Update release checklist
Add a version bump for clnrest and the main poetry.lock to the makefile
and update the release checklist to include the new target.

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2023-11-16 15:20:33 +01:00
Rusty Russell
2ea3c6155f Makefile: fix update-mocks dependencies.
It definitely needs generated source files.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-23 15:48:50 +10:30
Matt Whitlock
15b1ccb149 Makefile: use xargs -0 consistently
In its default mode, `xargs` interprets any whitespace as a delimiter,
and it interprets single and double quotes and backslashes specially.
This can play havoc when fed by tools that don't escape these special
characters in their output.

Fortunately, `xargs` (at least the GNU version of it) has a `-0` option
that changes the behavior so that only NUL characters act as delimiters,
and no other characters have any special meaning. Use this option
consistently to avoid any nasty surprises.

Note that `git-ls-files` has a `-z` option that causes it to terminate
filenames on output with a NUL character instead of a newline, and
`grep` has a `-z` option that causes it to operate on NUL-terminated
records rather than on lines. Use these options together with
`xargs -0`.

Note: This commit corrects an oversight introduced in
9d94687b0d.

Changelog-None
2023-09-27 13:01:39 +09:30
Rusty Russell
9d94687b0d Makefile: don't fail find -delete if we run in parallel.
Use xargs rm -f which doesn't care if it's already deleted.

```
find contrib/pyln-grpc-proto/pyln/ -type f -name "*.py.bak" -delete
find contrib/pyln-grpc-proto/pyln/ -type f -name "*.py.bak" -delete
find: cannot delete ‘contrib/pyln-grpc-proto/pyln/grpc/node_pb2_grpc.py.bak’: No such file or directory
make: *** [Makefile:390: contrib/pyln-grpc-proto/pyln/grpc/node_pb2.py] Error 1
make: *** Waiting for unfinished jobs....
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-21 20:08:24 +09:30
Rusty Russell
3d3bf87b34 build: remove all trace of DEVELOPER.
If you previously configured with `--enable-developer` we turn that into `--enable-debugbuild`.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: build: `--enable-developer` arg to configure (and DEVELOPER variables): use `./configure --enable-debugbuild` and `developer` setting at runtime.
2023-09-21 20:08:24 +09:30
Rusty Russell
da498c55bc doc: update to BOLT 6e85df448bfee7d10f26aabb06b8eba3d7505888 "BOLT2: change "reject" to "fail" in channel opening requirements (#1104)"
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-20 13:56:46 +09:30
Rusty Russell
4dfbaa3ebe bolt11: update to BOLT 7dda8f84edb8655766d83619cb41110c1bf43f85 "Bolt11: min_final_cltv_expiry_delta is optional, not required (#1100)"
Since we changed the default, it used to be required to set it.  That was a while ago, though, so we can make it optional again.

Changelog-Changed: Protocol: `invoice` no longer explicitly encodes `c` if it's the default (18)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-20 13:56:46 +09:30
Rusty Russell
98d2dc2d4c wire: update to BOLT 17ceba42dc6607b637a713a9bc051f63c1357edd "BOLT 4: onion message support."
We don't need to patch it in anymore, now it's merged.  However, we do
move the message itself from onion_wire.csv to peer_wire.csv (we
should get more sophisticated with our parsing, but this works for
now!).

The resulting peer_wire.csv is identical, the onion_wire.csv file is
slightly reordered.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-20 13:56:46 +09:30
Rusty Russell
b22202dccd doc: update to BOLT cded2df1fd0095c947af02a52e28b16c66bdfc5c "Fix undeclared reference in onion-routing"
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-20 13:56:46 +09:30
Rusty Russell
68a6084bab lightningd: update to BOLT 7d3ef5a6b20eb84982ea2bfc029497082adf20d8 "Allow unset onion_hash in invalid_onion_blinding (#1093)"
Explicitly allow all-zero in the onion_hash: we didn't do anything except log if it was unexpected anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-20 13:56:46 +09:30
Rusty Russell
6a2632d50b doc: update to BOLT 7f53a3e46ed59f19463447d5fa63b3e484c708a5 "BOLT 8: make it clear that there are *two* chaining keys."
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-20 13:56:46 +09:30
Rusty Russell
4413bfa6be wire: update to BOLT aad959a297ff66946effb165518143be15777dd6 "tools: remove option handling now last optional field was removed."
No effect on us, we didn't use this field anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-20 13:56:46 +09:30
Rusty Russell
df80a2c056 doc: update to BOLT 3747ba83022cd385093df2696ed342f1e41e31b3 "Remove requirements to disconnect on warnings"
Now we don't do that anymore (at least, for sending) we can update bolt quotes to match.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-20 13:56:46 +09:30
Rusty Russell
821d5f48ea doc: update to BOLT 50b2df24a27879e8329712c275db78876fd022fe "Update onion errors since we allow overpaying or under-CLTVing"
We already do this behaviour, we just didn't adjust comments.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-20 13:56:46 +09:30
Rusty Russell
0d70daa5ee doc: update to BOLT 29c14c6e12cbdf33f6b724094c81658a614d2e02 "fix typos"
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-20 13:56:46 +09:30
Rusty Russell
91cb4835d3 Makefile: don't delete python-generated files in distclean.
We do not, in fact, require Python to build, so we should still work
after `make distclean`.

Fixes:  #6536
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-15 20:27:24 +02:00
Rusty Russell
4431e4199f Makefile: regnerate config.vars if configure changes (recently PYTHON var).
We didn't want a rule to generate config.vars, since user should run configure manually
with their desired options.  However, we do want to be able to *refresh* it, because
otherwise, if we need a new configuration var for our Makefile, it won't get refreshed:

```
$ make
CC: cc -DBINTOPKGLIBEXECDIR="../libexec/c-lightning" -Wall -Wundef -Wmissing-prototypes -Wmissing-declarations -Wstrict-prototypes -Wold-style-definition -Werror -Wshadow=local -std=gnu11 -g -fstack-protector-strong  -I ccan -I external/libwally-core/include/ -I external/libwally-core/src/secp256k1/include/ -I external/jsmn/ -I external/libbacktrace/ -I external/gheap/ -I external/build-x86_64-linux-gnu/libbacktrace-build -I external/libsodium/src/libsodium/include -I external/libsodium/src/libsodium/include/sodium -I external/build-x86_64-linux-gnu/libsodium-build/src/libsodium/include -I . -I/usr/local/include  -I/usr/include/postgresql   -DCCAN_TAKE_DEBUG=1 -DCCAN_TAL_DEBUG=1 -DCCAN_JSON_OUT_DEBUG=1 -DSHACHAIN_BITS=48 -DJSMN_PARENT_LINKS  -DCOMPAT_V052=1 -DCOMPAT_V060=1 -DCOMPAT_V061=1 -DCOMPAT_V062=1 -DCOMPAT_V070=1 -DCOMPAT_V072=1 -DCOMPAT_V073=1 -DCOMPAT_V080=1 -DCOMPAT_V081=1 -DCOMPAT_V082=1 -DCOMPAT_V090=1 -DCOMPAT_V0100=1 -DCOMPAT_V0121=1  -DBUILD_ELEMENTS=1 -c -o
PYTHONPATH=contrib/msggen  contrib/msggen/msggen/__main__.py
LD: cc     config.vars  -Lexternal/build-x86_64-linux-gnu -lwallycore -lsecp256k1 -ljsmn -lbacktrace -lsodium -L/usr/local/include -lm -lsqlite3  -lz  -L/usr/lib/x86_64-linux-gnu -lpq -o
/bin/sh: 1: contrib/msggen/msggen/__main__.py: Permission denied
make: *** [Makefile:371: cln-grpc/proto/node.proto] Error 126
make: *** Waiting for unfinished jobs....
```

Here, PYTHON is the new var (here, unset).  If we were building a binary, we'd depend on config.h,
and the Makefile says how to refresh that.  But the Makefile includes config.vars (causing an
implicit dependency) so we really should have a rule to make that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-13 14:36:29 +09:30
Rusty Russell
e3b1549b64 configure: don't assume python3, create $PYTHON var.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-28 12:18:06 +02:00
Erik De Smedt
0a4ac04c00 Use python3 instead of python in Makefile
Debian likes this more
2023-08-27 15:16:00 +02:00
ShahanaFarooqui
620a61ee09 makefile: py plugins installed twice with make install
Steps to Reproduce: Run `sudo make install` then again `sudo make install` without uninstalling plugins in between.
First command will install the plugin at `/usr/local/libexec/c-lightning/plugins/clnrest` as expected but the
second command will install it inside the first folder like `/usr/local/libexec/c-lightning/plugins/clnrest/clnrest`.

Fix: Check and delete if the folder already exists in `install-program`.

Changelog-None
2023-08-05 17:12:17 +09:30
Lagrang3
c02f175a75 Import lqueue from CCAN
[ Split into separate commit --RR ]
Signed-off-by: Lagrang3 <eduardo.quintana@pm.me>
2023-07-31 12:58:56 +09:30
Shahana Farooqui
94205c094f plugins/clnrest: Update clnrest dependencies in scripts and Dockerfiles
Adding python3-json5 python3-flask python3-gunicorn flask_restx pyln-client
Also run `pip3 install -r plugins/clnrest/requirements.txt`
2023-07-25 19:20:16 +09:30
Shahana Farooqui
abcfda133f plugin: On-boarding default python plugins
plugin: On-boarding default python plugins
2023-07-25 19:20:16 +09:30
niftynei
f5bac83232 makefile: use recursive delete to target extraneous .baks
Make this a bit less difficult to see what's going on; use common
variables for the paths etc.

Also make the final `rm` step recursive, same as the above find step
which patches the files.

Files are in `contrib/pyln-grpc-proto/pyln/grpc`, but the upper level
find command acts at a level above.
2023-07-25 10:49:34 +09:30
Rusty Russell
e91475c3de Makefile: check-source-no-cppcheck.
Recent cppcheck doesn't like our code; until we fix that, make it easy
to run every other source check.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-07-22 11:08:04 +09:30
Rusty Russell
5132fea44e docs: fix up internal references now files have moved.
And tools/check-markdown.sh didn't do anything: remove it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-07-16 12:59:40 +09:30
niftynei
3a18d8b355 mac-fix: sed doesn't work the same on macos
https://stackoverflow.com/a/14813278

Found-by: @ddustin
2023-07-13 18:59:20 +09:30
Vincenzo Palazzo
da91e70d3e build: Regenerate the generated file as the default target
When modifying the schema, regenerating the file can be challenging.
This change sets the autogenerate file as the default target, m
aking the process more convenient and simplifying our workflow.

Fixes https://github.com/ElementsProject/lightning/issues/6365
Report-by: Dusty Daemon
Changelog-None
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2023-07-11 11:41:15 +09:30
Rusty Russell
0d1c46a5e5 Makefile: rebuild ccan when ccan_compat.h changes.
As it did recently:

```
/usr/bin/ld: bitcoin/pubkey.o: in function `pubkey_to_hash160':
/home/rusty/devel/cvs/lightning/bitcoin/pubkey.c:101: undefined reference to `ccan_ripemd160'
/usr/bin/ld: bitcoin/script.o: in function `hash160':
/home/rusty/devel/cvs/lightning/bitcoin/script.c:22: undefined reference to `ccan_ripemd160'
/usr/bin/ld: bitcoin/script.o: in function `bitcoin_wscript_htlc_offer':
/home/rusty/devel/cvs/lightning/bitcoin/script.c:662: undefined reference to `ccan_ripemd160'
/usr/bin/ld: bitcoin/script.o: in function `bitcoin_wscript_htlc_receive':
/home/rusty/devel/cvs/lightning/bitcoin/script.c:780: undefined reference to `ccan_ripemd160'
collect2: error: ld returned 1 exit status
make: *** [Makefile:660: devtools/create-gossipstore] Error 1
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-07-10 13:37:22 +02:00
Christian Decker
6583f0dead pyln: Create a dedicated pyln-grpc-proto package
Having the grpc bindings in the pyln-testing package was always a bit
strange, however it came with additional issues. Due to the way that
protos are handled by protobuf, any name clash, independently of where
we import the protos, would cause an import error. This usually
happens in diamond-pattern dependencies, and so pull out the generated
files into their own package that everyone else can rely on.

Changelog-None
2023-07-10 13:47:38 +09:30
Rusty Russell
fa596a9446 configure: --enable-debugbuild flag for debug builds.
This controls debug flags for the build, rather than --developer,
which is going away.

I thought about making this flag control the RUST_PROFILE too, but
it seems that we want that set to "release" for CI, whereas for the
C code we want --enable-debugbuild.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-07-09 16:49:48 +09:30
Bitkarrot
910630c497
add m1 native lib paths to Makefile
add m1 native lib paths to Makefile (also need PKG_CONFIG_PATH)

[ Squashed into a single commit --RR ]
2023-06-26 08:25:21 +09:30
Rusty Russell
69fc9c28bc Makefile: correctly erase generated contrib/pyln-testing/pyln/testing/ in distclean, and rebuild by default.
And add contrib/pyln-testing/pyln/testing/grpc2py.py since we didn't previously know
how to build it!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-23 13:58:31 +09:30
Matt Whitlock
8d737cc4bf Makefile: use grouped targets for recipes with multiple fixed outputs
See the section headed "Rules with Grouped Targets" on the Texinfo page
`(make)Multiple Targets`.

Without this fix, Make does not know that these recipes unconditionally
make *all* of their named targets regardless of which target triggers
their execution, and Make will blissfully execute multiple instances of
any such recipe in parallel, not only wasting CPU cycles but potentially
producing incorrect results if the recipe is not atomic in its effects
on the file system.  With this fix, Make understands that it need only
execute such a recipe once to make all of its targets.

In pursuit of the above, move and combine two redundant msggen recipes
into the top-level Makefile, and populate its grouped targets from the
subordinate Makefiles.

Changelog-None
2023-06-23 13:58:31 +09:30
Matt Morehouse
aa9356b961 make: allow Clang coverage instrumentation
Clang's coverage instrumentation [1] is the best I've seen, with
precision down to the expressions within a line of code. Add an option
to use this instrumentation for better coverage reports.

[1] https://clang.llvm.org/docs/SourceBasedCodeCoverage.html
2023-06-15 10:47:56 +09:30
Jon Griffiths
8b4136bad5 build: remove gmp
Core and secp have not used gmp for a very long time now (core disabled
it in 2015).

Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
2023-05-24 18:21:43 +09:30
Rusty Russell
e7d4c3175a build: remove --enable-experimental-features / EXPERIMENTAL_FEATURES
Changelog-EXPERIMENTAL: Build: all experimental features are now runtime-enabled; no more ./configure --enable-experimental-features
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-05-23 09:34:08 +09:30