Commit graph

13473 commits

Author SHA1 Message Date
Alex Myers
d8d2032ba5 gossipd: load pending spam node announcements without forgetting previous node announcements
Fixes: #6531
2023-08-12 09:29:16 +09:30
Alex Myers
53dd6cd44a pytest: test gossip_store loading of spam node announcement before channel update 2023-08-12 09:29:16 +09:30
Rusty Russell
dbe8ac3b22 gossipd: reduce gossip self-advertizement if we have many peers/channels.
dump_our_gossip() is mainly useful for propagating our gossip when we
are poorly connected, not when we have many peers.  @whitslack
reported excessive memory use queueing messages on a large node, so we
limit it beyond the first 5 peers, to 5 channels each.

This assumes we have ~ the same number of peers as channels, which
is probably reasonable.

In the long term, we should move this to connectd, which is properly
equipped to trickle out these messages.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes: #6540
2023-08-12 09:24:06 +09:30
Rusty Russell
66a5765725 common: add tal_arr_randomize() to shuffle an array.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-12 09:24:06 +09:30
Rusty Russell
846cec4f2a gossipd: ignore redundant node_announcement in gossip_store.
Don't know how this is happening, but it is not harmful to ignore it for now.

Fixes: #6531
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-11 12:38:07 +09:30
Rusty Russell
0281111ca1 common: restore OPT_SPLICE definition for the day it's ratified.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-10 15:52:53 +09:30
Dusty Daemon
c50e93d9fb splice: Move splice to experimental feature bit
This was recommended by @t-bast: if the final spec commits to something
compatible, we can simply advertize and accept both features, but if it
does change in incompatible ways we won't cause problems for nodes
who implement the official spec.

(I split this, so first, we remove the OPT_SPLICE entirely, to make
sure we caught them all. --RR)

Suggested-by: @t-bast
Changelog-None
2023-08-10 15:52:53 +09:30
Rusty Russell
5f8b77480c plugins: fix error report from bitcoin-cli exec failure.
We've stomped errno, so if exec fails we don't get a reliable result:

```
2023-08-07T17:58:45.713Z **BROKEN** plugin-bcli: bitcoin-cli exec failed: Bad file descriptor
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-09 19:53:47 +09:30
Lagrang3
961ad21281 renepay: add help for renepay & renepaystatus
Signed-off-by: Lagrang3 <eduardo.quintana@pm.me>
2023-08-09 11:43:58 +09:30
Rusty Russell
dabd6c6b70 pytest: run splicing test on every run.
EXPERIMENTAL_SPLICING=1 turns it on for *all* tests, to make sure we don't
accidentally break those.  But we can (and should!) run the splice test
under every possible CI scenario.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-08 14:31:47 +09:30
Rusty Russell
ac092c0bf9 hsmd: fix capability check for signing splices.
The nomenclature confusion mean that we were ANDING a capability
with a message number (29) which always returned non-zero.  We really
do need a new capability which we can hand to channeld to make these
splice txs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-08 14:31:47 +09:30
Rusty Russell
d95cfc0b64 hsmd: rename "capabilities" flags for hsm fds to "permissions"
I obviously like the word "capabilities" since I reused it to refer
to the HSM's overall features :(

Suggested-by: @ksedgwic
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-08 14:31:47 +09:30
junderw
32b88a2340 Fix: Remove Sync requirements on Futures returned in the Rust plugin library.
See: https://github.com/bitcoindevkit/bdk/issues/1047#issuecomment-1660645669

In general, futures produced by most libraries in the ecosystem of Rust, and bounds placed
on users of famous runtimes like tokio and its spawn method all lack Sync requirements.

Because of this, anyone who creates a callback using any sort of library that returns a
non-Sync future (which most libraries fit this description) inside of it will get some
cryptic error messages (async error messages still leave a lot to be desired).

Removing these Sync requirements will make the library more useful.
2023-08-08 09:45:42 +09:30
Peter Neuroth
29fea55980 doc: Fix typo in the description of fields private
Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
[ Regenerated man pages --RR ]
2023-08-08 09:45:20 +09:30
Rusty Russell
1fbe87f6e4 lightningd: use fsync not fdatasync.
Apparently MacOS doesn't always have fdatasync, so use fsync.  Even more importantly
check whether it succeeds!

Fixes: #6516
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-08 09:45:10 +09:30
Adi Shankara
98c805e267
fix flake8 errors in the script file (#6528)
* fix flake8 errors

* fix E126 error

* fix E123 error
2023-08-07 21:45:44 +05:30
Rusty Russell
54bcb10227 lightningd: fix bolt11 parsing in preapproveinvocie, sendonion, listsendpays and renepay
Since bolt11_decode now insists that any `lightning:` prefix be removed, we need
to make sure to use param_invstring not param_string for all bolt11 parameters:

```
2023-08-07T05:55:32.515Z **BROKEN** lightningd: FATAL SIGNAL 6 (version v23.08rc1-21-g0bf5ee6)
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: common/daemon.c:38 (send_backtrace) 0x55dd94934154
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: common/daemon.c:75 (crashdump) 0x55dd949342e6
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: ./signal/../sysdeps/unix/sysv/linux/x86_64/libc_sigaction.c:0 ((null)) 0x7f5cf5a3bcef
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: ./nptl/pthread_kill.c:44 (__pthread_kill_implementation) 0x7f5cf5a9226b
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: ./nptl/pthread_kill.c:78 (__pthread_kill_internal) 0x7f5cf5a9226b
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: ./nptl/pthread_kill.c:89 (__GI___pthread_kill) 0x7f5cf5a9226b
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: ../sysdeps/posix/raise.c:26 (__GI_raise) 0x7f5cf5a3bc45
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: ./stdlib/abort.c:79 (__GI_abort) 0x7f5cf5a227fb
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: ./assert/assert.c:92 (__assert_fail_base) 0x7f5cf5a2271a
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: ./assert/assert.c:101 (__GI___assert_fail) 0x7f5cf5a33595
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: common/bolt11.c:734 (bolt11_decode_nosig) 0x55dd94929967
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: common/bolt11.c:953 (bolt11_decode) 0x55dd9492a44f
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: lightningd/pay.c:1730 (json_listsendpays) 0x55dd948d7d72
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: lightningd/jsonrpc.c:658 (command_exec) 0x55dd948b525b
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: lightningd/jsonrpc.c:786 (rpc_command_hook_final) 0x55dd948b5876
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: lightningd/plugin_hook.c:285 (plugin_hook_call_) 0x55dd948f6446
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: lightningd/jsonrpc.c:874 (plugin_hook_call_rpc_command) 0x55dd948b5c77
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: lightningd/jsonrpc.c:984 (parse_request) 0x55dd948b6234
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: lightningd/jsonrpc.c:1090 (read_json) 0x55dd948b670f
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:59 (next_plan) 0x55dd94ac9bf4
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:407 (do_plan) 0x55dd94aca823
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:417 (io_ready) 0x55dd94aca865
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: ccan/ccan/io/poll.c:453 (io_loop) 0x55dd94accbff
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: lightningd/io_loop_with_timers.c:22 (io_loop_with_timers) 0x55dd948b33c4
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: lightningd/lightningd.c:1332 (main) 0x55dd948ba429
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: ../sysdeps/nptl/libc_start_call_main.h:58 (__libc_start_call_main) 0x7f5cf5a2350f
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: ../csu/libc-start.c:381 (__libc_start_main_impl) 0x7f5cf5a235c8
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x55dd94881e74
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0xffffffffffffffff
```

Fixes: #6524
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-None: broken in master since last release.
2023-08-07 18:46:48 +09:30
Rusty Russell
7a88900476 pytest: test for listsendpays and lightning: prefix crash.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-07 18:46:48 +09:30
Rusty Russell
91ea85be36 lightningd: close connection when HTLC addition times out.
I noticed this while debugging an issue with ACINQ, that we got upset,
but didn't trigger a reconnect cycle.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Protocol: We now close connection with a peer if adding an HTLC times out (which may be a TCP connectivity issue).
2023-08-07 18:45:29 +09:30
Rusty Russell
d4ed1c7f6f pytest: test that we close connection if adding an HTLC times out.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-07 18:45:29 +09:30
Rusty Russell
f556be5d82 renepay: allow it to die gracefully without crashing lightningd.
Suggested-by: @Lagrang3
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-07 17:13:29 +09:30
Rusty Russell
2042b50978 plugins/bcli: update minimum required bitcoind version.
Less than 22 is obsolete anyway, so we should increment this from 16.0 at least!

Closes: #6234
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-07 16:04:14 +09:30
ShahanaFarooqui
248b34acec docker: bitcoin and elements version update
Updated bitcoin and elements versions to 22.0.
2023-08-07 16:04:14 +09:30
Rusty Russell
91a58a0bdc channeld: don't send splice TLV fields unless negotiated.
This make ACINQ seize up, and not send revoke_and_ack.  Eventually,
this can cause a bad signature error, should payments go in both
directions, which is a separate bug, but this is the trigger.

See: #6500
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-07 15:22:38 +09:30
Christian Decker
0bf5ee6bba meta: Add the VLS team as codeowners of hsmd/hsmd_wire.csv
We use this file as a proxy for breaking changes in the signer
protocol. It may not catch all the breaking changes, but it's a
good first approximation.
2023-08-07 13:10:56 +09:30
Christian Decker
f56ba464b8 meta: Remove ZmnSCPxj from codeowners 2023-08-07 13:10:56 +09:30
Dusty Daemon
aa2df28faf splicing: Update documentation
Added documentation for splice_update & splice_signed and tweaked splice_init.

Added corresponding schemas for splice_* commands

Changelog-None
2023-08-07 12:38:36 +09:30
Rusty Russell
5faaa39773 docs: try to fix up readthedocs.io, so it has the latest man pages, but points to docs.corelighting.org for the rest
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-07 11:56:28 +09:30
Adi Shankara
49acafe653 random doc changes to test.
[Moved into a separate commit --RR]
2023-08-07 11:55:57 +09:30
Adi Shankara
be829a1475 add workflow to sync rpc commands 2023-08-07 11:55:57 +09:30
Adi Shankara
f64aaaf943 add script to sync rpc commands 2023-08-07 11:55:57 +09:30
Rusty Russell
84fe3dc940 build: more 32-bit printf fixes.
Reported-by: Shahana
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-07 11:52:40 +09:30
Rusty Russell
34b6731b82 lightningd: don't return to a default filter level if there are no per-file filters.
In this case, the user's default was info, but they specifically asked for debug
from one plugin.  Since there were no per-file filters, it set filtering to the
default level, info, and rejected it.  Since it's been explicitly filtered in,
we need to pass it at this point.

Reported-by: @wtogami
Fixes: #6503
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-06 14:37:49 +09:30
Rusty Russell
be2cbe784f pytest: test for filtering bug.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-06 14:37:49 +09:30
Rusty Russell
3191ff7e32 doc: make staticbackup output autogenerated in man page.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-06 12:34:36 +09:30
Christian Decker
9074341ef6 msggen: Add staticbackup call to list of managed methods
Changelog-Added: cln-grpc: Added `staticbackup` support to cln-grpc
2023-08-06 12:34:36 +09:30
Christian Decker
3e14056ce0 doc: The SCB fields are hex-encoded binary TLVs, not strings
This makes them smaller when passing them over a binary-capable encoding.
2023-08-06 12:34:36 +09:30
ShahanaFarooqui
33ab1ce78b docker: Install clnrest dependencies from rquirements.txt
Changelog-None
2023-08-06 12:34:19 +09:30
ShahanaFarooqui
18b928046b plugin/clnrest: Downgrading pyln-proto versino from 23.08 to 23.5.2
Bug fix: pip install from plugins/clnrest/requirements.txt was unable to find "not yet released" pyln-proto version 23.08.
2023-08-06 12:34:19 +09:30
Dusty Daemon
fb179f2d5e
splicing: Remove dependency on experimental_dual_fund
Splicing should work automatically on v1 or v2 channels so this requirement isn’t needed.

Changelog-None
[ Squashed fixup into a single commit --RR ]
2023-08-06 12:19:42 +09:30
ShahanaFarooqui
620a61ee09 makefile: py plugins installed twice with make install
Steps to Reproduce: Run `sudo make install` then again `sudo make install` without uninstalling plugins in between.
First command will install the plugin at `/usr/local/libexec/c-lightning/plugins/clnrest` as expected but the
second command will install it inside the first folder like `/usr/local/libexec/c-lightning/plugins/clnrest/clnrest`.

Fix: Check and delete if the folder already exists in `install-program`.

Changelog-None
2023-08-05 17:12:17 +09:30
Dusty Daemon
7c9f5e9847 renepay: Use macros instead of manual format types
Fixes 32-bit builds:

```
In file included from plugins/renepay/pay.c:5:
./plugins/renepay/pay_flow.h: In function 'fmt_payflow_key':
./plugins/renepay/pay_flow.h:54:17: error: format '%ld' expects argument of type 'long int', but argument 4 has type 'u64' {aka 'long long unsigned int'} [-Werror=format=]
   54 |                 "key: groupid=%ld, partid=%ld, payment_hash=%s",
      |                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   55 |                 k->groupid,k->partid,
      |                 ~~~~~~~~~~
      |                  |
      |                  u64 {aka long long unsigned int}
```

etc
2023-08-04 20:33:20 +09:30
Rusty Russell
881d7efd6e wallet: don't BROKEN log id payments.total_msat is null.
This was changed by mistake in 23fafe98e3: if
it's null we turn it into 0 (which is what the default call does, but it
does log BROKEN about it!):

```
2023-08-03T14:10:49.001Z **BROKEN** lightningd: Accessing a null column total_msat/15 in query SELECT  id, status, destination, msatoshi, payment_hash, timestamp, payment_preimage, path_secrets, route_nodes, route_channels, msatoshi_sent, description, bolt11, paydescription, failonionreply, total_msat, partid, local_invreq_id, groupid, completed_at FROM payments ORDER BY id;
```

Fixes: #6501
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-04 20:31:14 +09:30
Rusty Russell
86d63a16ca doc: give an example for multifundchannel
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Closes: #4642
2023-08-04 13:01:49 +09:30
Rusty Russell
568f277835 channeld: replace struct splice with struct splicing for OpenBSD.
Since it's only for transitory splicing info, the new name makes sense.

```
cc channeld/channeld.c
In file included from channeld/channeld.c:23:
./channeld/splice.h:37:8: error: redefinition of 'splice'
struct splice {
       ^
/usr/include/sys/socket.h:140:8: note: previous definition is here
struct  splice {
        ^
```

Reported-by: @grubles
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes: #6486
2023-08-03 13:43:35 +09:30
Rusty Russell
a2830e22d2 CHANGELOG: Update.
Co-authored-by: Alex Myers <95372134+endothermicdev@users.noreply.github.com>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-03 09:05:59 +09:30
Rusty Russell
0c20cc70ab pyln-client, pyln-proto, pyln-testing: update to new versions.
I actually did this manually to send 23.08 to Pypi a few days ago, as it's
required for clnrest.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-03 09:05:59 +09:30
Rusty Russell
0f6687ec7b codex32: use "cl" instead of "ms" as our HRP.
This was strongly recommended by Russell O'Connor: the "ms" implies that
it's a BIP-32 master secret, and this is CLN specific.

If we changed the hrp to "cln" it would be better, but apparently that
means we no longer fit in a "standard billfold metal wallet" (and
our code assumes a 2-byte prefix anyway).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-02 16:12:32 +09:30
Rusty Russell
67e43ea868 renepay: clean up ARC_ADDITIONAL_BITS.
It's expressed in bits, but really it's clearer as a quantity, given
how it's used.

Suggested-by: @Lagrang3
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-02 11:40:52 +09:30
Rusty Russell
15c8a6f6fe renepay: switch from arc_t to struct arc.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-02 11:40:52 +09:30