Commit Graph

13684 Commits

Author SHA1 Message Date
Rusty Russell
135180c1a0 renepay: don't re-parse bolt11 to get routehints.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-12 09:41:24 +09:30
Rusty Russell
2ceac6c11d renepay: simplify JSON handling in notification_sendpay_success.
Use json_scan(), and use the new pay_flow_from_notification() routine.

Also, the tal_dup_or_null can be tal_dup, since &preimage is never NULL.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-12 09:41:24 +09:30
Rusty Russell
72b316924d renepay: simplify JSON handling.
We can use json_scan(), and share a routine to map the notification
to the pay_flow.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-12 09:41:24 +09:30
Rusty Russell
fb50474efd renepay: get max group_id in single iteration.
Simplifies the logic since we bail out if there are two different
group ids in progress.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-12 09:41:24 +09:30
Rusty Russell
3b8217bd72 renepay: merge struct renepay and struct payment into one.
There are a few fields in `struct renepay` which are genuinely
transient, but it makes the code much harder to follow than simply
having a single structure.

More cleanups will follow, but this is the minimal set.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-12 09:41:24 +09:30
Rusty Russell
6d7cd1e729 renepay: make memleak simpler.
Simply tell it to scan the entire object.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-12 09:41:24 +09:30
Rusty Russell
0091dc4e6d renepay: move list_node to first member of struct payment.
Results in payments having a pointer to the start of the object,
which helps our memleak code.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-12 09:41:24 +09:30
Rusty Russell
008bd74c93 renepay: use more formal allocator pattern.
The general pattern for xxx_new is that it should populate all
fields, for encapsulation and so you never can have a half-formed
object.

This means a fair bit of work for now, but it pays off in the next
patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-12 09:41:24 +09:30
Rusty Russell
ca72a230f3 renepay: fix localmods.
You cannot refresh the gossmap with localmods applied, nor apply localmods
when others have applied localmods in the same process.

There are optimizations we could do, but for now always apply/unapply before
querying gossmap.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-12 09:41:24 +09:30
Rusty Russell
0a9b3d6740 renepay: remove attempt limit.
Time is what users care about, so remove this.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-12 09:41:24 +09:30
Rusty Russell
b5273610b2 renepay: remove unused all_flows field.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-12 09:41:24 +09:30
Alex Myers
d8d2032ba5 gossipd: load pending spam node announcements without forgetting previous node announcements
Fixes: #6531
2023-08-12 09:29:16 +09:30
Alex Myers
53dd6cd44a pytest: test gossip_store loading of spam node announcement before channel update 2023-08-12 09:29:16 +09:30
Rusty Russell
dbe8ac3b22 gossipd: reduce gossip self-advertizement if we have many peers/channels.
dump_our_gossip() is mainly useful for propagating our gossip when we
are poorly connected, not when we have many peers.  @whitslack
reported excessive memory use queueing messages on a large node, so we
limit it beyond the first 5 peers, to 5 channels each.

This assumes we have ~ the same number of peers as channels, which
is probably reasonable.

In the long term, we should move this to connectd, which is properly
equipped to trickle out these messages.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes: #6540
2023-08-12 09:24:06 +09:30
Rusty Russell
66a5765725 common: add tal_arr_randomize() to shuffle an array.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-12 09:24:06 +09:30
Rusty Russell
846cec4f2a gossipd: ignore redundant node_announcement in gossip_store.
Don't know how this is happening, but it is not harmful to ignore it for now.

Fixes: #6531
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-11 12:38:07 +09:30
Rusty Russell
0281111ca1 common: restore OPT_SPLICE definition for the day it's ratified.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-10 15:52:53 +09:30
Dusty Daemon
c50e93d9fb splice: Move splice to experimental feature bit
This was recommended by @t-bast: if the final spec commits to something
compatible, we can simply advertize and accept both features, but if it
does change in incompatible ways we won't cause problems for nodes
who implement the official spec.

(I split this, so first, we remove the OPT_SPLICE entirely, to make
sure we caught them all. --RR)

Suggested-by: @t-bast
Changelog-None
2023-08-10 15:52:53 +09:30
Rusty Russell
5f8b77480c plugins: fix error report from bitcoin-cli exec failure.
We've stomped errno, so if exec fails we don't get a reliable result:

```
2023-08-07T17:58:45.713Z **BROKEN** plugin-bcli: bitcoin-cli exec failed: Bad file descriptor
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-09 19:53:47 +09:30
Lagrang3
961ad21281 renepay: add help for renepay & renepaystatus
Signed-off-by: Lagrang3 <eduardo.quintana@pm.me>
2023-08-09 11:43:58 +09:30
Rusty Russell
dabd6c6b70 pytest: run splicing test on every run.
EXPERIMENTAL_SPLICING=1 turns it on for *all* tests, to make sure we don't
accidentally break those.  But we can (and should!) run the splice test
under every possible CI scenario.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-08 14:31:47 +09:30
Rusty Russell
ac092c0bf9 hsmd: fix capability check for signing splices.
The nomenclature confusion mean that we were ANDING a capability
with a message number (29) which always returned non-zero.  We really
do need a new capability which we can hand to channeld to make these
splice txs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-08 14:31:47 +09:30
Rusty Russell
d95cfc0b64 hsmd: rename "capabilities" flags for hsm fds to "permissions"
I obviously like the word "capabilities" since I reused it to refer
to the HSM's overall features :(

Suggested-by: @ksedgwic
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-08 14:31:47 +09:30
junderw
32b88a2340 Fix: Remove Sync requirements on Futures returned in the Rust plugin library.
See: https://github.com/bitcoindevkit/bdk/issues/1047#issuecomment-1660645669

In general, futures produced by most libraries in the ecosystem of Rust, and bounds placed
on users of famous runtimes like tokio and its spawn method all lack Sync requirements.

Because of this, anyone who creates a callback using any sort of library that returns a
non-Sync future (which most libraries fit this description) inside of it will get some
cryptic error messages (async error messages still leave a lot to be desired).

Removing these Sync requirements will make the library more useful.
2023-08-08 09:45:42 +09:30
Peter Neuroth
29fea55980 doc: Fix typo in the description of fields private
Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
[ Regenerated man pages --RR ]
2023-08-08 09:45:20 +09:30
Rusty Russell
1fbe87f6e4 lightningd: use fsync not fdatasync.
Apparently MacOS doesn't always have fdatasync, so use fsync.  Even more importantly
check whether it succeeds!

Fixes: #6516
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-08 09:45:10 +09:30
Adi Shankara
98c805e267
fix flake8 errors in the script file (#6528)
* fix flake8 errors

* fix E126 error

* fix E123 error
2023-08-07 21:45:44 +05:30
Rusty Russell
54bcb10227 lightningd: fix bolt11 parsing in preapproveinvocie, sendonion, listsendpays and renepay
Since bolt11_decode now insists that any `lightning:` prefix be removed, we need
to make sure to use param_invstring not param_string for all bolt11 parameters:

```
2023-08-07T05:55:32.515Z **BROKEN** lightningd: FATAL SIGNAL 6 (version v23.08rc1-21-g0bf5ee6)
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: common/daemon.c:38 (send_backtrace) 0x55dd94934154
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: common/daemon.c:75 (crashdump) 0x55dd949342e6
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: ./signal/../sysdeps/unix/sysv/linux/x86_64/libc_sigaction.c:0 ((null)) 0x7f5cf5a3bcef
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: ./nptl/pthread_kill.c:44 (__pthread_kill_implementation) 0x7f5cf5a9226b
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: ./nptl/pthread_kill.c:78 (__pthread_kill_internal) 0x7f5cf5a9226b
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: ./nptl/pthread_kill.c:89 (__GI___pthread_kill) 0x7f5cf5a9226b
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: ../sysdeps/posix/raise.c:26 (__GI_raise) 0x7f5cf5a3bc45
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: ./stdlib/abort.c:79 (__GI_abort) 0x7f5cf5a227fb
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: ./assert/assert.c:92 (__assert_fail_base) 0x7f5cf5a2271a
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: ./assert/assert.c:101 (__GI___assert_fail) 0x7f5cf5a33595
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: common/bolt11.c:734 (bolt11_decode_nosig) 0x55dd94929967
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: common/bolt11.c:953 (bolt11_decode) 0x55dd9492a44f
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: lightningd/pay.c:1730 (json_listsendpays) 0x55dd948d7d72
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: lightningd/jsonrpc.c:658 (command_exec) 0x55dd948b525b
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: lightningd/jsonrpc.c:786 (rpc_command_hook_final) 0x55dd948b5876
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: lightningd/plugin_hook.c:285 (plugin_hook_call_) 0x55dd948f6446
2023-08-07T05:55:32.515Z **BROKEN** lightningd: backtrace: lightningd/jsonrpc.c:874 (plugin_hook_call_rpc_command) 0x55dd948b5c77
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: lightningd/jsonrpc.c:984 (parse_request) 0x55dd948b6234
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: lightningd/jsonrpc.c:1090 (read_json) 0x55dd948b670f
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:59 (next_plan) 0x55dd94ac9bf4
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:407 (do_plan) 0x55dd94aca823
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:417 (io_ready) 0x55dd94aca865
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: ccan/ccan/io/poll.c:453 (io_loop) 0x55dd94accbff
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: lightningd/io_loop_with_timers.c:22 (io_loop_with_timers) 0x55dd948b33c4
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: lightningd/lightningd.c:1332 (main) 0x55dd948ba429
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: ../sysdeps/nptl/libc_start_call_main.h:58 (__libc_start_call_main) 0x7f5cf5a2350f
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: ../csu/libc-start.c:381 (__libc_start_main_impl) 0x7f5cf5a235c8
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x55dd94881e74
2023-08-07T05:55:32.516Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0xffffffffffffffff
```

Fixes: #6524
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-None: broken in master since last release.
2023-08-07 18:46:48 +09:30
Rusty Russell
7a88900476 pytest: test for listsendpays and lightning: prefix crash.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-07 18:46:48 +09:30
Rusty Russell
91ea85be36 lightningd: close connection when HTLC addition times out.
I noticed this while debugging an issue with ACINQ, that we got upset,
but didn't trigger a reconnect cycle.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Protocol: We now close connection with a peer if adding an HTLC times out (which may be a TCP connectivity issue).
2023-08-07 18:45:29 +09:30
Rusty Russell
d4ed1c7f6f pytest: test that we close connection if adding an HTLC times out.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-07 18:45:29 +09:30
Rusty Russell
f556be5d82 renepay: allow it to die gracefully without crashing lightningd.
Suggested-by: @Lagrang3
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-07 17:13:29 +09:30
Rusty Russell
2042b50978 plugins/bcli: update minimum required bitcoind version.
Less than 22 is obsolete anyway, so we should increment this from 16.0 at least!

Closes: #6234
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-07 16:04:14 +09:30
ShahanaFarooqui
248b34acec docker: bitcoin and elements version update
Updated bitcoin and elements versions to 22.0.
2023-08-07 16:04:14 +09:30
Rusty Russell
91a58a0bdc channeld: don't send splice TLV fields unless negotiated.
This make ACINQ seize up, and not send revoke_and_ack.  Eventually,
this can cause a bad signature error, should payments go in both
directions, which is a separate bug, but this is the trigger.

See: #6500
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-07 15:22:38 +09:30
Christian Decker
0bf5ee6bba meta: Add the VLS team as codeowners of hsmd/hsmd_wire.csv
We use this file as a proxy for breaking changes in the signer
protocol. It may not catch all the breaking changes, but it's a
good first approximation.
2023-08-07 13:10:56 +09:30
Christian Decker
f56ba464b8 meta: Remove ZmnSCPxj from codeowners 2023-08-07 13:10:56 +09:30
Dusty Daemon
aa2df28faf splicing: Update documentation
Added documentation for splice_update & splice_signed and tweaked splice_init.

Added corresponding schemas for splice_* commands

Changelog-None
2023-08-07 12:38:36 +09:30
Rusty Russell
5faaa39773 docs: try to fix up readthedocs.io, so it has the latest man pages, but points to docs.corelighting.org for the rest
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-07 11:56:28 +09:30
Adi Shankara
49acafe653 random doc changes to test.
[Moved into a separate commit --RR]
2023-08-07 11:55:57 +09:30
Adi Shankara
be829a1475 add workflow to sync rpc commands 2023-08-07 11:55:57 +09:30
Adi Shankara
f64aaaf943 add script to sync rpc commands 2023-08-07 11:55:57 +09:30
Rusty Russell
84fe3dc940 build: more 32-bit printf fixes.
Reported-by: Shahana
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-07 11:52:40 +09:30
Rusty Russell
34b6731b82 lightningd: don't return to a default filter level if there are no per-file filters.
In this case, the user's default was info, but they specifically asked for debug
from one plugin.  Since there were no per-file filters, it set filtering to the
default level, info, and rejected it.  Since it's been explicitly filtered in,
we need to pass it at this point.

Reported-by: @wtogami
Fixes: #6503
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-06 14:37:49 +09:30
Rusty Russell
be2cbe784f pytest: test for filtering bug.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-06 14:37:49 +09:30
Rusty Russell
3191ff7e32 doc: make staticbackup output autogenerated in man page.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-06 12:34:36 +09:30
Christian Decker
9074341ef6 msggen: Add staticbackup call to list of managed methods
Changelog-Added: cln-grpc: Added `staticbackup` support to cln-grpc
2023-08-06 12:34:36 +09:30
Christian Decker
3e14056ce0 doc: The SCB fields are hex-encoded binary TLVs, not strings
This makes them smaller when passing them over a binary-capable encoding.
2023-08-06 12:34:36 +09:30
ShahanaFarooqui
33ab1ce78b docker: Install clnrest dependencies from rquirements.txt
Changelog-None
2023-08-06 12:34:19 +09:30
ShahanaFarooqui
18b928046b plugin/clnrest: Downgrading pyln-proto versino from 23.08 to 23.5.2
Bug fix: pip install from plugins/clnrest/requirements.txt was unable to find "not yet released" pyln-proto version 23.08.
2023-08-06 12:34:19 +09:30