Commit graph

15107 commits

Author SHA1 Message Date
Rusty Russell
be054dcb3f askrene: make the flow.[ch] files compile.
This adapts them to their new locations, and copies a few more routines.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-07 20:35:30 +09:30
Rusty Russell
1db5cf6dea askrene: simply fail if a flow amount exceeds 64 bits.
Rather than handling failure, simply report and exit the plugin.
Simplifies error handling.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-07 20:35:30 +09:30
Rusty Russell
7bf399cac5 askrene: remove code which tries to handle tal failures.
tal does not fail: the default handler (which we use) aborts.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-07 20:35:30 +09:30
Rusty Russell
37793b0ac2 askrene/flow: don't omit initial hop in flow_spend.
That will be done in the caller, not here.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-07 20:35:30 +09:30
Rusty Russell
e4b84f1ffb askrene: copy flow and dijkstra from renepay.
Still don't actually try compiling them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-07 20:35:30 +09:30
Rusty Russell
d109fcb568 askrene: simplify minflow()
We let the caller choose mu, and iterate if necessary: it can also
check its limits for fees, etc.  Rationalize it to 0-100 inclusive for
human consumption.

This means we don't loop internally, and in fact there's only one
failure mode: we cannot find enough capacity.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-07 20:35:30 +09:30
Rusty Russell
5999467dce askrene: copy mcf.[ch] from renepay with minimal modifications.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-07 20:35:30 +09:30
Rusty Russell
22a32e6e09 pytest: test file for askrene.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-07 20:35:30 +09:30
Rusty Russell
990bbdde27 askrene: fast lookup for capacities.
We don't know anything about most channels, so we create an array of
fp16_t containing them.  We zero out ones where we do know something,
and use the previous code as the slow path.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-07 20:35:30 +09:30
Rusty Russell
eb335ad763 askrene: always set a dummy constraint when we add a local channel.
This means we never have to look up a local channel when asked the capacity.

We mark these dummy constraints with an MAX timestamp.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-07 20:35:30 +09:30
Rusty Russell
37bdaff8e2 askrene: flesh out getroutes() a little.
We apply all the gossmods for the layers they specified, and create a
naive routine to give the capacity of a channel given those layers.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-07 20:35:30 +09:30
Rusty Russell
c17c240f79 askrene: reservation implementation.
They tell us what paths they're using, so we can adjust capacity estimates
accordingly.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>


Header from folded patch 'reserve-fixup.patch':

fixup! askrene: reservation implementation.
2024-08-07 20:35:30 +09:30
Rusty Russell
d7f983a5b5 askrene: add layers infrastructure.
These are the repositories of all information.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>


Header from folded patch 'layers-fixup.patch':

fixup! askrene: add layers infrastructure.
2024-08-07 20:35:30 +09:30
Rusty Russell
ad209182ba askrene: skeleton which does JSON API.
All the infrastructure and interfaces, but it doesn't do anything yet.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-07 20:35:30 +09:30
Rusty Russell
9b19eada30 common: promote useful routines from renepay.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-07 20:35:30 +09:30
Rusty Russell
a29d135bee bitcoin: add short_channel_id_dir_eq.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-07 20:35:30 +09:30
Rusty Russell
9935e28358 common: new parameter parsing routines.
param_u16 (for delay).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-07 20:35:30 +09:30
Rusty Russell
11bfbf5deb libplugin: add data pointer for plugin convenience.
This avoids globals (and means memleak traverses the variables!): we
only change over the test plugin though, to avoid unnecessary churn.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-07 20:35:30 +09:30
Rusty Russell
e11bab8bbb gossmap: don't process channel_announcement until amount is present.
This simplifies the callers significantly: all channel_announcements now
have an amount, so gossmap_chan_get_capacity() only fails on a local
modification.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-07 20:35:30 +09:30
Rusty Russell
4d593463fa doc: document askrene.
It's experimental, so API may change, but it definitely does need explanation!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-07 20:35:30 +09:30
Rusty Russell
8dbc1c5f2e offers: fix crash when receiving response to offer without offer_issuer_id
Now we actually check the other fields too, as per BOLT!

Reported-by: https://github.com/hMsats
Fixes: #7513
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-06 19:09:27 -07:00
Rusty Russell
28b93e1b5f pytest: make GenChannel arguments explicit.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-07 11:18:55 +09:30
Rusty Russell
e3f06b2602 devtools/gossmap-compress: print out node ids.
This helps code using generate_gossip_store() too, since it can map its identifiers
to the nodeids which were used.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-07 11:18:55 +09:30
Rusty Russell
53faab0838 pytest: add routine to generate gossmap by topology.
We marshal it into the "compressed" format and get the decompresser to
build the actual gossmap.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-07 11:18:55 +09:30
Rusty Russell
99e8e9246e devtools/gossmap-compress: use transparent zlib compression if available.
Before:
```
-rw-rw-r-- 1 rusty rusty 1643258 Jul 26 09:51 compressed
```

After:
```
-rw-rw-r-- 1 rusty rusty 508332 Jul 26 09:49 compressed
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-07 11:18:55 +09:30
Rusty Russell
c93b4aafb2 devtools/gossmap-compress: decompress code.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-07 11:18:55 +09:30
Rusty Russell
cf936d296e devtools: add gossmap-compress to give minimal representation of gossmap topology.
Simple format, which doesn't include node information, just the channels.

Example:

```
$ ls -l gossip-store-2024-06-26
-rw------- 1 rusty rusty 98815543 Jul 26 09:47 gossip-store-2024-06-26
$ ./devtools/gossmap-compress -v compress gossip-store-2024-06-26 compressed
18693 nodes
61437 channels
46148 disabled channels (32620 no update)
9690 unique capacities
85 unique htlc_min
6867 unique htlc_max
807 unique basefee
2521 unique propfee
94 unique delay
$ ls -l compressed
-rw-rw-r-- 1 rusty rusty 1643258 Jul 26 09:51 compressed
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-07 11:18:55 +09:30
daywalker90
0a73f918c1 macos: fix ci and autoclean
Changelog-None
2024-08-07 10:57:26 +09:30
ShahanaFarooqui
a82f1feb3a pyln-client: Deprecate category, description and long description from method
Category, description and long description from `json_command` and `plugin_command` have been removed in favour of getting them from json schema.  Reference PR: Add categories in RPC documentation #7485

Deprecating them in pyln-client as well. They will remove completely in future releases.

Changelog-Deprecated: pyln-client: category, description and long descriptions for RPC commands are deprecated now.
2024-08-07 10:19:55 +09:30
Rusty Russell
bb400238b1 gossipd: always ask first peer for all the gossip.
This is a hack, but we've had nodes missing gossip.  LDK does this,
so until we get a better workaround, use this one.

Changelog-Changed: Protocol: we now always ask the first peer for all its gossip.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-07 10:15:03 +09:30
Rusty Russell
855d7366c7 CI: don't fall over if cannot connect to upload server.
```
>           warnings.warn(f"Error reporting testrun: {e}: {e.read()}")
E           AttributeError: 'URLError' object has no attribute 'read'
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-06 02:00:35 -05:00
Rusty Russell
f77d4d7097 offers: fix flake in fetchinvoice.
In CI, this would sometimes fail: we would timeout waiting for the
fetchinvoice reply.  Never happened locally, so was annoying to debug.

What happened was simple: we called injectonionmessage then when it
returned, put the "sent" object in the linked list so we could recognize
any reply onion messages.

However, we were getting that reply before the plugin processed the response
to injectonionmessage.  This is possible because there are two fds for
plugins: one for it to receive notifications and hooks (like onion messages)
and one for normal RPC usage (like commands to inject onion messages).

The fix is simple: put in the list *before* calling JSON RPC.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-05 16:35:15 +02:00
Rusty Russell
2e06d814e5 lightningd: fix false positive in leak detection.
We ask each channeld to report its leaks, and keep a refcount of how
many are outstanding.  When the channeld replies, or dies, we decrement
the count in a destructor.

But if the last channeld we're waiting for dies, we can call the
destructor in an unexpected place, and thus run leak detection when
we were partway through some operation, which gives a false positive.
Here's a backtrace showing it:

```
0x5f93405897e3 send_backtrace
	common/daemon.c:33
0x5f93405381cf finish_report
	lightningd/memdump.c:139
0x5f93405382a0 leak_detect_req_done
	lightningd/memdump.c:172
0x5f9340705c41 notify
	ccan/ccan/tal/tal.c:243
0x5f9340705ca5 del_tree
	ccan/ccan/tal/tal.c:437
0x5f9340705ce8 del_tree
	ccan/ccan/tal/tal.c:447
0x5f93407061f3 tal_free
	ccan/ccan/tal/tal.c:532
0x5f9340563f5a subd_release_channel
	lightningd/subd.c:924
0x5f934050fb91 channel_set_owner
	lightningd/channel.c:31
0x5f9340511b84 channel_err
	lightningd/channel.c:1081
0x5f93405121a3 channel_fail_transient
	lightningd/channel.c:1095
0x5f934054e547 peer_channels_cleanup
	lightningd/peer_control.c:187
0x5f9340550411 peer_connected
	lightningd/peer_control.c:1689
0x5f9340525101 connectd_msg
	lightningd/connect_control.c:629
```

Instead, do the lightningd detection up-front, where it's in a
clean environment.

Reported-by: Shahana Farooqui
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-04 11:47:54 +02:00
Rusty Russell
0b4b4c5e03 lightningd: don't create a second peer on stub recover if one already exists.
This was confusing to debug, but if the peer already exists we must not
create a second one.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-04 10:48:36 +09:30
Rusty Russell
bf549130d6 global: update BOLT12 quotes.
This is a final sweep to match the current BOLT12 text:

	1563d13999d342680140c693de0b9d65aa522372 ("More bolt12 test vectors.")

Only two code changes, to change the order of checks to match the bolt,
and to give a warning on decode if a path is empty.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-01 12:31:48 +09:30
Rusty Russell
f2a7b19281 plugins/offers: handle invreq_paths in invoice_requests, set them if necessary.
Changelog-EXPERIMENTAL: offers: `invoicerequest` will set a blinded path if we're an unannounced node.
Changelog-EXPERIMENTAL: offers: `sendinvoice` will use a blinded path in an invoice_request, if specified.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-01 12:31:48 +09:30
Rusty Russell
71270ae795 lightningd: make the caller set invreq_metadata and invreq_payer_id for createinvoicerequest.
It's an internal undocumented interface, which makes this change less painful.

We *do* check that the invreq_metadata maps to the given invreq_payer_id, which would
is required for us to sign it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-01 12:31:48 +09:30
Rusty Russell
74ef03d361 lightningd: use tweak on node_id of different key, for createinvoicerequest.
It's an internal difference, so doesn't actually break compatibility
(it would if we tried to prove we owned an old invoicerequest, but we
don't have infrastructure for that anyway).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-01 12:31:48 +09:30
Rusty Russell
fa33a2fece lightningd: make plugins set metadata for invoice requests.
They can do it now: before it would have been awkward to look up previous
payments to match it up for recurring offers (which need to use the same
key, hence the same invreq_metadata).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-01 12:31:48 +09:30
Rusty Russell
401d85fce0 lightningd: make invreq metdata deterministic, not random for recurring invoices.
This is going to allow us to move it out from lightingd into plugin,
easily.  It's legal because the combination of offer id and label must
be unique (with recurrence, we use the same metadata anyay, since they
want to correlate with prior payments anyway).

We already broke recurrence in this PR, so we don't need another note to say
we're changing the key derivation.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-01 12:31:48 +09:30
Rusty Russell
98f07e92c2 common: fix up some minor typos in BOLT quotes.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-01 12:31:48 +09:30
Rusty Russell
e8a38f111d hsmd: create an hsm variant to sign a bolt12 invoice using a tweak on our key.
The current interface, if given a tweak, uses a *different secret key*
and tweaks it.  This was an early experiment: we will switch to using
a secret tweak for invoice_requests like we do for invoice path ids.

To make sure there's no funny business, *hsmd* hashes to form the
tweak (i.e. no zero tweaks!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-01 12:31:48 +09:30
Rusty Russell
09401e34b6 common/bolt12_id: generate alias tweak.
For now we only use a fake id for requesting invoices (as a payer_key), but we
will eventually use this generically, and we want plugins to be able to map them
too, so use the same scheme as path_id: a generated secret using the makesecret
API.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-01 12:31:48 +09:30
Rusty Russell
ee47b9370f common/bolt12_id: new common routine for creating path secrets.
invoice_path_id is actually a generic path_id thing, so rename it.

We're going to use the same scheme for path secrets and the tweak to
node_id when we create a fake pubkey for invoice_requests, so a new
header is appropriate.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-01 12:31:48 +09:30
Rusty Russell
363e1ee44c common: update test to encode new test vectors now we handle them correctly.
1. Missing offer_description iff offer_amount also missing.
2. Missing offer_issuer_id iff offer_paths is present.
3. Short channel id on introduction point.
4. Experimental range.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-01 12:31:48 +09:30
Rusty Russell
d740795139 common/bolt12: allow missing offer_issuer_id.
The latest spec allows this to be omitted iff there is a blinded path
and it would be made up anyway.

In that case, the key they will use to sign the invoice will be the final
blinded key in the path we use.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-01 12:31:48 +09:30
Rusty Russell
2ecf5e6bd5 BOLT12: reject zero-length blinded paths.
This is a good idea, but also specifically called out in the latest BOLT spec.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-01 12:31:48 +09:30
Rusty Russell
18fc1b82b0 wire: add new field invreq_paths from spec.
This is the blinded path for offerless invoice_requests.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-01 12:31:48 +09:30
Rusty Russell
0f0a0cf153 doc: mark schema offer_node_id and warning_missing_offer_node_id deprecated.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-01 12:31:48 +09:30
Rusty Russell
925e90ee8b wire: rename "offer_node_id" to "offer_issuer_id" to match latest BOLT.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-01 12:31:48 +09:30