Commit graph

1534 commits

Author SHA1 Message Date
Rusty Russell
88024fa8d6 common/bolt12, offers plugin: handle experimental ranges in bolt12 correctly.
The latest draft allows these experimental ranges, which involves more
changes than I expected.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: offers: handle experimental ranges in offers/invoice_requests/invoices.
2024-07-23 09:54:47 +09:30
Rusty Russell
ea4025b1ed plugins/offers: fix JSON output of decode when there are unknown fields.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-23 09:54:47 +09:30
Rusty Russell
7735eb6767 doc: allow "offer_description" to be missing in decode.
This is now allowed to be omitted.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-23 09:54:47 +09:30
Rusty Russell
af90fdc0bb common/utils: macros to help get copy/compare across different types right.
Things are often equivalent but different types:
1. u8 arrays in libwally.
2. sha256
3. Secrets derived via sha256
4. txids

Rather than open-coding a BUILD_ASSERT & memcpy, create a macro to do it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-23 09:54:47 +09:30
Joseph Goulden
89ede8a208 plugins/sql: allow datetime functions
Changelog-Changed: Plugins:  now allows date and time sqlite functions.
2024-07-19 15:40:06 +09:30
Rusty Russell
91cd68920c pay: handle case where we are head of blinded path, and next hop is scid.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-18 10:53:55 +09:30
Rusty Russell
f00f832b96 plugins/pay: pay to invoices where first hop is a short_channel_id_dir.
Changelog-Added: Protocol: pay can now pay to bolt12 invoices if entry to blinded hop is specified as a short_channel_id (rather than node id).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-18 10:53:55 +09:30
Rusty Russell
b29b96aae8 common: hoist scidd->pubkey conversion function into gossmap.
We will want to use it in the pay plugin too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-18 10:53:55 +09:30
Rusty Russell
272d313e05 offers: add self-fetchinvoices for offers, and self-pay for the resulting invoice.
Changelog-Added: offers: we can now self-fetch and self-pay BOLT12 offers and invoices.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-18 10:53:55 +09:30
Rusty Russell
9faffeb2e6 plugins/offers: create blinded path if necessary.
This also lets us test offers and invoices work if we are the start of
the blinded path.

Changelog-Added: offers: automatically add a blinded path from a peer if we have no public channels, so unannounced nodes can have offers too.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-18 10:53:55 +09:30
Rusty Russell
55982fe549 offers: generalize "best peer" logic into core.
We use this for invoices published by unannounced nodes: want
something very similar for offers, so generalize and expose it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-18 10:53:55 +09:30
Rusty Russell
ca1b469c1a plugin/pay: Handle paying to blinded path starting with us.
Ideally, we would be able to submit a payment exactly as an incoming
HTLC would, but our forwarding and decoding logic is currently very
much tied to the HTLC.  It would be wonderful to detach that and
have an "injectonion" interface which was unwrapped like any other
(see "injectonionmessage") which would handle self-pay without any
special paths.

Since I'm not prepared to rewrite that all now, instead we use an
interface to decrypt the first hop if it's us, and use the remainder
of the blinded path.

Changelog-Fixed: plugins: pay can now pay a bolt12 invoice even if we, ourselves, are the head of the blinded path within it.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-18 10:53:55 +09:30
Rusty Russell
e8b959ac77 plugins/offers: allow reversal of a single-element path for reply.
Don't refuse, this will happen if we are the head of the blinded path,
for example.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-18 10:53:55 +09:30
Rusty Russell
47c1ca8d85 plugins/pay: separate route destination and pay destination.
For bolt12, we have blinded paths so we route to the head of the blinded
path, which may not be the same as the final payment destination.

This matters mainly for detecting self-pay.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-18 10:53:55 +09:30
Rusty Russell
dacf81672e plugins/offers: improve reply path logic.
We generate a reply path by simply reversing the outgoing path:

	A->B->C gives reply path B->A
	A->B gives reply path A

But if we are not a public node, we can't use ourselves as the first
entry of the reply path: this happens if we directly connect to the
head of a blinded path (as we now support).

In this case, give the entire path as a blinded path.  We could do
this all the time, but there are some cases where nodes don't like
sending replies where the node itself is the head of the blinded
path (like CLN v24.05 or before!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-17 10:57:11 +02:00
Rusty Russell
ea5ea14df4 establish_onion_path: allow establishment via any connected peer.
We only need a connection with a peer, not an actual channel.  So
add all peers to the local gossmap.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-17 10:57:11 +02:00
Rusty Russell
0d4b9ad6ce offers: check they use the blinded path if one is specified.
Now we can specify a blinded path in an offer, we have to check they
used it (or didn't use it, if we didn't have one!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-17 10:57:11 +02:00
Rusty Russell
27578d5c1d plugins/offers: add dev-paths option for specifying offers blinded paths.
This will let us test, at least, as we implement fetching invoices from
them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-17 10:57:11 +02:00
Rusty Russell
92eb84d45f fetchinvoice: don't call establish_onion_path ourselves, now send_message does it.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-17 10:57:11 +02:00
Rusty Russell
1a4256842c fetchinvoice: handle blinded paths in offers.
We iterate through the blinded paths until we can use one, and because we use
the modern code, we properly join paths if we need to route more than one hop
to reach the start of the blinded path.

Changelog-EXPERIMENTAL: fixed: fetchinvoice tries all blinded paths until one is usable, and handles case where we have to route more than one hop to reach the entry point.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-17 10:57:11 +02:00
Rusty Russell
ef641e81c6 fetchinvoice: don't stash path in sent object, hand directly.
Simplification.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-17 10:57:11 +02:00
Rusty Russell
a6cfdea71b fetchinvoice: use common/onion_message routines.
This is significantly simpler, too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-17 10:57:11 +02:00
Rusty Russell
386f7ab64c plugins/offers: don't establish_onion_path at start of processing invoice_request
send_onion_reply() does that for us now, so we don't need to do it up-front.

Simplifies the code quite a bit.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-17 10:57:11 +02:00
Rusty Russell
a43705194d plugins/offers: make send_onion_reply() handle making transient connections.
And make it use inject_onionmessage.  This means we have to be more careful
in createinvoice_done where we had a payload field allocated off tmpctx.

The new code correctly handles the case where we find a path (not just
a peer!) to the start of a blinded path, and need to join the paths.

It worked before if we had to connect directly, just not in the case
where we actually found a usable route of more than 1 hop.

Changelog-EXPERIMENTAL: fixed: onionmessage replies now work even if we need to route to the start of the blinded reply path.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-17 10:57:11 +02:00
Rusty Russell
fa9574da6e plugins/offers: use common/onion_message routines in invoice_request handling.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-17 10:57:11 +02:00
Rusty Russell
8c81d6a0b9 plugin/offers: add inject_onionmessage helper.
This will obsolete the existing calls to RPC "sendonionmessage", but
we transition by introducing it separately.  It's designed to work with
the common/onion_message routines and "injectonionmessage".

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-17 10:57:11 +02:00
Rusty Russell
d664d52342 plugins/offers: neaten fetchinvoice integration.
We already parse some fields, so hand them directly rather than
having fetchinvoice behave as if it's a raw hook.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-17 10:57:11 +02:00
Rusty Russell
a782ea75b5 plugins: move fetchinvoice functionality into offers plugin.
This means only a single gossmap, and they already share the fetchinvoice-noconnect option
and autoconnect code.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Plugins: the `fetchinvoice` plugin has been combined into the `offers` plugin.
2024-07-17 10:57:11 +02:00
Vincenzo Palazzo
05f5976a9c bolt12: partial support for minimal offer
Making sure that we are able to fetch the invoice
from an offer without description

E           pyln.client.lightning.RpcError: RPC call failed: method:
offer, payload: {'amount': '2msat'}, error: {'code': -32602, 'message':
'bolt12: Offer does not contain a description: invalid token
\'"lno1qgsqvgnwgcg35z6ee2h3yczraddm72xrfua9uve2rlrm9deu7xyfzrcgqyppvggz953rvg9rtxj8lalh43z8epwydjfrmffn3y3p5qz5cywpu09rr4vs"\''}

Link: https://github.com/ElementsProject/lightning/issues/7405
Link: https://github.com/ElementsProject/lightning/issues/7404
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2024-07-11 14:51:56 +09:30
Rusty Russell
0c7ce27a39 plugins/offers: establish_onion_path can have a simple boolean arg for connect_disable.
It's always the same string, so simplify the interface.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-09 15:09:29 +02:00
Rusty Russell
34052f9960 bitcoin: hoist scid hash into common code.
A second user is coming.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-09 15:09:29 +02:00
Rusty Russell
f64d48e716 plugins/pay: fix crash if we try to self-pay a bolt12 invoice.
It doesn't work but at least now it doesn't crash!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-09 15:09:29 +02:00
Rusty Russell
ec2b626630 plugins/pay: don't crash if getroute is empty.
This can't happen because we go the self-pay path in this case, but
once we fix that for bolt12, this can be reached.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-09 15:09:29 +02:00
ShahanaFarooqui
4b69c57d75 plugins: Remove requirements.txt for python plugins
Removing requirements.txt from clnrest and wss-proxy plugins due to their ever changing dependencies. We will only be dependent on pyproject.toml for more stable results.

Reference: https://github.com/ElementsProject/lightning/pull/7416#issuecomment-2181492069

Changelog-None.
2024-07-03 12:20:05 -07:00
niftynei
c6ac45d743 fundchannel: remove crash on race condition
If the peer sends you their signatures before we return from the *first*
openchannel_update then the state would still be in
MULTIFUNDCHANNEL_STARTED (not UPDATED) and we'd incorrectly switch the
state to MULTIFUNDCHANNEL_SIGNED, which would plunge us headfirst into
`check_sigs_ready`.

However the `mfc->txid` hadn't been set yet, so we'd crash when trying
to confirm that we've got sigs for the correct txid. Oops.

To fix this, we simply invert the check such that the *only* state that
will move into SIGNED is SECURED
2024-07-03 09:11:52 +09:30
ShahanaFarooqui
39626b9f8f clnrest: Fixes ssl_version deprecated warning
Reference Issue #6931: Fixes `Warning: option `ssl_version` is deprecated and it is ignored. Use ssl_context instead.`

Changelog-None.
2024-06-28 10:54:44 +09:30
ShahanaFarooqui
9f599365f9 clnrest: Updated requirements.txt (without coincurve)
As suggested by @cdecker, I compared `poetry show --tree` results from before/after `cd plugins/clnrest/ && poetry export --output requirements.txt && pip install -r requirements.txt` to check for any significant difference. There is no difference in the tree.

But poetry export removed coincurve & other dependencies from clnrest's requirements.txt.

Changelog-None.
2024-06-28 10:46:14 +09:30
Rusty Russell
9fd29d35dd common: move now-enlarged command_fail_badparam into its own source file.
It's getting a bit awkward to inline now: it's non-trivial.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-25 08:47:50 +09:30
Rusty Russell
e0e38c2cd5 common: new function command_log to log something about a specific command.
Needs implementations for lightningd and libplugin, since they both use this
infrastructure.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-25 08:47:50 +09:30
Rusty Russell
ead211e5e4 autoclean: call list in easy stages.
listforwards on a large node can easily run out of memory.  Sip, don't
gulp!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-20 16:14:38 +09:30
Rusty Russell
c62e1f432f autoclean: use filter to reduce processing.
We can filter down to only the list* fields we need.  In the case of a
node with 1M forwards, this reduces listforwards time from 5 seconds
to 4 seconds.  It will also reduce memory consumption.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-20 16:14:38 +09:30
Rusty Russell
c7ce9c55df libplugin: add plugin_option_dev_dynamic (our first dynamic dev option coming).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-20 16:14:38 +09:30
Rusty Russell
1f9ddd24b2 libplugin: support filtering of outgoing commands.
This is crude, handing a raw JSON string, but it works.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-20 16:14:38 +09:30
Rusty Russell
d2465d4498 autoclean: rework logic to clarify subsystems.
There are really three subsystems: invoices, forwards and sendpays,
each of which has two variants we care about (successes and failures).
If we split the code that way, we can extract the core differences in
each of these cases and share most of the logic.

It's a bit awkward to iterate over each "subsystem" in the JSON
parameter sense, so we have some iteration code to do that where we
need to.

The result is going to be much easier to paginate!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-20 16:14:38 +09:30
Rusty Russell
83eed5ce8e autoclean: use a pointer for our global clean_info.
We're about to make this a more complex struct, so introducing a
new_clean_info() function unifies the code paths.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-20 16:14:38 +09:30
Rusty Russell
84b6601bb3 autoclean: remove autocleaninvoice command.
Changelog-Removed: JSON-RPC: `autocleaninvoice` command (deprecated v22.11, EOL v24.02)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-20 16:14:38 +09:30
ShahanaFarooqui
35101c072a commando: update removal version for rpcs
We documented them as deprecated in v23.08 but unintentionally didn't deprecate them in the code. Thus we are starting their actual deprecation cycle from v24.08 release.

Updating removal version for commands `commando-rune`, `commando-listrunes`, and `commando-blacklist`

Changelog-None.
2024-06-18 14:06:06 +09:30
Rusty Russell
d1d9d7ee6f offers: remove deprecated @ prefix.
Somehow, this documentation got lost during the Great Rewrite, so
restore that too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: JSON-RPC: `offer` removed `@` prefix support from `recurrence_base` (use `recurrence_start_any_period` set to `false`)
2024-06-14 11:30:26 +09:30
Rusty Russell
fc2f1f42c6 bcli: remove deprecated estimatefees responses.
Changelog-Removed: Plugins: `estimatefees` returning feerates by name (e.g. "opening"); deprecated in v23.05.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-14 11:30:26 +09:30
Vincenzo Palazzo
f2551091ca offer: make the blockheight nullable
However, it is possible to avoid this kind of crash, because
our assert is "wrong" because blockheight can be 0 and the
assert can fail.

In addition, the `rpc_scan` already checks for nullability, so the
assert is redundant

2024-06-07T14:14:29.278Z INFO    lightningd: v24.05-modded
2024-06-07T14:14:37.158Z INFO    lightningd: v24.05-modded
2024-06-07T14:14:29.329Z INFO    lightningd: Creating configuration directory /root/.lightning/signet
2024-06-07T14:14:37.589Z INFO    plugin-bcli: bitcoin-cli initialized and connected to bitcoind.
2024-06-07T14:14:29.715Z INFO    lightningd: Creating database
2024-06-07T14:14:37.591Z UNUSUAL lightningd: Waiting for initial block download (this can take a while!)
2024-06-07T14:14:29.752Z UNUSUAL hsmd: HSM: created new hsm_secret file
2024-06-07T14:14:37.603Z INFO    plugin-reckless-wrapper.py: initializing reckless-wrapper.py.
2024-06-07T14:14:29.798Z INFO    plugin-bcli: bitcoin-cli initialized and connected to bitcoind.
2024-06-07T14:14:29.802Z UNUSUAL lightningd: Waiting for initial block download (this can take a while!)
offers: plugins/offers.c:1235: init: Assertion `blockheight' failed.
2024-06-07T14:14:29.814Z INFO    plugin-reckless-wrapper.py: initializing reckless-wrapper.py.
offers: FATAL SIGNAL 6 (version v24.05-modded)
0x57ab55a56485 send_backtrace
2024-06-07T14:14:29.818Z INFO    plugin-chanbackup: Creating Emergency Recovery
   common/daemon.c:33
offers: plugins/offers.c:1235: init: Assertion `blockheight' failed.
offers: FATAL SIGNAL 6 (version v24.05-modded)
0x57ab55a5650d crashdump
   common/daemon.c:75
0x59f3ed3ba485 send_backtrace
0x7dd20b02213f ???
   common/daemon.c:33
   ???:0
0x59f3ed3ba50d crashdump
0x7dd20ae59ce1 ???
   common/daemon.c:75
   ???:0
0x7dd20ae43536 ???
0x764fa583813f ???
   ???:0
0x7dd20ae4340e ???
   ???:0
0x7dd20ae52661 ???
   ???:0
   ???:0
0x57ab55a3effe init
   plugins/offers.c:1235
0x764fa566fce1 ???
0x57ab55a4763f handle_init
   ???:0
   plugins/libplugin.c:1376
0x57ab55a478ee ld_command_handle
0x764fa5659536 ???
   plugins/libplugin.c:1797
0x57ab55a47ec0 ld_read_json_one
   plugins/libplugin.c:1979
   ???:0
0x57ab55a47f49 ld_read_json
0x764fa565940e ???
   plugins/libplugin.c:1999
   ???:0
0x57ab55b934cc next_plan
0x764fa5668661 ???
   ccan/ccan/io/io.c:60
   ???:0
0x57ab55b93953 do_plan
0x59f3ed3a2ffe init
   ccan/ccan/io/io.c:408
   plugins/offers.c:1235
0x57ab55b939ec io_ready
   ccan/ccan/io/io.c:418
0x59f3ed3ab63f handle_init
0x57ab55b95302 io_loop
   plugins/libplugin.c:1376
   ccan/ccan/io/poll.c:455
0x59f3ed3ab8ee ld_command_handle
0x57ab55a486dc plugin_main
   plugins/libplugin.c:1797
   plugins/libplugin.c:2209
0x59f3ed3abec0 ld_read_json_one
0x57ab55a3f36d main
   plugins/libplugin.c:1979
   plugins/offers.c:1285
0x59f3ed3abf49 ld_read_json
0x7dd20ae44d09 ???
   plugins/libplugin.c:1999
   ???:0
0x59f3ed4f74cc next_plan
0x57ab55a3c799 ???
   ccan/ccan/io/io.c:60
0x59f3ed4f7953 do_plan
   ???:0
   ccan/ccan/io/io.c:408
0xffffffffffffffff ???
0x59f3ed4f79ec io_ready
   ccan/ccan/io/io.c:418
   ???:0
offers: FATAL SIGNAL 11 (version v24.05-modded)
0x59f3ed4f9302 io_loop
   ccan/ccan/io/poll.c:455
0x57ab55a56485 send_backtrace
0x59f3ed3ac6dc plugin_main
   common/daemon.c:33
   plugins/libplugin.c:2209
0x57ab55a5650d crashdump
0x59f3ed3a336d main
   plugins/offers.c:1285
   common/daemon.c:75
0x7dd20b02213f ???
0x764fa565ad09 ???
   ???:0
   ???:0
0x59f3ed3a0799 ???
0x0 ???
   ???:0
   ???:0
2024-06-07T14:14:37.734Z INFO    plugin-offers: Killing plugin: exited before replying to init
0xffffffffffffffff ???
   ???:0
2024-06-07T14:14:37.734Z **BROKEN** plugin-offers: Plugin marked as important, shutting down lightningd!
offers: FATAL SIGNAL 11 (version v24.05-modded)
0x59f3ed3ba485 send_backtrace
   common/daemon.c:33
0x59f3ed3ba50d crashdump
   common/daemon.c:75
0x764fa583813f ???
   ???:0
0x0 ???
   ???:0
2024-06-07T14:14:29.823Z INFO    plugin-bookkeeper: Creating database
2024-06-07T14:14:29.883Z INFO    plugin-offers: Killing plugin: exited before replying to init
2024-06-07T14:14:29.883Z **BROKEN** plugin-offers: Plugin marked as important, shutting down lightningd!

Fixes: 9d75fbe237
Reported-by: @farscapian
Link: https://github.com/ElementsProject/lightning/issues/7378
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2024-06-14 10:39:02 +09:30