Commit graph

3147 commits

Author SHA1 Message Date
Lagrang3
62a86cc47f renepay: add daywalker90's test case
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2024-08-19 10:11:35 -07:00
Rusty Russell
054eeb8aa2 plugins/sql: fix crash when subobject is missing which has subobject inside it.
There's only one place (added in v24.02) where we have nested subobjects, and
if the outer sub-object is not there it crashes.

```
sql: FATAL SIGNAL 11 (version v24.08rc1-2-ge134f2f-modded)
0x5781d447239b send_backtrace
        common/daemon.c:33
0x5781d4472432 crashdump
        common/daemon.c:75
0x787bcd64531f ???
        ./signal/../sysdeps/unix/sysv/linux/x86_64/libc_sigaction.c:0
0x5781d447782a json_get_membern
        common/json_parse_simple.c:210
0x5781d44778cc json_get_member
        common/json_parse_simple.c:223
0x5781d445d0e2 process_json_obj
        plugins/sql.c:537
0x5781d445d10d process_json_obj
        plugins/sql.c:538
0x5781d445d598 process_json_list
        plugins/sql.c:684
0x5781d445d65a process_json_result
        plugins/sql.c:699
0x5781d445d7cb default_list_done
        plugins/sql.c:722
0x5781d446349d handle_rpc_reply
        plugins/libplugin.c:1016
0x5781d4463640 rpc_read_response_one
        plugins/libplugin.c:1202
0x5781d44636f1 rpc_conn_read_response
        plugins/libplugin.c:1226
0x5781d459e56c next_plan
        ccan/ccan/io/io.c:60
0x5781d459ea3d do_plan
        ccan/ccan/io/io.c:422
0x5781d459eafa io_ready
        ccan/ccan/io/io.c:439
0x5781d45a0469 io_loop
        ccan/ccan/io/poll.c:455
0x5781d4463dcb plugin_main
        plugins/libplugin.c:2230
```

Changelog-Fixed: Plugins: `sql` crash on querying `listpeerchannels` during channel establishment.
Fixes: https://github.com/ElementsProject/lightning/issues/7505
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-16 07:19:52 +09:30
Rusty Russell
a6c2f183d6 bookkeeper: fix up out-of-order migrations in rc1
This was introduced in rc1, resulting in:

```
2024-08-15T01:33:48.343Z **BROKEN** plugin-bookkeeper: query failed: plugins/bkpr/recorder.c:738: SELECT  e.id, e.account_id, a.name, e.origin, e.tag, e.credit, e.debit, e.output_value, e.currency, e.timestamp, e.blockheight, e.utxo_txid, e.outnum, e.spending_txid, e.payment_id, e.ignored, e.stealable, e.ev_desc, e.spliced FROM chain_events e LEFT OUTER JOIN accounts a ON e.account_id = a.id WHERE  e.spending_txid = ? AND e.account_id = ? AND e.utxo_txid = ? AND e.outnum = ?
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-16 07:18:29 +09:30
Rusty Russell
e2ec60a369 pytest: handle more expected broken messages.
Once we fix our broken log detection, we find some we were missing.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-15 16:38:12 +09:30
daywalker90
7ffd0a3936 cln-plugin: add multi options for String and i64
Changelog-Added: cln-plugin: add multi options for String and i64
2024-08-13 12:24:45 +09:30
Rusty Russell
48259afb70 lightningd: always broadcast our own gossip when it changes.
When a peer connects, we always send all our own gossip (even if they
had set the timestamp filters to filter it out).  But we weren't
forcing it out to them when it changed, so this logic only applied to
unstable or frequently-restarting nodes.

So now, we tell all the peers whenever we tell gossipd about our new
gossip.

Fixes: #7276
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: We now send current peers our changed gossip (even if they set timestamp_filter otherwise), not just on reconnect.
2024-08-12 16:30:29 +09:30
Rusty Russell
4886d228e7 pytest: test that we inform all peers about gossip changes.
In particular, those who've filtered it out.

This currently fails, as expected.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-12 16:30:29 +09:30
Rusty Russell
2d129e79ab pytest: enhance test_gossip_pruning
We didn't actually check that we *send* the refreshed gossip, just
that we print the message saying we're going to.

So check that everyone received updated gossip when this happens.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-12 16:30:29 +09:30
Rusty Russell
975dd76086 lightningd: create new return code for enableoffer.
Suggested-by: https://github.com/Lagrang3
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-11 11:27:10 +09:30
Vincenzo Palazzo
1e1edfd073 offer: allow re-enabling a previously disabled offer
Sometimes, for various reasons, a user disables an offer
and then wants to re-enable it. This should be allowed because,
from the CLN point of view, it is just an internal state.

If a user has constraints on the description of the invoice
because they are using services that link some sort of user ID
to an offer, it is important for the user to be able to re-enable the
offer, not create a new one. Creating a new offer would
require a different description.

Link: https://github.com/ElementsProject/lightning/issues/7360
Co-Developed-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2024-08-11 11:27:10 +09:30
ShahanaFarooqui
50faa7c122 script: Auto generate rpc examples for documentation
I will commit doc/schemas/lightning-*.json files separately to keep this commit easier to review.

- test-autogenerate-rpc-examples updates all example request & responses in doc/schemas/lightning-*.json files.

- Updated tools/fromschema.py to accommodate the sql JSON example requirement where it does not accept -o in the query but shell does (for queries containing the = sign).

Changelog-None.
2024-08-09 23:56:45 -07:00
ShahanaFarooqui
9c0f073a44 tests: Removed non-essential hsm copy step 2024-08-09 23:56:45 -07:00
Max Rantil
3e65ef4b12 lightningd: trim whitespaces from end of config parameters
Signed-off-by: Max Rantil <rantil@pm.me>
2024-08-10 15:20:59 +09:30
niftynei
0e99f2e718 bkpr: add two custom notifications that we listen for
It might be nice to let the bookkeeper keep track of external accounts
as well as the internal onchain wallet? To this end, we add some new
custom notifications, which the bookkeeper will ingest and add to its
ledger.

Suggested-By: @chrisguida

Changelog-Added: PLUGINS: `bookkeeper` now listens for two custom events: `utxo_deposit` and `utxo_spend`. This allows for 3rd party plugins to send onchain coin events to the `bookkeeper`.  See the new plugins/bkpr/README.md for details on how these work!
2024-08-09 10:21:44 +09:30
Alex Myers
a2e458047f reckless: add json output option
Also redirect config creation prompts to stderr in order to not interfere
with json output on stdout.

Changelog-Added: reckless provides json output with option flag -j/--json
2024-08-08 11:33:39 +09:30
Rusty Russell
f8b259d5e9 askrene: add "auto.localchans" layer.
This populates information on both topology (i.e. unannounced channels) and capacity for the local node using `listpeerchannels`.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-07 20:35:30 +09:30
Rusty Russell
61ccf18521 devtools/gossmap-compress: allow setting the nodeid explicitly for generated nodes.
This lets us make gossip which contains "real" nodes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-07 20:35:30 +09:30
Rusty Russell
296b3ce20c plugin/askrene: add "auto.sourcefree" layer.
This marks all channels around the source node as free (no delay, no fee).  This is normally what we want, if we are calculating a path for ourselves.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-07 20:35:30 +09:30
Rusty Russell
de19524b6d pytest: simple getroutes tests.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-07 20:35:30 +09:30
Rusty Russell
45814bf8ac plugins/askrene: attach getroutes call to MCF code.
Now getroutes actually does something!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-07 20:35:30 +09:30
Rusty Russell
22a32e6e09 pytest: test file for askrene.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-07 20:35:30 +09:30
Rusty Russell
11bfbf5deb libplugin: add data pointer for plugin convenience.
This avoids globals (and means memleak traverses the variables!): we
only change over the test plugin though, to avoid unnecessary churn.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-07 20:35:30 +09:30
Rusty Russell
28b93e1b5f pytest: make GenChannel arguments explicit.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-07 11:18:55 +09:30
Rusty Russell
e3f06b2602 devtools/gossmap-compress: print out node ids.
This helps code using generate_gossip_store() too, since it can map its identifiers
to the nodeids which were used.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-07 11:18:55 +09:30
Rusty Russell
53faab0838 pytest: add routine to generate gossmap by topology.
We marshal it into the "compressed" format and get the decompresser to
build the actual gossmap.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-07 11:18:55 +09:30
Rusty Russell
bb400238b1 gossipd: always ask first peer for all the gossip.
This is a hack, but we've had nodes missing gossip.  LDK does this,
so until we get a better workaround, use this one.

Changelog-Changed: Protocol: we now always ask the first peer for all its gossip.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-07 10:15:03 +09:30
Rusty Russell
f2a7b19281 plugins/offers: handle invreq_paths in invoice_requests, set them if necessary.
Changelog-EXPERIMENTAL: offers: `invoicerequest` will set a blinded path if we're an unannounced node.
Changelog-EXPERIMENTAL: offers: `sendinvoice` will use a blinded path in an invoice_request, if specified.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-01 12:31:48 +09:30
Rusty Russell
71270ae795 lightningd: make the caller set invreq_metadata and invreq_payer_id for createinvoicerequest.
It's an internal undocumented interface, which makes this change less painful.

We *do* check that the invreq_metadata maps to the given invreq_payer_id, which would
is required for us to sign it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-01 12:31:48 +09:30
Rusty Russell
74ef03d361 lightningd: use tweak on node_id of different key, for createinvoicerequest.
It's an internal difference, so doesn't actually break compatibility
(it would if we tried to prove we owned an old invoicerequest, but we
don't have infrastructure for that anyway).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-01 12:31:48 +09:30
ShahanaFarooqui
2ff3e55f08 plugin: Removing category, description and long_description from plugin_command struct
Changelog-None.
2024-07-31 14:42:58 +09:30
ShahanaFarooqui
b485a026f7 rpc: Removing description from json_command struct 2024-07-31 14:42:58 +09:30
ShahanaFarooqui
89c182e2be rpc: Removing category and verbose from json_command struct 2024-07-31 14:42:58 +09:30
Christian Decker
d496126af8 tests: Adjust tests for the lack of unilateral attempt now 2024-07-29 17:08:33 +02:00
Rusty Russell
88024fa8d6 common/bolt12, offers plugin: handle experimental ranges in bolt12 correctly.
The latest draft allows these experimental ranges, which involves more
changes than I expected.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: offers: handle experimental ranges in offers/invoice_requests/invoices.
2024-07-23 09:54:47 +09:30
Rusty Russell
91cd68920c pay: handle case where we are head of blinded path, and next hop is scid.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-18 10:53:55 +09:30
Rusty Russell
f00f832b96 plugins/pay: pay to invoices where first hop is a short_channel_id_dir.
Changelog-Added: Protocol: pay can now pay to bolt12 invoices if entry to blinded hop is specified as a short_channel_id (rather than node id).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-18 10:53:55 +09:30
Rusty Russell
272d313e05 offers: add self-fetchinvoices for offers, and self-pay for the resulting invoice.
Changelog-Added: offers: we can now self-fetch and self-pay BOLT12 offers and invoices.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-18 10:53:55 +09:30
Rusty Russell
9faffeb2e6 plugins/offers: create blinded path if necessary.
This also lets us test offers and invoices work if we are the start of
the blinded path.

Changelog-Added: offers: automatically add a blinded path from a peer if we have no public channels, so unannounced nodes can have offers too.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-18 10:53:55 +09:30
Rusty Russell
de0d371d20 lightningd: new internal JSONRPC "decryptencrypteddata"
I'm not sure about interface yet, so don't document.  It's ugly.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-18 10:53:55 +09:30
Rusty Russell
73e5d9a78a pytest: fix flake in test_fetchinvoice
On my local machine it can run into ratelimiting, because I miscounted the fetchinvoice
calls.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-18 10:53:55 +09:30
Rusty Russell
6a98de5830 pytest: add simple test for offers with paths.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-17 10:57:11 +02:00
Rusty Russell
ea5ea14df4 establish_onion_path: allow establishment via any connected peer.
We only need a connection with a peer, not an actual channel.  So
add all peers to the local gossmap.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-17 10:57:11 +02:00
Rusty Russell
27578d5c1d plugins/offers: add dev-paths option for specifying offers blinded paths.
This will let us test, at least, as we implement fetching invoices from
them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-17 10:57:11 +02:00
Rusty Russell
a782ea75b5 plugins: move fetchinvoice functionality into offers plugin.
This means only a single gossmap, and they already share the fetchinvoice-noconnect option
and autoconnect code.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Plugins: the `fetchinvoice` plugin has been combined into the `offers` plugin.
2024-07-17 10:57:11 +02:00
Vincenzo Palazzo
05f5976a9c bolt12: partial support for minimal offer
Making sure that we are able to fetch the invoice
from an offer without description

E           pyln.client.lightning.RpcError: RPC call failed: method:
offer, payload: {'amount': '2msat'}, error: {'code': -32602, 'message':
'bolt12: Offer does not contain a description: invalid token
\'"lno1qgsqvgnwgcg35z6ee2h3yczraddm72xrfua9uve2rlrm9deu7xyfzrcgqyppvggz953rvg9rtxj8lalh43z8epwydjfrmffn3y3p5qz5cywpu09rr4vs"\''}

Link: https://github.com/ElementsProject/lightning/issues/7405
Link: https://github.com/ElementsProject/lightning/issues/7404
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2024-07-11 14:51:56 +09:30
Rusty Russell
029034a71b config: onion messages are now always enabled.
Changelog-Added: Protocol: onion messages are now supported by default.
Changelog-Deprecated: Config: the --experimental-onion-messages option is ignored (on by default).
2024-07-10 13:34:00 +02:00
Rusty Russell
5d42600076 connectd: ratelimit onion messages
However fast we can handle them, it's antisocial to allow others to
make us spam the rest of the network.

Changelog-Protocol: onion messages: we limit incoming to 4 per second, allowing a little burst.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-10 13:34:00 +02:00
Rusty Russell
47584bd504 connectd: tie gossip query responses into ratelimiting code.
A bit tricky, since we get more than one message at a time.  However,
this just means we go over quota for a bit, and will get caught when
those are sent (we do this for a single message already, so it's not
that much worse).

Note: this not only limits sending, but it limits the actuall query
processing, which is nice.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-10 12:21:19 +09:30
Rusty Russell
401533667d connectd: throttle streaming gossip for peers.
We currently stream gossip as fast as we can, even if they start at
timestamp 0.  Instead, use a simple token bucket filter and only let
them have 1MB per second (500 bytes per second for testing).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Protocol: connectd: we now throttle outgoing gossip at 1MB/second per peer.
2024-07-10 12:21:19 +09:30
Rusty Russell
991f6dcb0f config: actually deprecate --experimental-anchors.
Document and enforce the --experimental-anchors deprecation, which was somehow missed in v24.02

Changelog-Deprecated: Config: the --experimental-anchors option is ignored (on by default since v24.02).
2024-07-09 15:09:29 +02:00