Commit Graph

3133 Commits

Author SHA1 Message Date
Alex Myers
a2e458047f reckless: add json output option
Also redirect config creation prompts to stderr in order to not interfere
with json output on stdout.

Changelog-Added: reckless provides json output with option flag -j/--json
2024-08-08 11:33:39 +09:30
Rusty Russell
f8b259d5e9 askrene: add "auto.localchans" layer.
This populates information on both topology (i.e. unannounced channels) and capacity for the local node using `listpeerchannels`.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-07 20:35:30 +09:30
Rusty Russell
61ccf18521 devtools/gossmap-compress: allow setting the nodeid explicitly for generated nodes.
This lets us make gossip which contains "real" nodes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-07 20:35:30 +09:30
Rusty Russell
296b3ce20c plugin/askrene: add "auto.sourcefree" layer.
This marks all channels around the source node as free (no delay, no fee).  This is normally what we want, if we are calculating a path for ourselves.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-07 20:35:30 +09:30
Rusty Russell
de19524b6d pytest: simple getroutes tests.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-07 20:35:30 +09:30
Rusty Russell
45814bf8ac plugins/askrene: attach getroutes call to MCF code.
Now getroutes actually does something!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-07 20:35:30 +09:30
Rusty Russell
22a32e6e09 pytest: test file for askrene.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-07 20:35:30 +09:30
Rusty Russell
11bfbf5deb libplugin: add data pointer for plugin convenience.
This avoids globals (and means memleak traverses the variables!): we
only change over the test plugin though, to avoid unnecessary churn.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-07 20:35:30 +09:30
Rusty Russell
28b93e1b5f pytest: make GenChannel arguments explicit.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-07 11:18:55 +09:30
Rusty Russell
e3f06b2602 devtools/gossmap-compress: print out node ids.
This helps code using generate_gossip_store() too, since it can map its identifiers
to the nodeids which were used.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-07 11:18:55 +09:30
Rusty Russell
53faab0838 pytest: add routine to generate gossmap by topology.
We marshal it into the "compressed" format and get the decompresser to
build the actual gossmap.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-07 11:18:55 +09:30
Rusty Russell
bb400238b1 gossipd: always ask first peer for all the gossip.
This is a hack, but we've had nodes missing gossip.  LDK does this,
so until we get a better workaround, use this one.

Changelog-Changed: Protocol: we now always ask the first peer for all its gossip.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-07 10:15:03 +09:30
Rusty Russell
f2a7b19281 plugins/offers: handle invreq_paths in invoice_requests, set them if necessary.
Changelog-EXPERIMENTAL: offers: `invoicerequest` will set a blinded path if we're an unannounced node.
Changelog-EXPERIMENTAL: offers: `sendinvoice` will use a blinded path in an invoice_request, if specified.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-01 12:31:48 +09:30
Rusty Russell
71270ae795 lightningd: make the caller set invreq_metadata and invreq_payer_id for createinvoicerequest.
It's an internal undocumented interface, which makes this change less painful.

We *do* check that the invreq_metadata maps to the given invreq_payer_id, which would
is required for us to sign it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-01 12:31:48 +09:30
Rusty Russell
74ef03d361 lightningd: use tweak on node_id of different key, for createinvoicerequest.
It's an internal difference, so doesn't actually break compatibility
(it would if we tried to prove we owned an old invoicerequest, but we
don't have infrastructure for that anyway).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-01 12:31:48 +09:30
ShahanaFarooqui
2ff3e55f08 plugin: Removing category, description and long_description from plugin_command struct
Changelog-None.
2024-07-31 14:42:58 +09:30
ShahanaFarooqui
b485a026f7 rpc: Removing description from json_command struct 2024-07-31 14:42:58 +09:30
ShahanaFarooqui
89c182e2be rpc: Removing category and verbose from json_command struct 2024-07-31 14:42:58 +09:30
Christian Decker
d496126af8 tests: Adjust tests for the lack of unilateral attempt now 2024-07-29 17:08:33 +02:00
Rusty Russell
88024fa8d6 common/bolt12, offers plugin: handle experimental ranges in bolt12 correctly.
The latest draft allows these experimental ranges, which involves more
changes than I expected.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: offers: handle experimental ranges in offers/invoice_requests/invoices.
2024-07-23 09:54:47 +09:30
Rusty Russell
91cd68920c pay: handle case where we are head of blinded path, and next hop is scid.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-18 10:53:55 +09:30
Rusty Russell
f00f832b96 plugins/pay: pay to invoices where first hop is a short_channel_id_dir.
Changelog-Added: Protocol: pay can now pay to bolt12 invoices if entry to blinded hop is specified as a short_channel_id (rather than node id).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-18 10:53:55 +09:30
Rusty Russell
272d313e05 offers: add self-fetchinvoices for offers, and self-pay for the resulting invoice.
Changelog-Added: offers: we can now self-fetch and self-pay BOLT12 offers and invoices.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-18 10:53:55 +09:30
Rusty Russell
9faffeb2e6 plugins/offers: create blinded path if necessary.
This also lets us test offers and invoices work if we are the start of
the blinded path.

Changelog-Added: offers: automatically add a blinded path from a peer if we have no public channels, so unannounced nodes can have offers too.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-18 10:53:55 +09:30
Rusty Russell
de0d371d20 lightningd: new internal JSONRPC "decryptencrypteddata"
I'm not sure about interface yet, so don't document.  It's ugly.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-18 10:53:55 +09:30
Rusty Russell
73e5d9a78a pytest: fix flake in test_fetchinvoice
On my local machine it can run into ratelimiting, because I miscounted the fetchinvoice
calls.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-18 10:53:55 +09:30
Rusty Russell
6a98de5830 pytest: add simple test for offers with paths.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-17 10:57:11 +02:00
Rusty Russell
ea5ea14df4 establish_onion_path: allow establishment via any connected peer.
We only need a connection with a peer, not an actual channel.  So
add all peers to the local gossmap.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-17 10:57:11 +02:00
Rusty Russell
27578d5c1d plugins/offers: add dev-paths option for specifying offers blinded paths.
This will let us test, at least, as we implement fetching invoices from
them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-17 10:57:11 +02:00
Rusty Russell
a782ea75b5 plugins: move fetchinvoice functionality into offers plugin.
This means only a single gossmap, and they already share the fetchinvoice-noconnect option
and autoconnect code.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Plugins: the `fetchinvoice` plugin has been combined into the `offers` plugin.
2024-07-17 10:57:11 +02:00
Vincenzo Palazzo
05f5976a9c bolt12: partial support for minimal offer
Making sure that we are able to fetch the invoice
from an offer without description

E           pyln.client.lightning.RpcError: RPC call failed: method:
offer, payload: {'amount': '2msat'}, error: {'code': -32602, 'message':
'bolt12: Offer does not contain a description: invalid token
\'"lno1qgsqvgnwgcg35z6ee2h3yczraddm72xrfua9uve2rlrm9deu7xyfzrcgqyppvggz953rvg9rtxj8lalh43z8epwydjfrmffn3y3p5qz5cywpu09rr4vs"\''}

Link: https://github.com/ElementsProject/lightning/issues/7405
Link: https://github.com/ElementsProject/lightning/issues/7404
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2024-07-11 14:51:56 +09:30
Rusty Russell
029034a71b config: onion messages are now always enabled.
Changelog-Added: Protocol: onion messages are now supported by default.
Changelog-Deprecated: Config: the --experimental-onion-messages option is ignored (on by default).
2024-07-10 13:34:00 +02:00
Rusty Russell
5d42600076 connectd: ratelimit onion messages
However fast we can handle them, it's antisocial to allow others to
make us spam the rest of the network.

Changelog-Protocol: onion messages: we limit incoming to 4 per second, allowing a little burst.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-10 13:34:00 +02:00
Rusty Russell
47584bd504 connectd: tie gossip query responses into ratelimiting code.
A bit tricky, since we get more than one message at a time.  However,
this just means we go over quota for a bit, and will get caught when
those are sent (we do this for a single message already, so it's not
that much worse).

Note: this not only limits sending, but it limits the actuall query
processing, which is nice.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-10 12:21:19 +09:30
Rusty Russell
401533667d connectd: throttle streaming gossip for peers.
We currently stream gossip as fast as we can, even if they start at
timestamp 0.  Instead, use a simple token bucket filter and only let
them have 1MB per second (500 bytes per second for testing).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Protocol: connectd: we now throttle outgoing gossip at 1MB/second per peer.
2024-07-10 12:21:19 +09:30
Rusty Russell
991f6dcb0f config: actually deprecate --experimental-anchors.
Document and enforce the --experimental-anchors deprecation, which was somehow missed in v24.02

Changelog-Deprecated: Config: the --experimental-anchors option is ignored (on by default since v24.02).
2024-07-09 15:09:29 +02:00
Rusty Russell
d7fa88b686 lightningd: clarify bolt12 "payment secret" logic.
We don't use the payment_secret in bolt12, but in onion_decode() we
do put the path_secret (if of correct length) into payment_secret.  Not
realizing this confused me, so document that, and make sure we insist
on it being present for bolt12.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-09 15:09:29 +02:00
Rusty Russell
cb4dd7489c lightningd: check command should return as much detail as possible.
If they're explicitly calling "check", don't obfuscate the result!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-25 08:47:50 +09:30
Rusty Russell
f33c5188ef common: don't mention the contents of invalid parameters when in non-dev mode.
Shahana decided this was the optimal UX path, though I insisted that we still
report the actual problem directly when in dev mode, as a compromise.

Suggested-by: https://github.com/Amperstrand
Changelog-Changed: JSON-RPC: Do not return the contents of invalid parameters in error messages, refer to logs (use 'check' to get full error messages)
Fixes: https://github.com/ElementsProject/lightning/issues/7338
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-25 08:47:50 +09:30
Rusty Russell
8288603bab pytest: fix flake in test_emergencyrecover
```
2024-06-24T05:09:32.5233603Z         assert l1.rpc.listfunds()["channels"][0]["state"] == "ONCHAIN"
2024-06-24T05:09:32.5234187Z >       assert l2.rpc.listfunds()["channels"][0]["state"] == "ONCHAIN"
2024-06-24T05:09:32.5234917Z E       AssertionError: assert 'FUNDING_SPEND_SEEN' == 'ONCHAIN'
2024-06-24T05:09:32.5235464Z E         - ONCHAIN
2024-06-24T05:09:32.5235773Z E         + FUNDING_SPEND_SEEN
2024-06-24T05:09:32.5236096Z 
2024-06-24T05:09:32.5236242Z tests/test_misc.py:2907: AssertionError
```

It's possible that l2 hasn't seen the onchain tx yet.  What we really want to do is
wait for it, then test that l1 really can spend the output it has recovered.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-24 21:12:43 +09:30
Rusty Russell
f642e4e551 pytest: fix test_feerate_stress race
```
2024-06-24T05:14:14.9939033Z >       l1.rpc.call('dev-feerate', [l2.info['id'], rate - 5])
2024-06-24T05:14:14.9939354Z 
2024-06-24T05:14:14.9939466Z tests/test_connection.py:3439: 
...
2024-06-24T05:14:14.9967617Z >           raise RpcError(method, payload, resp['error'])
2024-06-24T05:14:14.9968833Z E           pyln.client.lightning.RpcError: RPC call failed: method: dev-feerate, payload: ['022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59', 2290], error: {'code': -1, 'message': 'Peer bad state'}
```

The disconnect can actually take a while: wait for both sides to
believe they're reconnected.  Also wait a little to make sure the
feerate change doesn't create a bad signature.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-24 21:12:43 +09:30
Rusty Russell
bb64fc8ddc pytest: fix flake in test_feerate_stress.
If randomly l1done or l2done are zero, we will create invalid hex:

```
with pytest.raises(RpcError, match='WIRE_INCORRECT_OR_UNKNOWN_PAYMENT_DETAILS|WIRE_TEMPORARY_CHANNEL_FAILURE'):
>           l2.rpc.waitsendpay("{:064x}".format(l2done - 1), timeout=TIMEOUT)

tests/test_connection.py:3438: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
contrib/pyln-client/pyln/client/lightning.py:1361: in waitsendpay
    return self.call("waitsendpay", payload)
contrib/pyln-testing/pyln/testing/utils.py:715: in call
    schemas[0].validate(testpayload)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = Validator(schema={'pairedWith': [['partid', 'groupid']], 'properties': {'groupid': {'description': ['Grouping key...pa... {'description': ['A timeout in... the payment.'], 'type': 'u32'}}, 'required': ['payment_hash']}, format_checker=None)
args = ({'payment_hash': '-000000000000000000000000000000000000000000000000000000000000001', 'timeout': 180},)
kwargs = {}
error = <ValidationError: "'-000000000000000000000000000000000000000000000000000000000000001' is not of type 'hash'">

    def validate(self, *args, **kwargs):
        for error in self.iter_errors(*args, **kwargs):
>           raise error
E           jsonschema.exceptions.ValidationError: '-000000000000000000000000000000000000000000000000000000000000001' is not of type 'hash'
E           
E           Failed validating 'type' in schema['properties']['payment_hash']:
E               {'description': ['The hash of the *payment_preimage*.'], 'type': 'hash'}
E           
E           On instance['payment_hash']:
E               '-000000000000000000000000000000000000000000000000000000000000001'
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-24 21:12:43 +09:30
Rusty Russell
b748c1845c pytest: fix flake in test_closing.py::test_penalty_htlc_tx_timeout
lightningd-3 penalizes lightningd-2 but then it can see the preimage for the HTLC which
has already been timed out:

```
2024-06-24T02:41:29.4633900Z lightningd-3 2024-06-24T02:33:54.073Z **BROKEN** 022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59-onchaind-chan#1: HTLC already resolved by THEIR_HTLC_TIMEOUT_TO_THEM when we found preimage
```

This is fair: the test deliberately takes l3 offline for long enough
that the HTLC can get timed out.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-24 21:12:43 +09:30
Rusty Russell
e21b70cad0 pytest: fix default/description order in zeroconf test plugin.
Noticed as I was debugging.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-20 17:20:06 +09:30
Rusty Russell
1eb1db0e24 pyln-client: allow dynamic option setter to throw exceptions
And don't set the value unless it passes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-20 17:20:06 +09:30
Christian Decker
2737f2e97e pyln: Add and test a callback based setconfig listener
I was looking into using the `threading.Condition` but since we're
already rather heavily using callbacks, this allows us to stay
single-threaded, and not having to completely hook the `setconfig`
function.

Changelog-Added: pyln-client: Added a notification mechanism for config changes
2024-06-20 17:20:06 +09:30
Rusty Russell
572ccac0e5 pyln-client: allow dynamic=True add_option to actually allow updates.
We didn't actually *change* the value you'd see, when we got a setconfig call!

Changelog-Added: pyln-client: implement setconfig hook for plugins so you can see changes in `dynamic` options.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-20 17:20:06 +09:30
Rusty Russell
ead211e5e4 autoclean: call list in easy stages.
listforwards on a large node can easily run out of memory.  Sip, don't
gulp!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-20 16:14:38 +09:30
Rusty Russell
84b6601bb3 autoclean: remove autocleaninvoice command.
Changelog-Removed: JSON-RPC: `autocleaninvoice` command (deprecated v22.11, EOL v24.02)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-20 16:14:38 +09:30
Niklas Gögge
259015720e fuzz: Add input for from_bech32_charset off-by-one bug 2024-06-20 10:53:50 +09:30