Commit graph

3312 commits

Author SHA1 Message Date
Rusty Russell
f00f832b96 plugins/pay: pay to invoices where first hop is a short_channel_id_dir.
Changelog-Added: Protocol: pay can now pay to bolt12 invoices if entry to blinded hop is specified as a short_channel_id (rather than node id).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-18 10:53:55 +09:30
Rusty Russell
272d313e05 offers: add self-fetchinvoices for offers, and self-pay for the resulting invoice.
Changelog-Added: offers: we can now self-fetch and self-pay BOLT12 offers and invoices.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-18 10:53:55 +09:30
Rusty Russell
9faffeb2e6 plugins/offers: create blinded path if necessary.
This also lets us test offers and invoices work if we are the start of
the blinded path.

Changelog-Added: offers: automatically add a blinded path from a peer if we have no public channels, so unannounced nodes can have offers too.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-18 10:53:55 +09:30
Rusty Russell
de0d371d20 lightningd: new internal JSONRPC "decryptencrypteddata"
I'm not sure about interface yet, so don't document.  It's ugly.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-18 10:53:55 +09:30
Rusty Russell
73e5d9a78a pytest: fix flake in test_fetchinvoice
On my local machine it can run into ratelimiting, because I miscounted the fetchinvoice
calls.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-18 10:53:55 +09:30
Rusty Russell
6a98de5830 pytest: add simple test for offers with paths.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-17 10:57:11 +02:00
Rusty Russell
ea5ea14df4 establish_onion_path: allow establishment via any connected peer.
We only need a connection with a peer, not an actual channel.  So
add all peers to the local gossmap.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-17 10:57:11 +02:00
Rusty Russell
27578d5c1d plugins/offers: add dev-paths option for specifying offers blinded paths.
This will let us test, at least, as we implement fetching invoices from
them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-17 10:57:11 +02:00
Rusty Russell
a782ea75b5 plugins: move fetchinvoice functionality into offers plugin.
This means only a single gossmap, and they already share the fetchinvoice-noconnect option
and autoconnect code.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Plugins: the `fetchinvoice` plugin has been combined into the `offers` plugin.
2024-07-17 10:57:11 +02:00
Vincenzo Palazzo
05f5976a9c bolt12: partial support for minimal offer
Making sure that we are able to fetch the invoice
from an offer without description

E           pyln.client.lightning.RpcError: RPC call failed: method:
offer, payload: {'amount': '2msat'}, error: {'code': -32602, 'message':
'bolt12: Offer does not contain a description: invalid token
\'"lno1qgsqvgnwgcg35z6ee2h3yczraddm72xrfua9uve2rlrm9deu7xyfzrcgqyppvggz953rvg9rtxj8lalh43z8epwydjfrmffn3y3p5qz5cywpu09rr4vs"\''}

Link: https://github.com/ElementsProject/lightning/issues/7405
Link: https://github.com/ElementsProject/lightning/issues/7404
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2024-07-11 14:51:56 +09:30
Rusty Russell
029034a71b config: onion messages are now always enabled.
Changelog-Added: Protocol: onion messages are now supported by default.
Changelog-Deprecated: Config: the --experimental-onion-messages option is ignored (on by default).
2024-07-10 13:34:00 +02:00
Rusty Russell
5d42600076 connectd: ratelimit onion messages
However fast we can handle them, it's antisocial to allow others to
make us spam the rest of the network.

Changelog-Protocol: onion messages: we limit incoming to 4 per second, allowing a little burst.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-10 13:34:00 +02:00
Rusty Russell
47584bd504 connectd: tie gossip query responses into ratelimiting code.
A bit tricky, since we get more than one message at a time.  However,
this just means we go over quota for a bit, and will get caught when
those are sent (we do this for a single message already, so it's not
that much worse).

Note: this not only limits sending, but it limits the actuall query
processing, which is nice.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-10 12:21:19 +09:30
Rusty Russell
401533667d connectd: throttle streaming gossip for peers.
We currently stream gossip as fast as we can, even if they start at
timestamp 0.  Instead, use a simple token bucket filter and only let
them have 1MB per second (500 bytes per second for testing).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Protocol: connectd: we now throttle outgoing gossip at 1MB/second per peer.
2024-07-10 12:21:19 +09:30
Rusty Russell
991f6dcb0f config: actually deprecate --experimental-anchors.
Document and enforce the --experimental-anchors deprecation, which was somehow missed in v24.02

Changelog-Deprecated: Config: the --experimental-anchors option is ignored (on by default since v24.02).
2024-07-09 15:09:29 +02:00
Rusty Russell
d7fa88b686 lightningd: clarify bolt12 "payment secret" logic.
We don't use the payment_secret in bolt12, but in onion_decode() we
do put the path_secret (if of correct length) into payment_secret.  Not
realizing this confused me, so document that, and make sure we insist
on it being present for bolt12.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-07-09 15:09:29 +02:00
Rusty Russell
cb4dd7489c lightningd: check command should return as much detail as possible.
If they're explicitly calling "check", don't obfuscate the result!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-25 08:47:50 +09:30
Rusty Russell
f33c5188ef common: don't mention the contents of invalid parameters when in non-dev mode.
Shahana decided this was the optimal UX path, though I insisted that we still
report the actual problem directly when in dev mode, as a compromise.

Suggested-by: https://github.com/Amperstrand
Changelog-Changed: JSON-RPC: Do not return the contents of invalid parameters in error messages, refer to logs (use 'check' to get full error messages)
Fixes: https://github.com/ElementsProject/lightning/issues/7338
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-25 08:47:50 +09:30
Rusty Russell
8288603bab pytest: fix flake in test_emergencyrecover
```
2024-06-24T05:09:32.5233603Z         assert l1.rpc.listfunds()["channels"][0]["state"] == "ONCHAIN"
2024-06-24T05:09:32.5234187Z >       assert l2.rpc.listfunds()["channels"][0]["state"] == "ONCHAIN"
2024-06-24T05:09:32.5234917Z E       AssertionError: assert 'FUNDING_SPEND_SEEN' == 'ONCHAIN'
2024-06-24T05:09:32.5235464Z E         - ONCHAIN
2024-06-24T05:09:32.5235773Z E         + FUNDING_SPEND_SEEN
2024-06-24T05:09:32.5236096Z 
2024-06-24T05:09:32.5236242Z tests/test_misc.py:2907: AssertionError
```

It's possible that l2 hasn't seen the onchain tx yet.  What we really want to do is
wait for it, then test that l1 really can spend the output it has recovered.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-24 21:12:43 +09:30
Rusty Russell
f642e4e551 pytest: fix test_feerate_stress race
```
2024-06-24T05:14:14.9939033Z >       l1.rpc.call('dev-feerate', [l2.info['id'], rate - 5])
2024-06-24T05:14:14.9939354Z 
2024-06-24T05:14:14.9939466Z tests/test_connection.py:3439: 
...
2024-06-24T05:14:14.9967617Z >           raise RpcError(method, payload, resp['error'])
2024-06-24T05:14:14.9968833Z E           pyln.client.lightning.RpcError: RPC call failed: method: dev-feerate, payload: ['022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59', 2290], error: {'code': -1, 'message': 'Peer bad state'}
```

The disconnect can actually take a while: wait for both sides to
believe they're reconnected.  Also wait a little to make sure the
feerate change doesn't create a bad signature.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-24 21:12:43 +09:30
Rusty Russell
bb64fc8ddc pytest: fix flake in test_feerate_stress.
If randomly l1done or l2done are zero, we will create invalid hex:

```
with pytest.raises(RpcError, match='WIRE_INCORRECT_OR_UNKNOWN_PAYMENT_DETAILS|WIRE_TEMPORARY_CHANNEL_FAILURE'):
>           l2.rpc.waitsendpay("{:064x}".format(l2done - 1), timeout=TIMEOUT)

tests/test_connection.py:3438: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
contrib/pyln-client/pyln/client/lightning.py:1361: in waitsendpay
    return self.call("waitsendpay", payload)
contrib/pyln-testing/pyln/testing/utils.py:715: in call
    schemas[0].validate(testpayload)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = Validator(schema={'pairedWith': [['partid', 'groupid']], 'properties': {'groupid': {'description': ['Grouping key...pa... {'description': ['A timeout in... the payment.'], 'type': 'u32'}}, 'required': ['payment_hash']}, format_checker=None)
args = ({'payment_hash': '-000000000000000000000000000000000000000000000000000000000000001', 'timeout': 180},)
kwargs = {}
error = <ValidationError: "'-000000000000000000000000000000000000000000000000000000000000001' is not of type 'hash'">

    def validate(self, *args, **kwargs):
        for error in self.iter_errors(*args, **kwargs):
>           raise error
E           jsonschema.exceptions.ValidationError: '-000000000000000000000000000000000000000000000000000000000000001' is not of type 'hash'
E           
E           Failed validating 'type' in schema['properties']['payment_hash']:
E               {'description': ['The hash of the *payment_preimage*.'], 'type': 'hash'}
E           
E           On instance['payment_hash']:
E               '-000000000000000000000000000000000000000000000000000000000000001'
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-24 21:12:43 +09:30
Rusty Russell
b748c1845c pytest: fix flake in test_closing.py::test_penalty_htlc_tx_timeout
lightningd-3 penalizes lightningd-2 but then it can see the preimage for the HTLC which
has already been timed out:

```
2024-06-24T02:41:29.4633900Z lightningd-3 2024-06-24T02:33:54.073Z **BROKEN** 022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59-onchaind-chan#1: HTLC already resolved by THEIR_HTLC_TIMEOUT_TO_THEM when we found preimage
```

This is fair: the test deliberately takes l3 offline for long enough
that the HTLC can get timed out.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-24 21:12:43 +09:30
Rusty Russell
e21b70cad0 pytest: fix default/description order in zeroconf test plugin.
Noticed as I was debugging.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-20 17:20:06 +09:30
Rusty Russell
1eb1db0e24 pyln-client: allow dynamic option setter to throw exceptions
And don't set the value unless it passes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-20 17:20:06 +09:30
Christian Decker
2737f2e97e pyln: Add and test a callback based setconfig listener
I was looking into using the `threading.Condition` but since we're
already rather heavily using callbacks, this allows us to stay
single-threaded, and not having to completely hook the `setconfig`
function.

Changelog-Added: pyln-client: Added a notification mechanism for config changes
2024-06-20 17:20:06 +09:30
Rusty Russell
572ccac0e5 pyln-client: allow dynamic=True add_option to actually allow updates.
We didn't actually *change* the value you'd see, when we got a setconfig call!

Changelog-Added: pyln-client: implement setconfig hook for plugins so you can see changes in `dynamic` options.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-20 17:20:06 +09:30
Rusty Russell
ead211e5e4 autoclean: call list in easy stages.
listforwards on a large node can easily run out of memory.  Sip, don't
gulp!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-20 16:14:38 +09:30
Rusty Russell
84b6601bb3 autoclean: remove autocleaninvoice command.
Changelog-Removed: JSON-RPC: `autocleaninvoice` command (deprecated v22.11, EOL v24.02)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-20 16:14:38 +09:30
Niklas Gögge
259015720e fuzz: Add input for from_bech32_charset off-by-one bug 2024-06-20 10:53:50 +09:30
Rusty Russell
7f2aedc76f common: BOLT update: option_anchors_zero_fee_htlc_tx is now simply "option_anchors".
This is a difficult transition for us: this string appears in channel
types.  We make the transition now in the understanding that it will
be more difficult in future.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Changelog-Deprecated: JSON-RPC: `listpeers` `features` array string "option_anchors_zero_fee_htlc_tx": use "option_anchors" (spec renamed it).
Changelog-Added: JSON-RPC: `listpeers` `features` array string uses "option_anchors" for feature 22/23, following renaming in BOLT 9.
Changelog-Changed: JSON-RPC: `listclosedchannels`, `listpeerchannels`, `openchannel_update`, `openchannel_init`, `fundchannel`, `fundchannel_start` and `multifundchannel`: `channel_type` array `names` now contains "anchors" instead of "anchors_zero_fee_htlc_tx".
Changelog-Changed: lightningd: `--list-features-only` now lists "option_anchors" instead of "option_anchors_zero_fee_htlc_tx".
2024-06-19 15:54:24 +09:30
Rusty Russell
7d3d763b96 channeld: BOLT update, no longer allow creation of old (experimental-only!) non-zero-fee anchor channels.
These were removed from the spec.

We still support existing ones, though we were the only implementation
which ever did, and only in experimental mode, so we should be able to
upgrade them and avoid a forced close, with a bit of engineering...

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-19 15:54:24 +09:30
Rusty Russell
b6d7ee1f11 common: No longer support new channels without option_static_remotekey.
We still support *existing* channels.  Just not new ones (before they could,
in theory, explicitly ask for one).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-19 15:54:24 +09:30
Rusty Russell
1de569d38d devtools/gossipwith: use timestamp filter message not obsolete INIT_ROUTING_SYNC.
This means we do have to set the network correctly though,
and also we can get query messages from lightningd which we have to filter.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-19 15:54:24 +09:30
ShahanaFarooqui
35101c072a commando: update removal version for rpcs
We documented them as deprecated in v23.08 but unintentionally didn't deprecate them in the code. Thus we are starting their actual deprecation cycle from v24.08 release.

Updating removal version for commands `commando-rune`, `commando-listrunes`, and `commando-blacklist`

Changelog-None.
2024-06-18 14:06:06 +09:30
Rusty Russell
d1d9d7ee6f offers: remove deprecated @ prefix.
Somehow, this documentation got lost during the Great Rewrite, so
restore that too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: JSON-RPC: `offer` removed `@` prefix support from `recurrence_base` (use `recurrence_start_any_period` set to `false`)
2024-06-14 11:30:26 +09:30
Rusty Russell
818cf06139 common: translate legacy onion payloads.
We do this by literally creating the modern-style TLV, and pretending we found it in the onion.

This isolates us from messing with any callers, who don't even know.

Co-programmed-with: Alex Myers <alex@endothermic.dev>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes: https://github.com/ElementsProject/lightning/issues/7347
Changelog-Fixed: Protocol: forward legacy non-TLV onions which we removed in 22.11 and spec itself in Feb 2022.  Still sent by LND nodes who haven't seen our node_announcement.
2024-05-29 12:40:01 -05:00
Rusty Russell
a4732177bb pytest: test for forwarding legacy onion.
This fails, because l2 can't decode the onion:

```
lightningd-2 2024-05-28T21:43:35.137Z DEBUG   0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-chan#1: Rejecting their htlc 0 since onion is unprocessable WIRE_INVALID_ONION_HMAC ss=4202c24ea44d9029a2ea3abb24cded51da93164f8bb5cddce9cc824af9945435
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-05-29 12:40:01 -05:00
Rusty Russell
1d4783a756 lightningd: send CHANNEL_REESTABLISH ourselves on closed channels.
We used to fire up channeld to send this, but:
1. That's silly, we have all the information to make it ourselves.
2. We didn't do it if there was an error on the channel, which as of 24.02
   there always is!
3. When it did work, running channeld *stops* onchaind, indefinitely slowing recovery.

Fixes: https://github.com/Blockstream/greenlight/issues/433
Changelog-Fixed: Protocol: we once again send CHANNEL_REESTABLISH responses on closing channels.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-05-29 12:39:20 -05:00
Rusty Russell
f9e7d5653a pytest: test that we get REESTABLISH msg reply on closed channels.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-05-29 12:39:20 -05:00
Rusty Russell
964c0732cf pytest: remove xfail in test (accidental commit?)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-05-28 09:42:36 -05:00
Rusty Russell
877df5afdd lightningd: don't ignore fee limits on mutual close.
LDK will pick the *upper* limit (see: https://github.com/lightningdevkit/rust-lightning/issues/3014)!

It should not do this, but since you can set a manual range for mutual close, it's probably better to disable this option for close, as it's even more dangerous than documented.

Changelog-Changed: config/JSON: --ignore-fee-limits / setchannel ignorefeelimits no longer applies to mutual close.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes: https://github.com/ElementsProject/lightning/issues/7242
2024-05-22 08:26:51 -05:00
Rusty Russell
9556fa5210 pytest: test that ignore-fee-limits doesn't apply for mutual close.
Thought it's intuitive, it's not documented to, and LDK's current behavior makes it quite dangerous to do so.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-05-22 08:26:51 -05:00
Alex Myers
cbf343269b pytest: use reserve_unused_port even more
Uses the NodeFactory method to call reserve_unused_port
which replaced the ephemeral_port_reserve function. This
frees the reserved ports at teardown.
2024-05-17 13:56:18 -05:00
Christian Decker
36ef3db762 fixup! msggen: add listconfigs method 2024-05-17 16:30:43 +02:00
daywalker90
1dd16f72fb msggen: add listconfigs method
Changelog-None
2024-05-17 16:30:43 +02:00
daywalker90
6acfca7517 msggen: add routes to decode and decodepay
Changelog-None
2024-05-17 16:30:43 +02:00
daywalker90
a9ff3cb039 pytests: use reserve_unused_port() everywhere
Changelog-None
2024-05-16 17:31:02 +02:00
Erik De Smedt
7ba53dc828 Fix test for notification of custommsg
I forgot to add a `break` clause at the end of the test.

Without the break the test will keep waiting forever
on a `custommsg` that will never arrive
2024-05-16 14:00:18 +02:00
Erik De Smedt
c1062b1db7 Test for notifications over grpc 2024-05-16 14:00:18 +02:00
Rusty Russell
757e6f8a9b libplugin: allow display of default values.
This means we can see the values in listconfigs, even if we haven't set
them yet.

In particular, we now see the following:

* autoclean-cycle.value_int=3600
* bitcoin-rpcclienttimeout.value_int=60
* bitcoin-retry-timeout.value_int=60
* funder-max-their-funding.value_str=4294967295sat
* funder-per-channel-min.value_str=10000sat
* funder-reserve-tank.value_str=0sat
* funder-fund-probability.value_int=100

Changelog-Changed: plugins: libplugin now shows plugin option default values (where they're non-trivial)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-05-15 15:50:54 -05:00