Of course, we still will, since spendable is for a single HTLC, but
this also shows why we should treat *minimum* as the incorrect answer
if they cross, too.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes: https://github.com/ElementsProject/lightning/issues/7563
Based on great test case from https://github.com/daywalker90
```
E AssertionError: assert {'107x2x0/1': 'Path total 285720859 > spendable 285718000', '108x1x0/1': 'Path total 384721849 > spendable 384718000'} == {}
E Left contains 2 more items:
E {'107x2x0/1': 'Path total 285720859 > spendable 285718000',
E '108x1x0/1': 'Path total 384721849 > spendable 384718000'}
E Full diff:
E {
E - ,
E + '107x2x0/1': 'Path total 285720859 > spendable 285718000',
E + '108x1x0/1': 'Path total 384721849 > spendable 384718000',
E }
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
You need to know it to make an onion, and in theory if we decided to
fuzz it could be different for different paths.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
When a peer connects, we always send all our own gossip (even if they
had set the timestamp filters to filter it out). But we weren't
forcing it out to them when it changed, so this logic only applied to
unstable or frequently-restarting nodes.
So now, we tell all the peers whenever we tell gossipd about our new
gossip.
Fixes: #7276
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: We now send current peers our changed gossip (even if they set timestamp_filter otherwise), not just on reconnect.
We didn't actually check that we *send* the refreshed gossip, just
that we print the message saying we're going to.
So check that everyone received updated gossip when this happens.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Sometimes, for various reasons, a user disables an offer
and then wants to re-enable it. This should be allowed because,
from the CLN point of view, it is just an internal state.
If a user has constraints on the description of the invoice
because they are using services that link some sort of user ID
to an offer, it is important for the user to be able to re-enable the
offer, not create a new one. Creating a new offer would
require a different description.
Link: https://github.com/ElementsProject/lightning/issues/7360
Co-Developed-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
I will commit doc/schemas/lightning-*.json files separately to keep this commit easier to review.
- test-autogenerate-rpc-examples updates all example request & responses in doc/schemas/lightning-*.json files.
- Updated tools/fromschema.py to accommodate the sql JSON example requirement where it does not accept -o in the query but shell does (for queries containing the = sign).
Changelog-None.
It might be nice to let the bookkeeper keep track of external accounts
as well as the internal onchain wallet? To this end, we add some new
custom notifications, which the bookkeeper will ingest and add to its
ledger.
Suggested-By: @chrisguida
Changelog-Added: PLUGINS: `bookkeeper` now listens for two custom events: `utxo_deposit` and `utxo_spend`. This allows for 3rd party plugins to send onchain coin events to the `bookkeeper`. See the new plugins/bkpr/README.md for details on how these work!
Also redirect config creation prompts to stderr in order to not interfere
with json output on stdout.
Changelog-Added: reckless provides json output with option flag -j/--json
This populates information on both topology (i.e. unannounced channels) and capacity for the local node using `listpeerchannels`.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This marks all channels around the source node as free (no delay, no fee). This is normally what we want, if we are calculating a path for ourselves.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This avoids globals (and means memleak traverses the variables!): we
only change over the test plugin though, to avoid unnecessary churn.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This helps code using generate_gossip_store() too, since it can map its identifiers
to the nodeids which were used.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is a hack, but we've had nodes missing gossip. LDK does this,
so until we get a better workaround, use this one.
Changelog-Changed: Protocol: we now always ask the first peer for all its gossip.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: offers: `invoicerequest` will set a blinded path if we're an unannounced node.
Changelog-EXPERIMENTAL: offers: `sendinvoice` will use a blinded path in an invoice_request, if specified.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It's an internal undocumented interface, which makes this change less painful.
We *do* check that the invreq_metadata maps to the given invreq_payer_id, which would
is required for us to sign it.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It's an internal difference, so doesn't actually break compatibility
(it would if we tried to prove we owned an old invoicerequest, but we
don't have infrastructure for that anyway).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
The latest draft allows these experimental ranges, which involves more
changes than I expected.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: offers: handle experimental ranges in offers/invoice_requests/invoices.
Changelog-Added: Protocol: pay can now pay to bolt12 invoices if entry to blinded hop is specified as a short_channel_id (rather than node id).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This also lets us test offers and invoices work if we are the start of
the blinded path.
Changelog-Added: offers: automatically add a blinded path from a peer if we have no public channels, so unannounced nodes can have offers too.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We only need a connection with a peer, not an actual channel. So
add all peers to the local gossmap.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This means only a single gossmap, and they already share the fetchinvoice-noconnect option
and autoconnect code.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Plugins: the `fetchinvoice` plugin has been combined into the `offers` plugin.
Making sure that we are able to fetch the invoice
from an offer without description
E pyln.client.lightning.RpcError: RPC call failed: method:
offer, payload: {'amount': '2msat'}, error: {'code': -32602, 'message':
'bolt12: Offer does not contain a description: invalid token
\'"lno1qgsqvgnwgcg35z6ee2h3yczraddm72xrfua9uve2rlrm9deu7xyfzrcgqyppvggz953rvg9rtxj8lalh43z8epwydjfrmffn3y3p5qz5cywpu09rr4vs"\''}
Link: https://github.com/ElementsProject/lightning/issues/7405
Link: https://github.com/ElementsProject/lightning/issues/7404
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
Changelog-Added: Protocol: onion messages are now supported by default.
Changelog-Deprecated: Config: the --experimental-onion-messages option is ignored (on by default).
However fast we can handle them, it's antisocial to allow others to
make us spam the rest of the network.
Changelog-Protocol: onion messages: we limit incoming to 4 per second, allowing a little burst.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
A bit tricky, since we get more than one message at a time. However,
this just means we go over quota for a bit, and will get caught when
those are sent (we do this for a single message already, so it's not
that much worse).
Note: this not only limits sending, but it limits the actuall query
processing, which is nice.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We currently stream gossip as fast as we can, even if they start at
timestamp 0. Instead, use a simple token bucket filter and only let
them have 1MB per second (500 bytes per second for testing).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Protocol: connectd: we now throttle outgoing gossip at 1MB/second per peer.
Document and enforce the --experimental-anchors deprecation, which was somehow missed in v24.02
Changelog-Deprecated: Config: the --experimental-anchors option is ignored (on by default since v24.02).
We don't use the payment_secret in bolt12, but in onion_decode() we
do put the path_secret (if of correct length) into payment_secret. Not
realizing this confused me, so document that, and make sure we insist
on it being present for bolt12.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Shahana decided this was the optimal UX path, though I insisted that we still
report the actual problem directly when in dev mode, as a compromise.
Suggested-by: https://github.com/Amperstrand
Changelog-Changed: JSON-RPC: Do not return the contents of invalid parameters in error messages, refer to logs (use 'check' to get full error messages)
Fixes: https://github.com/ElementsProject/lightning/issues/7338
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
```
2024-06-24T05:09:32.5233603Z assert l1.rpc.listfunds()["channels"][0]["state"] == "ONCHAIN"
2024-06-24T05:09:32.5234187Z > assert l2.rpc.listfunds()["channels"][0]["state"] == "ONCHAIN"
2024-06-24T05:09:32.5234917Z E AssertionError: assert 'FUNDING_SPEND_SEEN' == 'ONCHAIN'
2024-06-24T05:09:32.5235464Z E - ONCHAIN
2024-06-24T05:09:32.5235773Z E + FUNDING_SPEND_SEEN
2024-06-24T05:09:32.5236096Z
2024-06-24T05:09:32.5236242Z tests/test_misc.py:2907: AssertionError
```
It's possible that l2 hasn't seen the onchain tx yet. What we really want to do is
wait for it, then test that l1 really can spend the output it has recovered.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
```
2024-06-24T05:14:14.9939033Z > l1.rpc.call('dev-feerate', [l2.info['id'], rate - 5])
2024-06-24T05:14:14.9939354Z
2024-06-24T05:14:14.9939466Z tests/test_connection.py:3439:
...
2024-06-24T05:14:14.9967617Z > raise RpcError(method, payload, resp['error'])
2024-06-24T05:14:14.9968833Z E pyln.client.lightning.RpcError: RPC call failed: method: dev-feerate, payload: ['022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59', 2290], error: {'code': -1, 'message': 'Peer bad state'}
```
The disconnect can actually take a while: wait for both sides to
believe they're reconnected. Also wait a little to make sure the
feerate change doesn't create a bad signature.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
lightningd-3 penalizes lightningd-2 but then it can see the preimage for the HTLC which
has already been timed out:
```
2024-06-24T02:41:29.4633900Z lightningd-3 2024-06-24T02:33:54.073Z **BROKEN** 022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59-onchaind-chan#1: HTLC already resolved by THEIR_HTLC_TIMEOUT_TO_THEM when we found preimage
```
This is fair: the test deliberately takes l3 offline for long enough
that the HTLC can get timed out.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
I was looking into using the `threading.Condition` but since we're
already rather heavily using callbacks, this allows us to stay
single-threaded, and not having to completely hook the `setconfig`
function.
Changelog-Added: pyln-client: Added a notification mechanism for config changes
We didn't actually *change* the value you'd see, when we got a setconfig call!
Changelog-Added: pyln-client: implement setconfig hook for plugins so you can see changes in `dynamic` options.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is a difficult transition for us: this string appears in channel
types. We make the transition now in the understanding that it will
be more difficult in future.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: JSON-RPC: `listpeers` `features` array string "option_anchors_zero_fee_htlc_tx": use "option_anchors" (spec renamed it).
Changelog-Added: JSON-RPC: `listpeers` `features` array string uses "option_anchors" for feature 22/23, following renaming in BOLT 9.
Changelog-Changed: JSON-RPC: `listclosedchannels`, `listpeerchannels`, `openchannel_update`, `openchannel_init`, `fundchannel`, `fundchannel_start` and `multifundchannel`: `channel_type` array `names` now contains "anchors" instead of "anchors_zero_fee_htlc_tx".
Changelog-Changed: lightningd: `--list-features-only` now lists "option_anchors" instead of "option_anchors_zero_fee_htlc_tx".
These were removed from the spec.
We still support existing ones, though we were the only implementation
which ever did, and only in experimental mode, so we should be able to
upgrade them and avoid a forced close, with a bit of engineering...
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We still support *existing* channels. Just not new ones (before they could,
in theory, explicitly ask for one).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This means we do have to set the network correctly though,
and also we can get query messages from lightningd which we have to filter.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We documented them as deprecated in v23.08 but unintentionally didn't deprecate them in the code. Thus we are starting their actual deprecation cycle from v24.08 release.
Updating removal version for commands `commando-rune`, `commando-listrunes`, and `commando-blacklist`
Changelog-None.
Somehow, this documentation got lost during the Great Rewrite, so
restore that too.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: JSON-RPC: `offer` removed `@` prefix support from `recurrence_base` (use `recurrence_start_any_period` set to `false`)
We do this by literally creating the modern-style TLV, and pretending we found it in the onion.
This isolates us from messing with any callers, who don't even know.
Co-programmed-with: Alex Myers <alex@endothermic.dev>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes: https://github.com/ElementsProject/lightning/issues/7347
Changelog-Fixed: Protocol: forward legacy non-TLV onions which we removed in 22.11 and spec itself in Feb 2022. Still sent by LND nodes who haven't seen our node_announcement.
This fails, because l2 can't decode the onion:
```
lightningd-2 2024-05-28T21:43:35.137Z DEBUG 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-chan#1: Rejecting their htlc 0 since onion is unprocessable WIRE_INVALID_ONION_HMAC ss=4202c24ea44d9029a2ea3abb24cded51da93164f8bb5cddce9cc824af9945435
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We used to fire up channeld to send this, but:
1. That's silly, we have all the information to make it ourselves.
2. We didn't do it if there was an error on the channel, which as of 24.02
there always is!
3. When it did work, running channeld *stops* onchaind, indefinitely slowing recovery.
Fixes: https://github.com/Blockstream/greenlight/issues/433
Changelog-Fixed: Protocol: we once again send CHANNEL_REESTABLISH responses on closing channels.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
LDK will pick the *upper* limit (see: https://github.com/lightningdevkit/rust-lightning/issues/3014)!
It should not do this, but since you can set a manual range for mutual close, it's probably better to disable this option for close, as it's even more dangerous than documented.
Changelog-Changed: config/JSON: --ignore-fee-limits / setchannel ignorefeelimits no longer applies to mutual close.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes: https://github.com/ElementsProject/lightning/issues/7242
Thought it's intuitive, it's not documented to, and LDK's current behavior makes it quite dangerous to do so.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This means we can see the values in listconfigs, even if we haven't set
them yet.
In particular, we now see the following:
* autoclean-cycle.value_int=3600
* bitcoin-rpcclienttimeout.value_int=60
* bitcoin-retry-timeout.value_int=60
* funder-max-their-funding.value_str=4294967295sat
* funder-per-channel-min.value_str=10000sat
* funder-reserve-tank.value_str=0sat
* funder-fund-probability.value_int=100
Changelog-Changed: plugins: libplugin now shows plugin option default values (where they're non-trivial)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This demonstrates a number of issues reported by Carla: no surprise
since there was no test!
(We create a one-hop blinded path when we only have private channels).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Our fetchinvoice always creates a reply path which terminates at their peer,
so we need a dev overrride for that.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
The feerate security margin is a multiplicative factor applied to the
feerate of some transactions in order to guarantee that the
transaction remains publishable and has a sufficient chance of being
confirmed, that we can base some of our decisions on that.
The multiplicative factor is >=1 and was so far a constant 2. This
might have been sensible in the low-fee environment, where the fees
are expected to oscillate, and almost guaranteeing that we will
eventually have rising feerates but in high-fee environments that is
no longer the case, and the 100% margin that the multiplicator 2
brings is excessive. We therefore opt to start out with 100%, then
linearly interpolate up to a given maxfeerate (which does not have to
be a real feerate ever reached, it just indicates the feerate after
which we apply the constant 10% margin.
Fixes#6974Closes#6976
[Fixed up all the other changes required, including spendable calcualtion
comments and unit test and pytest tests --RR]
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: offers: we now understand blinded paths which use a short-channel-id(+direction) as entry point.
We don't actually support it yet, but this threads through the type change,
puts it in "decode" etc.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
