Commit Graph

14298 Commits

Author SHA1 Message Date
Rusty Russell
bf9cdc597e pytest: fix flake in test_custommsg_triggers_notification
If listpeers on l2 is called too fast, then it won't return any peers, and
we will fail:

```
        # Connect l1 to l2
        l1.connect(l2)
>       wait_for(lambda: l2.rpc.listpeers(l1.info['id'])['peers'][0]['connected'])

tests/test_misc.py:2690:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
contrib/pyln-testing/pyln/testing/utils.py:88: in wait_for
    while not success():
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

>   wait_for(lambda: l2.rpc.listpeers(l1.info['id'])['peers'][0]['connected'])
E   IndexError: list index out of range
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-29 16:47:33 +10:30
Rusty Russell
6778f320d2 openingd: make channel-type rejection message clearer.
For example, lnprototest got the error 'You gave bad parameters: Did not support channel_type ' which doesn't make it clear that it's rejecting the empty channel type.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-29 13:40:34 +10:30
Rusty Russell
b805899b06 pytest: check that listpeerchannels gives the same channel type as returned from fund/openchannel
Could have done this in an earlier commit, but that would be a messy rebase.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-29 13:40:34 +10:30
Rusty Russell
17d0d006d2 fundchannel_start & multifundchannel: add channel_type.
Let's tell the caller what channel_type they got!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `fundchannel`, `multifundchannel`, `fundchannel_start` and `openchannel_init`: new field `channel_type`.
2024-01-29 13:40:34 +10:30
Rusty Russell
4cf4dcc584 dualopend: fix channel_type if we don't negotiate the default.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-29 13:40:34 +10:30
Rusty Russell
0b38b83aa3 common: move json_add_channel_type from lightningd/
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-29 13:40:34 +10:30
Rusty Russell
08e29ab47f lightningd: make option_static_remotekey compulsory.
As suggested in https://github.com/lightning/bolts/pull/1092.

We still support channels opened without it, but you can no longer open new ones without it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: `option_gossip_queries` is now required (advertized by all but 16 nodes)
2024-01-29 13:40:34 +10:30
Rusty Russell
e531ff83b3 lightningd: add dev-any-channel-type to allow obsolete channel types.
We're about to make static_remotekey compulsory, but we still want to
do tests for pre-existing channels.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-29 13:40:34 +10:30
Rusty Russell
3c2a57598f spenderp: add channel_type parameter to fundchannel / multifundchannel.
And add a request schema for multifundchannel.

Changelog-Added: JSON-RPC: `fundchannel` and `multifundchannel` now take an optional `channel_type` parameter.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-29 13:40:34 +10:30
Rusty Russell
e749aebbff fundchannel_start / openchannel_init: add a channel_type parameter to force channel type.
And add request schemas for openchannel_init and fundchannel_start.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `fundchannel_start` and `openchannel_init` now take an optional `channel_type` parameter.
2024-01-29 13:40:34 +10:30
Rusty Russell
a943a53658 common: channel_type_accept doesn't use their_features; remove parameter.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-29 13:40:34 +10:30
Rusty Russell
654a45dad6 common: add param_channel_type helper.
We use an array of bit numbers.  We could use an array of names, but the JSON typing is then harder.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-29 13:40:34 +10:30
Rusty Russell
031524fab5 openingd: correctly mark channel type zeroconf if negotiated and they offered minimum_depth=0.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: `channel_type` reflects option_zeroconf if explicitly negotiated.
2024-01-29 13:40:34 +10:30
Rusty Russell
f450dfeb55 lightningd: make option_gossip_queries compulsory.
As suggested in https://github.com/lightning/bolts/pull/1092.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: `option_gossip_queries` is now required (advertized by all but 11 nodes)
2024-01-29 13:40:34 +10:30
Rusty Russell
88a2146664 lightningd: make option_data_loss_protect compulsory.
As suggested in https://github.com/lightning/bolts/pull/1092.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: `option_data_loss_protect` is now required (advertized by all but 11 nodes)
2024-01-29 13:40:34 +10:30
Rusty Russell
5a1e511a85 lightningd: fix fd leak with check command on openchannel_init.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-29 13:40:34 +10:30
Christian Decker
45b298fca4 msggen: Make ChannelTypeName a native enum
It fails because this is an array of enum values, which is a
combination we hadn't seen before. Replacing this with a manually
managed enum, since it is likely we'll want to reuse it in the future.
2024-01-29 13:40:34 +10:30
Christian Decker
0fd9680be9 docs: Fix missing escapes for underscores in multifundchannel docs 2024-01-29 13:40:34 +10:30
Rusty Russell
bcc6da2942 lnprototest: update to origin/no-empty-channeltype for now.
Otherwise it fails because lnprototest expects us to allow a non-static-channel
channel type.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-29 13:40:34 +10:30
Devrandom
c5c2e9f123 channeld: use revoke_commitment_tx if hsmd supports 2024-01-29 12:02:37 +10:30
Devrandom
e6c7b58a8a hsmd: separate revoke_commitment_tx
ChangeLog-Added: Added hsmd_revoke_commitment_tx to ensure synchronization of local state with remote signers.
2024-01-29 12:02:37 +10:30
Ken Sedgwick
019169fc9a channeld: defer construction of revocation message until master sync 2024-01-29 12:02:37 +10:30
Ken Sedgwick
d463b8d900 channeld: add hsm_capabilities and add hsm_is_capable to common
Changelog-Added: Added hsm_capabilities and hsm_is_capable to channeld.
2024-01-29 12:02:37 +10:30
Lagrang3
b0054aad46 renepay: accomodate fees in the payment flow
Min. Cost Flow does not take into account fees when computing a flow
    with liquidity constraints.
    This is a work-around solution that reduces the amount on every route to
    respect the liquidity bound. The deficity in the delivered amount is
    solved by running MCF once again.

    Changes:

    1. the function `flow_complete` allocates amounts to send over the set of routes
       computed by the MCF algorithm, but it does not allocate more than liquidity
       bound of the route. For this reason `minflow` returns a set of routes that
       satisfy the liquidity bounds but it is not guaranteed that the total payment
       reaches the destination therefore there could a deficit in the delivery:
       `deficit = amount_to_deliver - delivering`.

    2. in the function `add_payflows` after `minflow` returns a set of routes we
       call `flows_fit_amount` that tries to a allocate the `deficit` in the routes
       that the MCF have computed.

    3. if the resulting flows pass all payment constraints then we update
        `amount_to_deliver = amount_to_deliver - delivering`, and the loop
        repeats as long as `amount_to_deliver` is not zero.

    In other words, the excess amount, beyond the liquidity bound,
    in the routes is removed and then we try to allocate it
    into known routes, otherwise we do a whole MCF again just for the
    remaining amount.

    Fixes issue #6599
2024-01-29 10:48:24 +10:30
niftynei
a4f92eac81 bkpr: now that we're not doing empty acct logging, we dont need this bool
We stopped doing empty journal logs, so we no longer need to switch
our log severity based on whether or not an account exists.

Should make bookkeeper less chatty and remove noisy logs

Changelog-None
2024-01-29 10:05:03 +10:30
niftynei
ec7044e78e bkpr: use currency in balance snapshot 2024-01-29 10:05:03 +10:30
niftynei
5484aaee33 bkpr: dont log unknown accounts with zero balances
We were putting out a lot of empty journal entries. Let's
stop doing that.

Now the wallet balance stays uninitialized until/unless you have
funds in it.

Fixes #5672
2024-01-29 10:05:03 +10:30
Rusty Russell
040af90e7d libplugin: remove global deprecated_apis flag.
And we don't need to handle 0.9 lightningd which didn't include
allow-deprecated-apis in getmanifest call.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-26 10:30:22 +10:30
Rusty Russell
8d68c608de commando: use deprecation API for missing ids.
In this case we don't have a matching "command", so we need a special
API.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-26 10:30:22 +10:30
Rusty Russell
5f899d0dd2 plugins/bcli: use per-command deprecation flags.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-26 10:30:22 +10:30
Rusty Russell
5c484cd9e6 libplugin: handle deprecated_oneshot notification.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-26 10:30:22 +10:30
Rusty Russell
2914d3adfc plugins: allow plugins to get per-connection deprecated state.
Unfortunately, this is awkward: we just copy through most requests,
so we can't easily add a "deprecation" field to each one.  So we do
a notification if the next command has a different deprecation status
than the global one, but that requires opt-in from the plugin.

We didn't previously document the subscriptions array, so do that now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Plugins: `deprecated_oneshot` notifiction subscription to change deprecated status for a single command.
2024-01-26 10:30:22 +10:30
Rusty Russell
fb0a1c64fd libplugin: get i-promise-to-fix-broken-api-user list from lightningd.
This means it now covers plugin parameters too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-26 10:30:22 +10:30
Rusty Russell
50e7c71dc7 lightningd: mark all internal deprecations by version.
I did some CHANGELOG and git digging to see when these were deprecated, and
some were very old (v0.8.2!).  But since they didn't warn users loudly, I
chose to do so this release only.

I renamed ld's `deprecated_apis` to `deprecated_ok` to make sure I
caught them all.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-26 10:30:22 +10:30
Rusty Russell
5a6c529c69 pyln.plugin: Allow deprecated to be a string.
Type-checking in Python is so loose you could already do this, but this updates the
mypy type annotations.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-26 10:30:22 +10:30
Rusty Russell
277ed2ccbf libplugin: support version strings for deprecations.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-26 10:30:22 +10:30
Rusty Russell
3281d8c0ab plugins: allow plugin options deprecated to be an array of strings.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Plugin: options and commands can specify deprecation start (and optional end) versions.
2024-01-26 10:30:22 +10:30
Rusty Russell
f18ce6a3ce plugins: allow deprecated for registered commands to be an array of versions.
We still accept boolean: the plugin may not want to commit to a deprecation schedule.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Plugins: rpcmethods and options can set `deprecated` to a pair of version strings, not just a boolean.
2024-01-26 10:30:22 +10:30
Rusty Russell
8e6eaf2511 common: allow JSON-RPC parameters to specify deprecation versions.
This infrastructure is use by both libplugin and lightningd.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-26 10:30:22 +10:30
Rusty Russell
2fe4ba01cd jsonrpc: add deprecations command to locally set deprecated apis on/off.
This command allows more fine-grained testing, without having to change the config of the
lightning node.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `deprecations` to enable/disable deprecated APIs from this caller.
2024-01-26 10:30:22 +10:30
Rusty Russell
a8bdde0667 plugins/sql: use per-command deprecations.
In this case the cmd is `sql` but the field we're talking about is from
a different command, so we need a new libplugin API.

Note: there are still no deprecations in any tables used by `sql`, so this
is a bit moot for now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-26 10:30:22 +10:30
Rusty Russell
7431b8b436 common: add command_deprecated_param_ok() and command_deprecated_out_ok()
Generic helpers for libplugin and lightningd.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-26 10:30:22 +10:30
Rusty Russell
4764ec794c pytest: note where we use very deprecated features, so tests don't rbeak.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-26 10:30:22 +10:30
Rusty Russell
a46f62bf06 doc: start a list of current deprecations.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-26 10:30:22 +10:30
Rusty Russell
d051a1ee67 lightningd: i-promise-to-fix-broken-api-user option.
This allows the user to specify the feature *by name*, and hopefully
complain to the developer to fix their code, knowing it will be removed entirely
in the next release!

Changelog-Added: config: `i-promise-to-fix-broken-api-user` allows for a one-release re-enablement of long-deprecated features.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-26 10:30:22 +10:30
Rusty Russell
1131e40ddf plugins/sql: do deprecations at request time, not schema loading.
When we allow deprecation to be set per-connection, we need to
generalize this approach.  Instead of filtering out deprecated
fields at schema loading, we need to load them, then refuse
to serve deprecated fields on a per-request basis.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-26 10:30:22 +10:30
Rusty Russell
46ba105518 plugins/sql: make columns an array of pointers.
`struct column` has a dynamically allocated member, which is neater if
it is a tal object itself (we allocated the member off the array, instead).
We're about to add two new members, so clean this up first.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-26 10:30:22 +10:30
Rusty Russell
dadbdf488c schemas: deprecated is now a range.
Don't assume removal is +6 months, but have a start deprecation/end support range.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-26 10:30:22 +10:30
Rusty Russell
f824a4de8f msggen: generalize version handling.
Updating this every release was just busywork, and it turns out we don't actually
care: if something is marked deprecated we want to make it optional, whenever
it is for, and the only real test is if it was added before our lowest-supported
version we can consider it non-optional.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-26 10:30:22 +10:30
Rusty Russell
94a539ee3d Makefile: add CLN_NEXT_VERSION, functions encoding deprecation schedule.
Each feature has a name, and says when deprecation begins and ends.

There's an API coming to allow you to re-enable on a per-feature basis
even if it's ended (as long as it's not been removed from the code ofc!).

Default end is 6 months after deprecation, i.e. we complain about it
at that point, if we can detect its use.

e.g, a standard deprecation in v24.05:

v24.02: allowed
v24.02 with mods: allowed

master after v24.02: allowed unless deprecated APIs disabled.
v24.05: allowed unless deprecated APIs disabled.
v24.08: allowed unless deprecated APIs disabled.

v24.11: allowed unless deprecated APIs disabled, but logs at BROKEN level.

v25.02: allowed only if --i-promise-to-fix-broken-api-user=FEATURE.
v25.05: code is actually removed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-26 10:30:22 +10:30