Commit graph

14225 commits

Author SHA1 Message Date
Lagrang3
b0054aad46 renepay: accomodate fees in the payment flow
Min. Cost Flow does not take into account fees when computing a flow
    with liquidity constraints.
    This is a work-around solution that reduces the amount on every route to
    respect the liquidity bound. The deficity in the delivered amount is
    solved by running MCF once again.

    Changes:

    1. the function `flow_complete` allocates amounts to send over the set of routes
       computed by the MCF algorithm, but it does not allocate more than liquidity
       bound of the route. For this reason `minflow` returns a set of routes that
       satisfy the liquidity bounds but it is not guaranteed that the total payment
       reaches the destination therefore there could a deficit in the delivery:
       `deficit = amount_to_deliver - delivering`.

    2. in the function `add_payflows` after `minflow` returns a set of routes we
       call `flows_fit_amount` that tries to a allocate the `deficit` in the routes
       that the MCF have computed.

    3. if the resulting flows pass all payment constraints then we update
        `amount_to_deliver = amount_to_deliver - delivering`, and the loop
        repeats as long as `amount_to_deliver` is not zero.

    In other words, the excess amount, beyond the liquidity bound,
    in the routes is removed and then we try to allocate it
    into known routes, otherwise we do a whole MCF again just for the
    remaining amount.

    Fixes issue #6599
2024-01-29 10:48:24 +10:30
niftynei
a4f92eac81 bkpr: now that we're not doing empty acct logging, we dont need this bool
We stopped doing empty journal logs, so we no longer need to switch
our log severity based on whether or not an account exists.

Should make bookkeeper less chatty and remove noisy logs

Changelog-None
2024-01-29 10:05:03 +10:30
niftynei
ec7044e78e bkpr: use currency in balance snapshot 2024-01-29 10:05:03 +10:30
niftynei
5484aaee33 bkpr: dont log unknown accounts with zero balances
We were putting out a lot of empty journal entries. Let's
stop doing that.

Now the wallet balance stays uninitialized until/unless you have
funds in it.

Fixes #5672
2024-01-29 10:05:03 +10:30
Rusty Russell
040af90e7d libplugin: remove global deprecated_apis flag.
And we don't need to handle 0.9 lightningd which didn't include
allow-deprecated-apis in getmanifest call.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-26 10:30:22 +10:30
Rusty Russell
8d68c608de commando: use deprecation API for missing ids.
In this case we don't have a matching "command", so we need a special
API.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-26 10:30:22 +10:30
Rusty Russell
5f899d0dd2 plugins/bcli: use per-command deprecation flags.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-26 10:30:22 +10:30
Rusty Russell
5c484cd9e6 libplugin: handle deprecated_oneshot notification.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-26 10:30:22 +10:30
Rusty Russell
2914d3adfc plugins: allow plugins to get per-connection deprecated state.
Unfortunately, this is awkward: we just copy through most requests,
so we can't easily add a "deprecation" field to each one.  So we do
a notification if the next command has a different deprecation status
than the global one, but that requires opt-in from the plugin.

We didn't previously document the subscriptions array, so do that now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Plugins: `deprecated_oneshot` notifiction subscription to change deprecated status for a single command.
2024-01-26 10:30:22 +10:30
Rusty Russell
fb0a1c64fd libplugin: get i-promise-to-fix-broken-api-user list from lightningd.
This means it now covers plugin parameters too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-26 10:30:22 +10:30
Rusty Russell
50e7c71dc7 lightningd: mark all internal deprecations by version.
I did some CHANGELOG and git digging to see when these were deprecated, and
some were very old (v0.8.2!).  But since they didn't warn users loudly, I
chose to do so this release only.

I renamed ld's `deprecated_apis` to `deprecated_ok` to make sure I
caught them all.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-26 10:30:22 +10:30
Rusty Russell
5a6c529c69 pyln.plugin: Allow deprecated to be a string.
Type-checking in Python is so loose you could already do this, but this updates the
mypy type annotations.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-26 10:30:22 +10:30
Rusty Russell
277ed2ccbf libplugin: support version strings for deprecations.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-26 10:30:22 +10:30
Rusty Russell
3281d8c0ab plugins: allow plugin options deprecated to be an array of strings.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Plugin: options and commands can specify deprecation start (and optional end) versions.
2024-01-26 10:30:22 +10:30
Rusty Russell
f18ce6a3ce plugins: allow deprecated for registered commands to be an array of versions.
We still accept boolean: the plugin may not want to commit to a deprecation schedule.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Plugins: rpcmethods and options can set `deprecated` to a pair of version strings, not just a boolean.
2024-01-26 10:30:22 +10:30
Rusty Russell
8e6eaf2511 common: allow JSON-RPC parameters to specify deprecation versions.
This infrastructure is use by both libplugin and lightningd.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-26 10:30:22 +10:30
Rusty Russell
2fe4ba01cd jsonrpc: add deprecations command to locally set deprecated apis on/off.
This command allows more fine-grained testing, without having to change the config of the
lightning node.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `deprecations` to enable/disable deprecated APIs from this caller.
2024-01-26 10:30:22 +10:30
Rusty Russell
a8bdde0667 plugins/sql: use per-command deprecations.
In this case the cmd is `sql` but the field we're talking about is from
a different command, so we need a new libplugin API.

Note: there are still no deprecations in any tables used by `sql`, so this
is a bit moot for now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-26 10:30:22 +10:30
Rusty Russell
7431b8b436 common: add command_deprecated_param_ok() and command_deprecated_out_ok()
Generic helpers for libplugin and lightningd.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-26 10:30:22 +10:30
Rusty Russell
4764ec794c pytest: note where we use very deprecated features, so tests don't rbeak.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-26 10:30:22 +10:30
Rusty Russell
a46f62bf06 doc: start a list of current deprecations.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-26 10:30:22 +10:30
Rusty Russell
d051a1ee67 lightningd: i-promise-to-fix-broken-api-user option.
This allows the user to specify the feature *by name*, and hopefully
complain to the developer to fix their code, knowing it will be removed entirely
in the next release!

Changelog-Added: config: `i-promise-to-fix-broken-api-user` allows for a one-release re-enablement of long-deprecated features.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-26 10:30:22 +10:30
Rusty Russell
1131e40ddf plugins/sql: do deprecations at request time, not schema loading.
When we allow deprecation to be set per-connection, we need to
generalize this approach.  Instead of filtering out deprecated
fields at schema loading, we need to load them, then refuse
to serve deprecated fields on a per-request basis.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-26 10:30:22 +10:30
Rusty Russell
46ba105518 plugins/sql: make columns an array of pointers.
`struct column` has a dynamically allocated member, which is neater if
it is a tal object itself (we allocated the member off the array, instead).
We're about to add two new members, so clean this up first.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-26 10:30:22 +10:30
Rusty Russell
dadbdf488c schemas: deprecated is now a range.
Don't assume removal is +6 months, but have a start deprecation/end support range.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-26 10:30:22 +10:30
Rusty Russell
f824a4de8f msggen: generalize version handling.
Updating this every release was just busywork, and it turns out we don't actually
care: if something is marked deprecated we want to make it optional, whenever
it is for, and the only real test is if it was added before our lowest-supported
version we can consider it non-optional.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-26 10:30:22 +10:30
Rusty Russell
94a539ee3d Makefile: add CLN_NEXT_VERSION, functions encoding deprecation schedule.
Each feature has a name, and says when deprecation begins and ends.

There's an API coming to allow you to re-enable on a per-feature basis
even if it's ended (as long as it's not been removed from the code ofc!).

Default end is 6 months after deprecation, i.e. we complain about it
at that point, if we can detect its use.

e.g, a standard deprecation in v24.05:

v24.02: allowed
v24.02 with mods: allowed

master after v24.02: allowed unless deprecated APIs disabled.
v24.05: allowed unless deprecated APIs disabled.
v24.08: allowed unless deprecated APIs disabled.

v24.11: allowed unless deprecated APIs disabled, but logs at BROKEN level.

v25.02: allowed only if --i-promise-to-fix-broken-api-user=FEATURE.
v25.05: code is actually removed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-26 10:30:22 +10:30
Rusty Russell
ac4c396537 lightningd: reuse code for "dynamic" getmanifest response parsing.
And clean up weird indent.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-26 10:30:22 +10:30
Rusty Russell
ba2c912a39 config: remove disable-ip-discovery
Changelog-Removed: Config: `disable-ip-discovery` (deprecated in v23.02): use `announce-addr-discovered`
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-26 10:30:22 +10:30
Rusty Russell
a2dadfe46b funder: remove lease-fee-base-msat
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Experimental: Plugins: `funder` option "lease-fee-base-msat" removed (deprecated in v0.11, use "lease-fee-base-sat")
2024-01-26 10:30:22 +10:30
Rusty Russell
f342630b92 wallet: remove P2SH support.
Seriously, it's taproot time, let's get rid of p2sh wrapped segwit.

Changelog-Removed: wallet: removal of p2sh-segwit addresses; newaddr won't issue them, we won't watch them for new funds (deprecated in *23.02*)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-26 10:30:22 +10:30
Rusty Russell
222ca17333 pyln-client: remove long-obsolete support for msatoshi arg.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-26 10:30:22 +10:30
Rusty Russell
66c93dcfcf common: remove parameter aliases.
We currently don't have any!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-26 10:30:22 +10:30
Rusty Russell
5036605501 common: remove no-longer-required p_req_dup_ok hack.
This was for "msatoshi" and "amount_msat" in routes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-26 10:30:22 +10:30
Rusty Russell
5ef4779edc lightningd: remove msatoshi alias for amount_msat.
Changelog-Removed: JSON-RPC: `invoice`, `sendonion`, `sendpay`, `pay`, `keysend`, `fetchinvoice`, `sendinvoice`: `msatoshi` argument (deprecated 0.12.0). Use `amount_msat`.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-26 10:30:22 +10:30
Rusty Russell
0946a5ac35 lightningd: clean up notification infrastructure.
The `struct notification` lost type-safety, but avoided a redundant
string.  The string is better, I think.

Since all notifications now contain an object of same name (some have
deprecated fields outside that), we can add helpers to do that, too.

Also, add some const (easy to do now we're typesafe!)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-26 10:30:22 +10:30
Lagrang3
7c7d13d6dc renepay: add self-pay feature 2024-01-24 14:42:45 +10:30
singlebeam
71e41165e3 Update configuration.md 2024-01-23 12:44:08 +01:00
Matt Morehouse
3260d7dd5e fuzz: initial fuzz-bolt12-invoice-decode corpus 2024-01-23 20:21:48 +10:30
Matt Morehouse
1c18269cd1 fuzz: initial fuzz-bolt12-invrequest-decode corpus 2024-01-23 20:21:48 +10:30
Matt Morehouse
2073d9771d fuzz: initial fuzz-bolt12-offer-decode corpus 2024-01-23 20:21:48 +10:30
Matt Morehouse
3af61e087c fuzz: initial fuzz-bolt12-bech32-decode corpus 2024-01-23 20:21:48 +10:30
Matt Morehouse
7d05f07a2b fuzz: test bolt12 decoding
Add fuzz targets for decoding of offers, invoice requests, and invoices.
Mutators are shared in bolt12.h.
2024-01-23 20:21:48 +10:30
Matt Morehouse
b73f921a91 fuzz: test bolt12 bech32 decoding 2024-01-23 20:21:48 +10:30
Lagrang3
b19b08d749 fix CI test: uninitialized variable 2024-01-22 15:10:08 +10:30
Lagrang3
020a03fc3b fix CIL maybe-uninitialized variable 2024-01-22 15:10:08 +10:30
Lagrang3
102dce2515 error handling in mcf.c
Adds a "canonical" error handling in the module mcf.c,
adhering to the same convention we have adopted in flow.c.
So that when a function fails it signals the caller with an invalid
return value, or bool false, and it communicates an error message.
2024-01-22 15:10:08 +10:30
Lagrang3
acc8e8e96f move function linear_fee_cost to mcf 2024-01-22 15:10:08 +10:30
Lagrang3
7edf3c2a55 remove unnecessary arguments
Functions chan_extra_can_send and chan_extra_cannot_send
do not need to know the amount, because flow is already allocated in
htlc_total.
2024-01-22 15:10:08 +10:30
Lagrang3
feef829362 remove declaration of derive_mu 2024-01-22 15:10:08 +10:30