Commit graph

15395 commits

Author SHA1 Message Date
Peter Neuroth
0a6870f13d rust-libs: Bump versions
We bump the version to minor version 0.2 to allow ourselves to bump on
patch level when we need to fix a bug or update a dependency.

Changelog-Changed: Plugins: `cln-grpc` Upgrade tonic version and
introduce new versioning scheme.

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2024-09-22 00:12:39 -07:00
Peter Neuroth
a6366c2814 rust-libs: Upgrade tonic to v0.12
Tonic had some breaking changes since 0.8. We want to be closer to newer
versions for downstream consumers to be able to benefit from changes in tonic.

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2024-09-22 00:12:39 -07:00
Jesse de Wit
2798b4b7be cln-grpc: add anchors/even to primitives
The `anchors/even` channel type was missing from the grpc bindings.
It is now the default channel type, so `fundchannel` over grpc fails
with:

```
Failed to deserialize response : unknown variant `anchors/even`,
expected one of `static_remotekey/even`, `anchor_outputs/even`,
`anchors_zero_fee_htlc_tx/even`, `scid_alias/even`, `zeroconf/even`
```

Changelog-Changed: Channel type `anchors/even` was added
to the grpc bindings.
2024-09-20 17:01:04 -07:00
Rusty Russell
aba67a757c lightningd: rename state_change_entry to channel_state_change, and use pointers.
This name is clearer than the old one.

And since the struct contains a string, it's more natural for the
struct to be the tal parent of the string so it's a real object.  This means
we need an array of pointers, so each struct can be its own tal object.

wallet_state_change_get is hoisted higher in the code and made static.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-21 06:20:18 +09:30
Rusty Russell
0440700132 lightningd: move the state changes into struct channel.
And instead of loading them in listpeerchannels, use them.  This means
listpeerchannels no longer touches the db.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: JSONRPC: `listpeerchannels` (and thus, pay) sped up on very large nodes.
2024-09-21 06:20:18 +09:30
Rusty Russell
eb979980a9 db: clean up channel_stats handling.
Indirection via a string and an enum is just adding confusion here.

And move the `struct channel_stats` into channel.h.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-21 06:20:18 +09:30
Rusty Russell
3aace10828 listpeerchannels: use struct channel stats, don't fetch from db.
This avoids a db lookup on every iteration of listpeerchannels, which
can be slow on large nodes (Postgres, I assume).

We can now simply add the fields we want to channel load, and remove
wallet_channel_stats_load entirely.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-21 06:20:18 +09:30
Rusty Russell
762d624810 lightningd: move stats into struct channel.
Like other fields, mirror them into the db.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-21 06:20:18 +09:30
Lagrang3
0aa52b7fdd askrene: remove unused function
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2024-09-19 12:16:53 +09:30
Rusty Russell
9b60f6cc6d askrene: re-check min_htlc violations after correcting for MCF rounding.
Thanks to @Lagrang3 for spotting this!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-19 12:16:53 +09:30
Rusty Russell
8c551ae48a askrene: move flow refining code to its own file.
askrene.c was getting quite long, and this is self-contained.

The only code change is a convenience accessor for the per-htlc-cost
hash table.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-19 12:16:53 +09:30
Rusty Russell
9d966b46a7 askrene: take into account the reduction in "spendable" with additional HTLCs.
"spendable" is for a single HTLC: if we own the channel, this amount
decreases with every HTLC, as we have to pay fees.  We have access to this since
we call listpeerchannels anyway, so we can calculate the additional costs and
use it in the refining phase.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-19 12:16:53 +09:30
Rusty Russell
3e96955c72 pytest: test askrene treats htlc_maximum_msat and htlc_minimum_msat restrictions properly.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-19 12:16:53 +09:30
Rusty Russell
db29a2d6b5 askrene: don't have get_flow_paths() handle htlc_max, htlc_min and extra millisats.
We don't actually hit the htlc_max cases, since the flow code already
constrains us to that.

And handling htlc_min is better done in the caller, where diagnostics
are better (basically, we should eliminate them, and if that means no
route, give a clear error message).

And the refinement step can handle any extra millisats from rounding.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-19 12:16:53 +09:30
Rusty Russell
f0331cd82e askrene: add a "refining" step to add fees and handle corner cases.
This is the root cause of the problem worked around in 50949b7b9c
"askrene: hack in some padding so we don't overflow capacities."

When adding fees to flows, we didn't recheck the boundary conditions: in
renepay this is done by routebuilder.

Fortunately, we can use our "reservations" infrastructure to temporarily
use capacity as we process flows, so we handle the cases where they are
not independent correclty.

My assumption is that the resulting errors are small, so we divide
them between the remaining flows based on highest-to-least
probability.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-19 12:16:53 +09:30
Rusty Russell
5883aa85ca askrene: rename struct flow amount to delivers.
This is clearer: it's the final amount, not the amount we send!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-19 12:16:53 +09:30
Rusty Russell
829954ac71 askrene: remove struct flow probability member.
Simply calculate it when we need it, which means we don't have to keep it
up-to-date as we tweak the flow.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-19 12:16:53 +09:30
Rusty Russell
923e37a545 askrene: remove unused flow routines.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-19 12:16:53 +09:30
Rusty Russell
17191259ea patch libplugin-batch.patch 2024-09-19 12:16:53 +09:30
Rusty Russell
1362448352 common/bolt12: do more required checks in invoice_decode.
Rather than making the callers do this, make the invoice decoder perform
the various sanity checks.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-19 12:16:53 +09:30
Rusty Russell
1b8a64f5e6 askrene: apply "auto.sourcefree" to channels created in layers, too.
We had a workaround for channels added by "auto.local", but instead we
should make it work properly.

I didn't do this before because we can't manipulate the localmods while
they're applied, but it's simple to do it in two stages.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-19 12:16:53 +09:30
Rusty Russell
06957dc832 pytest: separate out routine which checks only some fields of getroutes.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-19 12:16:53 +09:30
Rusty Russell
e90be8d957 pyln-testing: add gossip_store_file arg to get_node()
This makes it much easier to use generated gossip_store files.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-19 12:16:53 +09:30
Rusty Russell
5ea049ea12 pytest: duplicate test_live_spendable as a canned gossmap test.
Not quite the same, as it doesn't have the "auto.local" layer, but it exhibits
the same problem if we revert the fix for test_live_spendable.

And it's much faster!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-19 12:16:53 +09:30
Rusty Russell
b11642538e pytest: ensure there are no duplicate paths in test_live_spendable, and check total.
There aren't, but make sure.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-19 12:16:53 +09:30
Rusty Russell
679f46f733 common/amount: rename amount_sat_zero/amount_msat_zerp -> amount_sat_is_zero/amount_msat_is_zero.
I used `amount_msat_eq(x, AMOUNT_MSAT(0))` because I forgot this
function existed.  I probably missed it because the name is surprising,
so add "is" in there to make it clear it's a boolean function.

You'll note almost all the places which did use it are Eduardo's and
Lisa's code, so maybe it's just me.

Fix up a few places which I could use it, too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-19 12:16:53 +09:30
Rusty Russell
f05f871c92 common/amount: add amount_msat_accumulate()
Saves some typing, and is clearer than checking if both args really
are the same!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-19 12:16:53 +09:30
Rusty Russell
6150d1b3fa patch general-node-id-assign.patch 2024-09-19 12:16:53 +09:30
Rusty Russell
379a4ee16a common/amount: add routine to calculate fees backwards.
If I put in X, how much can I get out after fees are subtracted?

This was inspired by Eduardo's channel_maximum_forward in renepay, which
is basically the same thing.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-19 12:16:53 +09:30
Rusty Russell
db74ca7cbe lightningd: remove incorrect old-payment-htlc-delete logic.
This was incorrect once we stopped removing old payments on failure,
which was the reason we had to remove the HTLCs.

It also removed by partid, which is wrong since it should have done
old_payment->partid and old_payment->groupid!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-19 12:16:53 +09:30
ShahanaFarooqui
75c616c7b3 Makefile: Next version updated to 24.11
Changelog-None.
2024-09-18 16:59:27 +09:30
ShahanaFarooqui
5ba1824fd5 doc: Release checklist update 2024-09-18 16:59:27 +09:30
ShahanaFarooqui
8558993f77 doc: Pull request template update 2024-09-18 16:59:27 +09:30
ShahanaFarooqui
3d3e86e22c test: CI error fixes
- Removes CI value error for Broken logs
- Fixes CI errors due to deprecated listconfigs 'important-plugins'
- Removed listchannels deprecated local test

Changelog-None.
2024-09-18 16:59:27 +09:30
Michael Schmoock
02176f7004 pytests: extend the offline mode testcase 2024-09-17 19:53:25 +02:00
ShahanaFarooqui
7725ca1dbc meta: Add changelog for hotfix release v24.08.1
Changelog-None.
2024-09-17 10:24:24 -07:00
Rusty Russell
f0cf088489 lightningd: don't crash if plugin dies during initial plugins_init.
Thanks to Michael Schmook for the excellent bug report.

Fixes: https://github.com/ElementsProject/lightning/issues/7671
Changelog-Fixed: lightningd: no longer crash if a plugin dies during lightningd startup.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-17 10:24:01 -07:00
Rusty Russell
f4f6279b17 pytest: thoroughly test plugin death during startup.
We modify broken.py to be able to fail at different points,
and test that during startup.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-17 10:24:01 -07:00
Rusty Russell
4578748aca plugins/Makefile: fail if jq command fails.
Nested within the loop, jq would fail silently.

This can happen if jq is too old.

Fixes: https://github.com/ElementsProject/lightning/issues/7657
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-17 17:02:22 +02:00
Rusty Russell
5c870c90a6 configure: ensure that we have jq.
Otherwise we can get other crashes it seems?

Fixes: https://github.com/ElementsProject/lightning/issues/7657
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-17 17:02:22 +02:00
laanwj
9e8dd15a99 cln-plugin: Change default log level filter back to INFO
In commit 60e1532dd8 (released in crate 0.1.8), which switched the
logging framework to tracing-subscriber, the default log level filter
was (accidentally) set to ERROR and above, instead of INFO and above.

Change this back to INFO as it was before. It can still be overridden
with CLN_PLUGIN_LOG.

Follows the example in https://docs.rs/tracing-subscriber/latest/tracing_subscriber/filter/struct.EnvFilter.html#method.from_env

Closes #7658.

Changelog-Fixed: cln-plugin: Change default log level filter back to INFO
2024-09-17 12:00:27 +09:30
Rusty Russell
bfe0557f0b topology: fix overzealous deprecation check in listpeerchannels.
It's an output field (which we don't complain about), not an input field!

Fixes: https://github.com/ElementsProject/lightning/issues/7652
Changelog-Fixed: Logging: removed bogus "**BROKEN** plugin-topology: DEPRECATED API USED: listchannels.include_private" message.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-16 16:03:48 +09:30
Rusty Russell
7e2ea3ed04 pytest: add test for bogus deprecation BROKEN log.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-16 16:03:48 +09:30
laanwj
c8660c85ba schemas: "description" field in "wait(any)invoice" is optional since BOLT12
For BOLT12 invoices, the "description" is field missing in the invoice
responses. Update the schemas accordingly:

- `doc/schemas/lightning-waitanyinvoice.json`
- `doc/schemas/lightning-waitinvoice.json`

Also commit the generated msggen, cln-grpc, cln-rpc and pyln-grpc-proto files.

Changelog-Fixed: schemas: Make description in `Wait(any)invoiceResponse` optional to handle BOLT12
2024-09-16 16:03:07 +09:30
Rusty Russell
5bd3d51131 subds: fix case where we keep retrying on EOF.
Our low-level ccan/io IO routines return three values:
-1: error.
0: call me again, I'm not finished.
1: I'm done, go onto the next thing.

In the last release, we tweaked the sematics of "-1": we now opportunistically
call a routine which returns 0 once more, in case there's more data.  We use errno to
distinguish between "EAGAIN" which means there wasn't any data, and real errors.

However, if the underlying read() returns 0 (which it does when the peer has closed
the other end) the value of errno is UNDEFINED.  If it happens to be EAGAIN, we will
call it again, rather than closing.  This causes us to spin: in particular people reported
hsmd consuming 100% of CPU.

The ccan/io read code handled this by setting errno to 0 in this case, but our own
wire low-level routines *did not*.

Fixes: https://github.com/ElementsProject/lightning/issues/7655
Changelog-Fixed: Fixed intermittant bug where hsmd (particularly, but also lightningd) could use 100% CPU.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-13 11:13:53 -07:00
Rusty Russell
c5fc1b55d8 lightningd: make sure channels always have a local alias.
If we pull and old channel from the database, it might not.

Fixes: https://github.com/ElementsProject/lightning/issues/7645
Changelog-Fixes: lightningd: crash when starting channeld for older channel with no local alias.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-13 14:09:58 +09:30
Lagrang3
2bd9c2285a lightningd: log if builtin plugin fails to start
Lightningd should log if a builtin plugin fails to start instead of
silently skip over it.

Changelog-Add: log message if builtin plugin fails to start.

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2024-09-10 08:34:49 -07:00
Rusty Russell
78b9ccf432 channeld: fix crash with pre-TLV peers.
send_tlvs is NULL if no special features are supported, and peer
sets `next_to_send` anyway:

```
0x5ed1c6719538 peer_reconnect
channeld/channeld.c:5205
0x5ed1c6719dab init_channel
channeld/channeld.c:5959
0x5ed1c6719f04 main
channeld/channeld.c:6005
```

Backport: v24.08
Fixes: https://github.com/ElementsProject/lightning/issues/7486
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-07 13:55:45 +02:00
evansmj
873e8b4ef8 docs: clarify writing-json-schemas.md and autogenerate-rpc-examples.py
Spelling and clarity on how to update schemas when adding RPCs.
2024-09-05 17:01:04 -07:00
ShahanaFarooqui
66d4d0f8fc doc: After release instructions update
- Updated `release-checklist` with new GitHub actions for Docker and PyPI publishing.
- Updated `repro` with information about the `cl-repro.sh` script.

Changelog-None.
2024-09-05 16:28:31 -07:00