Commit Graph

15484 Commits

Author SHA1 Message Date
Rusty Russell
9d966b46a7 askrene: take into account the reduction in "spendable" with additional HTLCs.
"spendable" is for a single HTLC: if we own the channel, this amount
decreases with every HTLC, as we have to pay fees.  We have access to this since
we call listpeerchannels anyway, so we can calculate the additional costs and
use it in the refining phase.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-19 12:16:53 +09:30
Rusty Russell
3e96955c72 pytest: test askrene treats htlc_maximum_msat and htlc_minimum_msat restrictions properly.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-19 12:16:53 +09:30
Rusty Russell
db29a2d6b5 askrene: don't have get_flow_paths() handle htlc_max, htlc_min and extra millisats.
We don't actually hit the htlc_max cases, since the flow code already
constrains us to that.

And handling htlc_min is better done in the caller, where diagnostics
are better (basically, we should eliminate them, and if that means no
route, give a clear error message).

And the refinement step can handle any extra millisats from rounding.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-19 12:16:53 +09:30
Rusty Russell
f0331cd82e askrene: add a "refining" step to add fees and handle corner cases.
This is the root cause of the problem worked around in 50949b7b9c
"askrene: hack in some padding so we don't overflow capacities."

When adding fees to flows, we didn't recheck the boundary conditions: in
renepay this is done by routebuilder.

Fortunately, we can use our "reservations" infrastructure to temporarily
use capacity as we process flows, so we handle the cases where they are
not independent correclty.

My assumption is that the resulting errors are small, so we divide
them between the remaining flows based on highest-to-least
probability.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-19 12:16:53 +09:30
Rusty Russell
5883aa85ca askrene: rename struct flow amount to delivers.
This is clearer: it's the final amount, not the amount we send!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-19 12:16:53 +09:30
Rusty Russell
829954ac71 askrene: remove struct flow probability member.
Simply calculate it when we need it, which means we don't have to keep it
up-to-date as we tweak the flow.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-19 12:16:53 +09:30
Rusty Russell
923e37a545 askrene: remove unused flow routines.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-19 12:16:53 +09:30
Rusty Russell
17191259ea patch libplugin-batch.patch 2024-09-19 12:16:53 +09:30
Rusty Russell
1362448352 common/bolt12: do more required checks in invoice_decode.
Rather than making the callers do this, make the invoice decoder perform
the various sanity checks.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-19 12:16:53 +09:30
Rusty Russell
1b8a64f5e6 askrene: apply "auto.sourcefree" to channels created in layers, too.
We had a workaround for channels added by "auto.local", but instead we
should make it work properly.

I didn't do this before because we can't manipulate the localmods while
they're applied, but it's simple to do it in two stages.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-19 12:16:53 +09:30
Rusty Russell
06957dc832 pytest: separate out routine which checks only some fields of getroutes.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-19 12:16:53 +09:30
Rusty Russell
e90be8d957 pyln-testing: add gossip_store_file arg to get_node()
This makes it much easier to use generated gossip_store files.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-19 12:16:53 +09:30
Rusty Russell
5ea049ea12 pytest: duplicate test_live_spendable as a canned gossmap test.
Not quite the same, as it doesn't have the "auto.local" layer, but it exhibits
the same problem if we revert the fix for test_live_spendable.

And it's much faster!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-19 12:16:53 +09:30
Rusty Russell
b11642538e pytest: ensure there are no duplicate paths in test_live_spendable, and check total.
There aren't, but make sure.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-19 12:16:53 +09:30
Rusty Russell
679f46f733 common/amount: rename amount_sat_zero/amount_msat_zerp -> amount_sat_is_zero/amount_msat_is_zero.
I used `amount_msat_eq(x, AMOUNT_MSAT(0))` because I forgot this
function existed.  I probably missed it because the name is surprising,
so add "is" in there to make it clear it's a boolean function.

You'll note almost all the places which did use it are Eduardo's and
Lisa's code, so maybe it's just me.

Fix up a few places which I could use it, too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-19 12:16:53 +09:30
Rusty Russell
f05f871c92 common/amount: add amount_msat_accumulate()
Saves some typing, and is clearer than checking if both args really
are the same!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-19 12:16:53 +09:30
Rusty Russell
6150d1b3fa patch general-node-id-assign.patch 2024-09-19 12:16:53 +09:30
Rusty Russell
379a4ee16a common/amount: add routine to calculate fees backwards.
If I put in X, how much can I get out after fees are subtracted?

This was inspired by Eduardo's channel_maximum_forward in renepay, which
is basically the same thing.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-19 12:16:53 +09:30
Rusty Russell
db74ca7cbe lightningd: remove incorrect old-payment-htlc-delete logic.
This was incorrect once we stopped removing old payments on failure,
which was the reason we had to remove the HTLCs.

It also removed by partid, which is wrong since it should have done
old_payment->partid and old_payment->groupid!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-19 12:16:53 +09:30
ShahanaFarooqui
75c616c7b3 Makefile: Next version updated to 24.11
Changelog-None.
2024-09-18 16:59:27 +09:30
ShahanaFarooqui
5ba1824fd5 doc: Release checklist update 2024-09-18 16:59:27 +09:30
ShahanaFarooqui
8558993f77 doc: Pull request template update 2024-09-18 16:59:27 +09:30
ShahanaFarooqui
3d3e86e22c test: CI error fixes
- Removes CI value error for Broken logs
- Fixes CI errors due to deprecated listconfigs 'important-plugins'
- Removed listchannels deprecated local test

Changelog-None.
2024-09-18 16:59:27 +09:30
Michael Schmoock
02176f7004 pytests: extend the offline mode testcase 2024-09-17 19:53:25 +02:00
ShahanaFarooqui
7725ca1dbc meta: Add changelog for hotfix release v24.08.1
Changelog-None.
2024-09-17 10:24:24 -07:00
Rusty Russell
f0cf088489 lightningd: don't crash if plugin dies during initial plugins_init.
Thanks to Michael Schmook for the excellent bug report.

Fixes: https://github.com/ElementsProject/lightning/issues/7671
Changelog-Fixed: lightningd: no longer crash if a plugin dies during lightningd startup.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-17 10:24:01 -07:00
Rusty Russell
f4f6279b17 pytest: thoroughly test plugin death during startup.
We modify broken.py to be able to fail at different points,
and test that during startup.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-17 10:24:01 -07:00
Rusty Russell
4578748aca plugins/Makefile: fail if jq command fails.
Nested within the loop, jq would fail silently.

This can happen if jq is too old.

Fixes: https://github.com/ElementsProject/lightning/issues/7657
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-17 17:02:22 +02:00
Rusty Russell
5c870c90a6 configure: ensure that we have jq.
Otherwise we can get other crashes it seems?

Fixes: https://github.com/ElementsProject/lightning/issues/7657
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-17 17:02:22 +02:00
laanwj
9e8dd15a99 cln-plugin: Change default log level filter back to INFO
In commit 60e1532dd8 (released in crate 0.1.8), which switched the
logging framework to tracing-subscriber, the default log level filter
was (accidentally) set to ERROR and above, instead of INFO and above.

Change this back to INFO as it was before. It can still be overridden
with CLN_PLUGIN_LOG.

Follows the example in https://docs.rs/tracing-subscriber/latest/tracing_subscriber/filter/struct.EnvFilter.html#method.from_env

Closes #7658.

Changelog-Fixed: cln-plugin: Change default log level filter back to INFO
2024-09-17 12:00:27 +09:30
Rusty Russell
bfe0557f0b topology: fix overzealous deprecation check in listpeerchannels.
It's an output field (which we don't complain about), not an input field!

Fixes: https://github.com/ElementsProject/lightning/issues/7652
Changelog-Fixed: Logging: removed bogus "**BROKEN** plugin-topology: DEPRECATED API USED: listchannels.include_private" message.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-16 16:03:48 +09:30
Rusty Russell
7e2ea3ed04 pytest: add test for bogus deprecation BROKEN log.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-16 16:03:48 +09:30
laanwj
c8660c85ba schemas: "description" field in "wait(any)invoice" is optional since BOLT12
For BOLT12 invoices, the "description" is field missing in the invoice
responses. Update the schemas accordingly:

- `doc/schemas/lightning-waitanyinvoice.json`
- `doc/schemas/lightning-waitinvoice.json`

Also commit the generated msggen, cln-grpc, cln-rpc and pyln-grpc-proto files.

Changelog-Fixed: schemas: Make description in `Wait(any)invoiceResponse` optional to handle BOLT12
2024-09-16 16:03:07 +09:30
Rusty Russell
5bd3d51131 subds: fix case where we keep retrying on EOF.
Our low-level ccan/io IO routines return three values:
-1: error.
0: call me again, I'm not finished.
1: I'm done, go onto the next thing.

In the last release, we tweaked the sematics of "-1": we now opportunistically
call a routine which returns 0 once more, in case there's more data.  We use errno to
distinguish between "EAGAIN" which means there wasn't any data, and real errors.

However, if the underlying read() returns 0 (which it does when the peer has closed
the other end) the value of errno is UNDEFINED.  If it happens to be EAGAIN, we will
call it again, rather than closing.  This causes us to spin: in particular people reported
hsmd consuming 100% of CPU.

The ccan/io read code handled this by setting errno to 0 in this case, but our own
wire low-level routines *did not*.

Fixes: https://github.com/ElementsProject/lightning/issues/7655
Changelog-Fixed: Fixed intermittant bug where hsmd (particularly, but also lightningd) could use 100% CPU.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-13 11:13:53 -07:00
Rusty Russell
c5fc1b55d8 lightningd: make sure channels always have a local alias.
If we pull and old channel from the database, it might not.

Fixes: https://github.com/ElementsProject/lightning/issues/7645
Changelog-Fixes: lightningd: crash when starting channeld for older channel with no local alias.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-13 14:09:58 +09:30
Lagrang3
2bd9c2285a lightningd: log if builtin plugin fails to start
Lightningd should log if a builtin plugin fails to start instead of
silently skip over it.

Changelog-Add: log message if builtin plugin fails to start.

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2024-09-10 08:34:49 -07:00
Rusty Russell
78b9ccf432 channeld: fix crash with pre-TLV peers.
send_tlvs is NULL if no special features are supported, and peer
sets `next_to_send` anyway:

```
0x5ed1c6719538 peer_reconnect
channeld/channeld.c:5205
0x5ed1c6719dab init_channel
channeld/channeld.c:5959
0x5ed1c6719f04 main
channeld/channeld.c:6005
```

Backport: v24.08
Fixes: https://github.com/ElementsProject/lightning/issues/7486
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-07 13:55:45 +02:00
evansmj
873e8b4ef8 docs: clarify writing-json-schemas.md and autogenerate-rpc-examples.py
Spelling and clarity on how to update schemas when adding RPCs.
2024-09-05 17:01:04 -07:00
ShahanaFarooqui
66d4d0f8fc doc: After release instructions update
- Updated `release-checklist` with new GitHub actions for Docker and PyPI publishing.
- Updated `repro` with information about the `cl-repro.sh` script.

Changelog-None.
2024-09-05 16:28:31 -07:00
Jesse de Wit
15eb1e6ff5 cln-tracer: readme usage clarifications 2024-09-05 15:49:03 -07:00
ShahanaFarooqui
1a71e15cf3 github: Read tag on HEAD in Github Actions
Running `git describe` in local setup correctly identifies the tag available on the tag. But `docker buildx` via actions does not identify it without `--tags` param.

Reference Issue: https://github.com/ElementsProject/lightning/issues/7626

Changelog-Fixed: Docker image created via github actions correctly reads the tag available on the HEAD.
2024-09-05 12:31:49 -07:00
ShahanaFarooqui
ad37328600 docker: git reset to avoid modded suffix
Changelog-Fixed: Core lightning's version will not be suffixed with -modded anymore.
2024-09-05 12:31:49 -07:00
ShahanaFarooqui
bb72ad39b1 github: Actions trigger of tag push
Changelog-Fixed: Regex expression for github actions tags push trigger.
2024-09-05 12:31:49 -07:00
ShahanaFarooqui
84b3e80d16 test: testplugpass requirements.txt updated to install local pyln-client
Python was installing `pyln-client` directly from the server for testplugpass plugin. This commit is updating the requirements.txt file to install pyln-client with absolute local path.
2024-09-03 23:24:12 -07:00
Rusty Russell
46fe1d34e6 pyln-client: try to bump version
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-03 23:24:12 -07:00
Rusty Russell
dc32e2c0dc pyln-client: don't refer to now-non-existent long_desc member.
This breaks reckless:

```
[2024-09-01 12:24:14,755] DEBUG:       doc = method.long_desc if method.long_desc is not None else "No documentation found"\n[2024-09-01 12:24:14,755] DEBUG:                                 ^^^^^^^^^^^^^^^^\n[2024-09-01 12:24:14,755] DEBUG:   AttributeError: \'Method\' object has no attribute \'long_desc\'\n[2024-09-01 12:24:14,755] ERROR: plugin testing failed\n[2024-09-01 12:24:14,770] WARNING: testplugpass: installation aborted\n'
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-03 23:24:12 -07:00
ShahanaFarooqui
5ec5580a58 meta: update changelog for v24.08
Changelog-None
2024-08-28 21:49:21 -07:00
ShahanaFarooqui
6c25d2095c docker: Cleaned up Dockerfile's Core Lightning installation
With PR #7618, Core Lightning installation with relative paths has been fixed and can be used again.

Changelog-None.
2024-08-28 17:31:40 -07:00
Rusty Russell
624d8b79f7 lightningd: fix up installs in subdirectories.
Commit a1fdeee76b "Makefile: clean up install path handling."
broke the ability to configure with one path and then run in a
different path.  Turns out people actually do this!  So, we have
to use relative paths, compared to our existing binary.

And we can't use path_rel, because that requires that the path
exist (thanks @Lagrang3!).

Fixes: https://github.com/ElementsProject/lightning/issues/7595
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-29 08:14:16 +09:30
ShahanaFarooqui
48fa438b15 script: Updating SHASUMS for updated zlib1g-dev in Ubuntu Noble
Changelog-Fixed: core lightning's reproducible build for Ubuntu v24.04 (noble).
2024-08-26 17:10:22 -07:00