Commit Graph

15609 Commits

Author SHA1 Message Date
Rusty Russell
f0cf088489 lightningd: don't crash if plugin dies during initial plugins_init.
Thanks to Michael Schmook for the excellent bug report.

Fixes: https://github.com/ElementsProject/lightning/issues/7671
Changelog-Fixed: lightningd: no longer crash if a plugin dies during lightningd startup.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-17 10:24:01 -07:00
Rusty Russell
f4f6279b17 pytest: thoroughly test plugin death during startup.
We modify broken.py to be able to fail at different points,
and test that during startup.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-17 10:24:01 -07:00
Rusty Russell
4578748aca plugins/Makefile: fail if jq command fails.
Nested within the loop, jq would fail silently.

This can happen if jq is too old.

Fixes: https://github.com/ElementsProject/lightning/issues/7657
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-17 17:02:22 +02:00
Rusty Russell
5c870c90a6 configure: ensure that we have jq.
Otherwise we can get other crashes it seems?

Fixes: https://github.com/ElementsProject/lightning/issues/7657
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-17 17:02:22 +02:00
laanwj
9e8dd15a99 cln-plugin: Change default log level filter back to INFO
In commit 60e1532dd8 (released in crate 0.1.8), which switched the
logging framework to tracing-subscriber, the default log level filter
was (accidentally) set to ERROR and above, instead of INFO and above.

Change this back to INFO as it was before. It can still be overridden
with CLN_PLUGIN_LOG.

Follows the example in https://docs.rs/tracing-subscriber/latest/tracing_subscriber/filter/struct.EnvFilter.html#method.from_env

Closes #7658.

Changelog-Fixed: cln-plugin: Change default log level filter back to INFO
2024-09-17 12:00:27 +09:30
Rusty Russell
bfe0557f0b topology: fix overzealous deprecation check in listpeerchannels.
It's an output field (which we don't complain about), not an input field!

Fixes: https://github.com/ElementsProject/lightning/issues/7652
Changelog-Fixed: Logging: removed bogus "**BROKEN** plugin-topology: DEPRECATED API USED: listchannels.include_private" message.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-16 16:03:48 +09:30
Rusty Russell
7e2ea3ed04 pytest: add test for bogus deprecation BROKEN log.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-16 16:03:48 +09:30
laanwj
c8660c85ba schemas: "description" field in "wait(any)invoice" is optional since BOLT12
For BOLT12 invoices, the "description" is field missing in the invoice
responses. Update the schemas accordingly:

- `doc/schemas/lightning-waitanyinvoice.json`
- `doc/schemas/lightning-waitinvoice.json`

Also commit the generated msggen, cln-grpc, cln-rpc and pyln-grpc-proto files.

Changelog-Fixed: schemas: Make description in `Wait(any)invoiceResponse` optional to handle BOLT12
2024-09-16 16:03:07 +09:30
Rusty Russell
5bd3d51131 subds: fix case where we keep retrying on EOF.
Our low-level ccan/io IO routines return three values:
-1: error.
0: call me again, I'm not finished.
1: I'm done, go onto the next thing.

In the last release, we tweaked the sematics of "-1": we now opportunistically
call a routine which returns 0 once more, in case there's more data.  We use errno to
distinguish between "EAGAIN" which means there wasn't any data, and real errors.

However, if the underlying read() returns 0 (which it does when the peer has closed
the other end) the value of errno is UNDEFINED.  If it happens to be EAGAIN, we will
call it again, rather than closing.  This causes us to spin: in particular people reported
hsmd consuming 100% of CPU.

The ccan/io read code handled this by setting errno to 0 in this case, but our own
wire low-level routines *did not*.

Fixes: https://github.com/ElementsProject/lightning/issues/7655
Changelog-Fixed: Fixed intermittant bug where hsmd (particularly, but also lightningd) could use 100% CPU.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-13 11:13:53 -07:00
Rusty Russell
c5fc1b55d8 lightningd: make sure channels always have a local alias.
If we pull and old channel from the database, it might not.

Fixes: https://github.com/ElementsProject/lightning/issues/7645
Changelog-Fixes: lightningd: crash when starting channeld for older channel with no local alias.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-13 14:09:58 +09:30
Lagrang3
2bd9c2285a lightningd: log if builtin plugin fails to start
Lightningd should log if a builtin plugin fails to start instead of
silently skip over it.

Changelog-Add: log message if builtin plugin fails to start.

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2024-09-10 08:34:49 -07:00
Rusty Russell
78b9ccf432 channeld: fix crash with pre-TLV peers.
send_tlvs is NULL if no special features are supported, and peer
sets `next_to_send` anyway:

```
0x5ed1c6719538 peer_reconnect
channeld/channeld.c:5205
0x5ed1c6719dab init_channel
channeld/channeld.c:5959
0x5ed1c6719f04 main
channeld/channeld.c:6005
```

Backport: v24.08
Fixes: https://github.com/ElementsProject/lightning/issues/7486
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-07 13:55:45 +02:00
evansmj
873e8b4ef8 docs: clarify writing-json-schemas.md and autogenerate-rpc-examples.py
Spelling and clarity on how to update schemas when adding RPCs.
2024-09-05 17:01:04 -07:00
ShahanaFarooqui
66d4d0f8fc doc: After release instructions update
- Updated `release-checklist` with new GitHub actions for Docker and PyPI publishing.
- Updated `repro` with information about the `cl-repro.sh` script.

Changelog-None.
2024-09-05 16:28:31 -07:00
Jesse de Wit
15eb1e6ff5 cln-tracer: readme usage clarifications 2024-09-05 15:49:03 -07:00
ShahanaFarooqui
1a71e15cf3 github: Read tag on HEAD in Github Actions
Running `git describe` in local setup correctly identifies the tag available on the tag. But `docker buildx` via actions does not identify it without `--tags` param.

Reference Issue: https://github.com/ElementsProject/lightning/issues/7626

Changelog-Fixed: Docker image created via github actions correctly reads the tag available on the HEAD.
2024-09-05 12:31:49 -07:00
ShahanaFarooqui
ad37328600 docker: git reset to avoid modded suffix
Changelog-Fixed: Core lightning's version will not be suffixed with -modded anymore.
2024-09-05 12:31:49 -07:00
ShahanaFarooqui
bb72ad39b1 github: Actions trigger of tag push
Changelog-Fixed: Regex expression for github actions tags push trigger.
2024-09-05 12:31:49 -07:00
ShahanaFarooqui
84b3e80d16 test: testplugpass requirements.txt updated to install local pyln-client
Python was installing `pyln-client` directly from the server for testplugpass plugin. This commit is updating the requirements.txt file to install pyln-client with absolute local path.
2024-09-03 23:24:12 -07:00
Rusty Russell
46fe1d34e6 pyln-client: try to bump version
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-03 23:24:12 -07:00
Rusty Russell
dc32e2c0dc pyln-client: don't refer to now-non-existent long_desc member.
This breaks reckless:

```
[2024-09-01 12:24:14,755] DEBUG:       doc = method.long_desc if method.long_desc is not None else "No documentation found"\n[2024-09-01 12:24:14,755] DEBUG:                                 ^^^^^^^^^^^^^^^^\n[2024-09-01 12:24:14,755] DEBUG:   AttributeError: \'Method\' object has no attribute \'long_desc\'\n[2024-09-01 12:24:14,755] ERROR: plugin testing failed\n[2024-09-01 12:24:14,770] WARNING: testplugpass: installation aborted\n'
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-03 23:24:12 -07:00
ShahanaFarooqui
5ec5580a58 meta: update changelog for v24.08
Changelog-None
2024-08-28 21:49:21 -07:00
ShahanaFarooqui
6c25d2095c docker: Cleaned up Dockerfile's Core Lightning installation
With PR #7618, Core Lightning installation with relative paths has been fixed and can be used again.

Changelog-None.
2024-08-28 17:31:40 -07:00
Rusty Russell
624d8b79f7 lightningd: fix up installs in subdirectories.
Commit a1fdeee76b "Makefile: clean up install path handling."
broke the ability to configure with one path and then run in a
different path.  Turns out people actually do this!  So, we have
to use relative paths, compared to our existing binary.

And we can't use path_rel, because that requires that the path
exist (thanks @Lagrang3!).

Fixes: https://github.com/ElementsProject/lightning/issues/7595
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-29 08:14:16 +09:30
ShahanaFarooqui
48fa438b15 script: Updating SHASUMS for updated zlib1g-dev in Ubuntu Noble
Changelog-Fixed: core lightning's reproducible build for Ubuntu v24.04 (noble).
2024-08-26 17:10:22 -07:00
ShahanaFarooqui
c9552e4228 meta: Update changelog for 24.08rc3
Release candidate 3 for v24.08
2024-08-26 11:53:42 -07:00
Rusty Russell
9d88ce3b59 pytest: suppress failing renepay test for Liquid.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-23 18:52:15 +09:30
Rusty Russell
50949b7b9c askrene: hack in some padding so we don't overflow capacities.
Of course, we still will, since spendable is for a single HTLC, but
this also shows why we should treat *minimum* as the incorrect answer
if they cross, too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes: https://github.com/ElementsProject/lightning/issues/7563
2024-08-23 18:52:15 +09:30
Rusty Russell
fafda82d82 askrene: fix up our handling of htlc_max.
It seems we didn't handle it correctly: we need to cap the first
segment as well as the others, as far as I can tell.

Also, it can be less than the maximum capacity.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-23 18:52:15 +09:30
Rusty Russell
975326ab5d askrene: round capacity *down* when converting to fp16.
Conversion is lossy, and we don't want to spend more than the channel,
so it's conservative to round down here.

This doesn't actually help our test though!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-23 18:52:15 +09:30
Rusty Russell
af011f9dac pytest: make spendable test for askrene work properly, give a better name.
Based on great test case from https://github.com/daywalker90

```
E       AssertionError: assert {'107x2x0/1': 'Path total 285720859 > spendable 285718000', '108x1x0/1': 'Path total 384721849 > spendable 384718000'} == {}
E         Left contains 2 more items:
E         {'107x2x0/1': 'Path total 285720859 > spendable 285718000',
E          '108x1x0/1': 'Path total 384721849 > spendable 384718000'}
E         Full diff:
E           {
E         -  ,
E         +  '107x2x0/1': 'Path total 285720859 > spendable 285718000',
E         +  '108x1x0/1': 'Path total 384721849 > spendable 384718000',
E           }
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-23 18:52:15 +09:30
ShahanaFarooqui
4e309dc478 docs: Added info for NFS version
Changelog-None.
2024-08-22 11:52:19 -07:00
ShahanaFarooqui
382b496c5f github: Update FreeBSD Test
- Passing bitcoind version from matrix

- cleanup: Removed extra tabs and `-` from pypi.yml

Changelog-Fixed: Github action for FreeBSD setup
2024-08-22 20:42:29 +09:30
ShahanaFarooqui
3a4dd5f032 github: Update Github actions for Pypi publishing
Changelog-Fixed: Fixed the already existing Pypi publishing actions file.
2024-08-21 10:06:49 -07:00
ShahanaFarooqui
dadf9a78e2 docker: Fix for cryptography v42 upgrade
- **Poetry Issue:** The Docker build for `linux/arm/v7` failed in recent RC releases on the Poetry installation step in the `builder-python` stage. This issue occurred because the `builder-python` stage builds on target's arch but poetry was unable to install on arm/v7 without rust >= v1.56.1.
    - **Solution:** Instead of installing poetry on the `builder-python` stage, we leveraged the existing multi-arch `builder` stage, which already had Poetry. Now, we export the dependencies from `pyproject.toml` to `requirements.txt` within the `builder` stage and then copy `requirements.txt` to the `builder-python` stage for pip installation.

- **Cryptography installation Issue:** python installations for `pyln-proto` started failing due to Cryptography upgrade from v41 to v42 (#7475). It is because now Cryptography needs cargo/rust also.
    - **Solution:** Installing cargo in `builder-python` stage also.

- **Configure Prefix Issue:** Previously, we used `RUN ./configure --prefix=/tmp/lightning_install --enable-static` in the `builder` image and then copied `/tmp/lightning_install` from the `builder` stage to `/usr/local` in the `final` stage. However, this approach is now causing errors due to missing binaries/plugins at their default locations.
    - **Solution:** We are now configuring the installation to use the default location (`/usr/local`). To prevent the local image size from increasing by up to 87MB, instead of copying the entire `/usr/local/` directory, we are explicitly copying only the core lightning binaries.

Changelog-Fixed: Fixes failing Docker build for `arm32` arch.
2024-08-20 21:25:07 -07:00
ShahanaFarooqui
5d1b5b5faa docker: GitHub actions testing for docker release
- Temporarily adding `rc` tag trigger for testing `Build and push multi-platform docker images` action flow before the final release.

- Added some variable inputs for testing like repo, platforms, etc.

- Added more logs for future debugging.
2024-08-20 21:25:07 -07:00
Rusty Russell
76ef283b02 lightningd: fix another trivial memleak when running installed version.
```
**BROKEN** lightningd: MEMLEAK: 0x55ff8e8ec268
**BROKEN** lightningd:   label=char[]
**BROKEN** lightningd:   alloc:
**BROKEN** lightningd:     ccan/ccan/tal/tal.c:488 (tal_alloc_)
**BROKEN** lightningd:     ccan/ccan/tal/tal.c:517 (tal_alloc_arr_)
**BROKEN** lightningd:     ccan/ccan/tal/str/str.c:81 (tal_vfmt_)
**BROKEN** lightningd:     ccan/ccan/tal/str/str.c:37 (tal_fmt_)
**BROKEN** lightningd:     lightningd/lightningd.c:557 (find_subdaemons_and_plugins)
**BROKEN** lightningd:     lightningd/lightningd.c:1196 (main)
**BROKEN** lightningd:   parents:
**BROKEN** lightningd:     struct plugins
```

Changelog-None: Introduced this release.
Fixes: https://github.com/ElementsProject/lightning/issues/7590
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-20 18:51:39 -07:00
ShahanaFarooqui
203500d6f0 git: Update PR template
Changelog-None.
2024-08-19 16:00:41 -07:00
Rusty Russell
b99fd02393 pytest: add spendable tests for askrene.
Make sure we're not exceeding the spendable amount of a local channel.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-19 10:12:51 -07:00
Rusty Russell
7fb7234da1 askrene: change finalcltv to final_cltv, and return it in response.
You need to know it to make an onion, and in theory if we decided to
fuzz it could be different for different paths.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-19 10:12:51 -07:00
Rusty Russell
96f3637184 doc: note that getroutes return is different than getroute.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-19 10:12:51 -07:00
Lagrang3
62a86cc47f renepay: add daywalker90's test case
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2024-08-19 10:11:35 -07:00
Lagrang3
5400989552 renepay: bugfix: apply channel filter globally
Channel filter must apply to the modified gossmap+localmods,
otherwise we disable local channels with htlcmax=0.

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2024-08-19 10:11:35 -07:00
Lagrang3
4fdc74f100 renepay: take it easy with local channels
Add a little bit of uncertainty to the local channels to avoid
consuming precisely all spendable_msat on our side, which leads to
temporary channel failure if the spendable_msat changes during the
course of the payment.

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2024-08-19 10:11:35 -07:00
Lagrang3
e41625fb29 renepay: bugfix refreshgossmap
Refresh gossmap once before we read the routehints.
Make sure that if channels in the routehints are public,
we retrieve their capacities from gossip.

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2024-08-19 10:11:35 -07:00
Lagrang3
d39b0092ec renepay: bugfix reserved htlc in MCF
Reserved HTLCs were underestimated by floor (mathematical function)
use ceil instead.

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2024-08-19 10:11:35 -07:00
Lagrang3
bb68e65b7a renepay: add precondition check + error msg
- add more checks
- add more error messages
- compute probabilities without fees during MCF
- compute probabilities with fees during get_routes

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2024-08-19 10:11:35 -07:00
Lagrang3
684d6dff50 renepay: patch to assertion htlc_total<=known_max
In theory we should not have htlc_total<=known_max.
But for some strange race condition we do sometimes.
Until we find a solution to ensure the correct state
of the uncertainty network we remove the assertion.
Thanks to signed arithmetic and MIN guards, the rest
of the code in linearize_channel can handle the
weird cases with known_max<htlc_total.

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2024-08-19 10:11:35 -07:00
ShahanaFarooqui
680f81f2aa Adding PR Template 2024-08-19 04:16:42 -07:00
ShahanaFarooqui
52f9678deb meta: Update changelog for 24.08rc2
Release candidate 2 for v24.08
2024-08-15 21:20:35 -07:00