Commit graph

15967 commits

Author SHA1 Message Date
nicolas.dorier
36b3883215 Fix: getblockfrompeer was always requesting block to the first peer
Changelog-Fixed: getblockfrompeer was always requesting block to the first peer
2025-02-12 11:03:14 -06:00
Lagrang3
dfd9e815ba lightning-cli: fix the access to man pages
Changelog-Fixed: lightning-cli: access to man pages from the installed directory.

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-02-12 14:20:56 +10:30
Lagrang3
37150c1f49 renepay: add test_unannounced
Changelog-None.

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-02-12 13:36:06 +10:30
Lagrang3
552bb1c88c renepay: decode error onion
When paying with injectpaymentonion we need to manually decode the error
from the onion.

Changelog-None.

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-02-12 13:36:06 +10:30
Lagrang3
158fb91062 renepay: resolve self payments with fake node
Always use a fake destination node, the self-payments are no longer a
corner case for the routing problem in this way. Also it is ok for
get_routes to return routes with zero length.

Changelog-None.

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-02-12 13:36:06 +10:30
Lagrang3
ef67fe335d renepay: use injectpaymentonion
Use injectpaymentonion for payments with |routes|=0,
ie. self-payments and blinded paths where our node is
the first node in the blinded path.

Changelog-None.

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-02-12 13:36:06 +10:30
Lagrang3
3fbf73cde5 renepay: simplify rpc error callback functions
Changelog-None.

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-02-12 13:36:06 +10:30
Lagrang3
92863e30a5 renepay: more informative renesendpay error reply
Changelog-None.

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-02-12 13:36:06 +10:30
Lagrang3
3ae8b5e969 renepay: don't fail parsing sendpay error
Changelog-None.

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-02-12 13:36:06 +10:30
Lagrang3
74effa31d2 renepay: make self payments with renesendpay
Changelog-None.

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-02-12 13:36:06 +10:30
Lagrang3
a04010c871 renepay: collect shared secrets
Collect the shared secrets when making a payment request.
We would need this if we use injectpaymentonion instead of sendonion.

Changelog-None.

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-02-12 13:36:06 +10:30
Lagrang3
d7fba2d75b renepay: add test for simple offer
Changelog-None.

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-02-12 13:36:06 +10:30
Lagrang3
6c165e6d9c renepay: refactor create_onion
Refactor create_onion function, now we could use the same function to
build onions for sendonion and injectpaymentonion.

Changelog-None.

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-02-12 13:36:06 +10:30
Lagrang3
843c0b7d8c renepay: add function blinded path to json
Changelog-None.

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-02-12 13:36:06 +10:30
Lagrang3
90789287ff renepay: skip test_previous_sendpays
sendonion RPC does not allow to set the total amount in lightningd's
wallet, therefore it mixing sendpay and sendonion payment parts would
not work. That means for the time being we cannot complete a payment
initialized with sendpay until we add a total_amount parameter to
sendonion.

Changelog-None.

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-02-12 13:36:06 +10:30
Lagrang3
47061f15ce renepay: fix error handling
Fix error handling since we moved from sendpay to sendonion rpc.
With sendonion once a route fails we don't get the scid and node_id that
failed along the route, so we have to deduce those from our own internal
data.

Changelog-None

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-02-12 13:36:06 +10:30
Lagrang3
5672c29952 renepay: use our own sendpay rpc
Use renesendpay to send the payment allowing to pay to BOLT12 invoices
from a higher level interface.

Changelog-Add: renepay: Add support for BOLT12 payments

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-02-12 13:36:06 +10:30
Lagrang3
6ab3bbbac5 renepay: change of variable name for clarity
Changing route.amount to route.amount_deliver
for clarity. This variable hold the value that
the route delivers to destination.

Changelog-None

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-02-12 13:36:06 +10:30
Lagrang3
5b7f0eacf4 renepay: add rpc that replaces sendpay
Add an rpc to renepay that is similar to sendpay that
handles BOLT11 and BOLT12 payments.
This is not the most elegant solution but it is a workaround
until we implement it into lightningd which has more development
friction.

Changelog-None.

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-02-12 13:36:06 +10:30
Lagrang3
af66bf2235 renepay: enable routing through blinded paths
Enable routing through blinded paths using fake channels in local
gossmods.

Changelog-None

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-02-12 13:36:06 +10:30
Lagrang3
1311223da5 renepay: parse bolt12 invoices
A first step towards supporting bolt12 invoices and blinded paths.

Changelog-None

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-02-12 13:36:06 +10:30
gudnuf
4a280148f1 single plugin_notification_serialize function 2025-02-12 12:54:10 +10:30
gudnuf
61482e5f45 new notifications: plugin_stopped and plugin_started
[Added version tag into documentation, to show it was added in 25.02 --RR]
Changelog-Added: Plugins: new nofitications `plugin_stopped` and `plugin_started`
2025-02-12 12:54:10 +10:30
Rusty Russell
c3362b057c BOLT12: remove -offers from bolt12 quotes, update them.
Typo fixes and wording changes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-02-11 20:19:01 -06:00
Rusty Russell
c22cc11af5 fetchinvoice: allow user to specify bip353 name corresponding to how they got offer.
Changelog-Added: JSON-RPC: `fetchinvoice` BIP 353 name support (`bip353`).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-02-11 20:19:01 -06:00
Rusty Russell
22a2366901 offers: enforce restrictions in incoming bip353 name fields.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-02-11 20:19:01 -06:00
Rusty Russell
ba3e85bb43 decode: handle new bip353 fields.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-02-11 20:19:01 -06:00
Rusty Russell
3d8238fef9 bolt12: add inv invreq_bip_353_name field to invoice.
All things are supposed to be mirrored from invoice_request into invoice.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-02-11 20:19:01 -06:00
Rusty Russell
7b00e610f6 wire: extract bolt12 by default, now it's merged.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-02-11 20:19:01 -06:00
Rusty Russell
c93ff8e2da wire: update to bolt version which has offers included.
We don't need our patch to add the fields any more.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-02-11 20:19:01 -06:00
Rusty Russell
7d62129dbf wire: update bolts which include sciddir_or_pubkey in blinded path.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-02-11 20:19:01 -06:00
Rusty Russell
77eef3b62e lightningd: update bolts to clarify channel_update timing.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-02-11 20:19:01 -06:00
Rusty Russell
fa188c80ca common: update bolts to include hash value in bolt11 test vectors.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-02-11 20:19:01 -06:00
Rusty Russell
b879cb475f common: update bolt to neaten pubkey descriptions.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-02-11 20:19:01 -06:00
Rusty Russell
f20bea605b wire: fix extracted files.
Changes were made to the wire files manually, but we're supposed to edit the
patches so they can be reproduced.
2025-02-11 20:19:01 -06:00
Alex Myers
c9b01b60a0 unit-tests: cleanup traces after testing
Otherwise it appears to be a leak:

==612637== 11,264 bytes in 1 blocks are still reachable in loss record 1 of 1
==612637==    at 0x484D953: calloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==612637==    by 0x1301F2: trace_init (trace.c:153)
==612637==    by 0x13065D: trace_span_start (trace.c:263)
==612637==    by 0x173968: db_open_ (utils.c:367)
==612637==    by 0x17AE43: create_test_wallet (run-wallet.c:1313)
==612637==    by 0x17C726: test_shachain_crud (run-wallet.c:1548)
==612637==    by 0x18300E: main (run-wallet.c:2329)

Changelog-None
2025-02-11 19:16:16 -06:00
Alex Myers
39753f1fd6 pytest: Address race condition in test_gossip_jsonrpc
In the time it takes connectd to flush the log message for
, gossipd can already have
the announcement sent.

lightningd-1 2025-02-11T15:26:06.745Z DEBUG 022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59-channeld-chan#1: peer_in WIRE_ANNOUNCEMENT_SIGNATURES
lightningd-2 2025-02-11T15:26:06.887Z DEBUG 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-channeld-chan#1: peer_in WIRE_ANNOUNCEMENT_SIGNATURES
lightningd-1 2025-02-11T15:26:06.897Z TRACE gossipd: Received node_announcement for node 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518
lightningd-1 2025-02-11T15:26:06.915Z DEBUG 022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59-connectd: peer_out WIRE_ANNOUNCEMENT_SIGNATURES

Changelog-None
2025-02-11 19:16:16 -06:00
Rusty Russell
1820423cbc common: fix memcpy error in Fischer-Yates shuffle.
Reported by Grubles on ARM64:

```
VALGRIND=1 valgrind -q --error-exitcode=7 --track-origins=yes --leak-check=full --show-reachable=yes --errors-for-leak-kinds=all common/test/run-tal_arr_randomize > /dev/null
==151138== Source and destination overlap in memcpy(0x4d69f08, 0x4d69f08, 8)
==151138==    at 0x48CB68C: __GI_memcpy (vg_replace_strmem.c:1147)
==151138==    by 0x41B50B: tal_arr_randomize_ (pseudorand.c:84)
==151138==    by 0x41BB07: main (run-tal_arr_randomize.c:166)
==151138==
make: *** [Makefile:750: unittest/common/test/run-tal_arr_randomize] Error 7
```

It is correct: you can't overlap src and dst in memcpy.  It *probably* works in
this case, but it's undefined!

Fixes: https://github.com/ElementsProject/lightning/issues/7030
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-02-11 16:54:08 -06:00
Rusty Russell
d554206d7e gossipd: fix bogus message when dying channel is pruned.
```
2025-01-23T12:31:52.528Z DEBUG   gossipd: Pruning channel 839050x1246x0 from network view (ages 1736283379 and 1737600120)
2025-01-27T00:32:01.631Z DEBUG   gossipd: Pruning channel 839050x1246x0 from network view (ages 0 and 1737686520)
2025-01-27T00:50:05.998Z **BROKEN** gossipd: Dying channel 839050x1246x0 already deleted?
```

Easiest not to prune in this case.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-02-11 15:11:47 -06:00
Rusty Russell
2b4b1479ed gossipd: check that gossmap code sees updates from gossip_store writes.
After analyzing various weird cases where we ended up with duplicate
gossip_store entries, it could be explained by us not fully processing
the gossip store.

It's not clear that my assumptions that we would always see our own writes
are true: technically this may require an fsync().  So we now add the
check, and do an fsync and try again.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: gossipd: more sanity checks that we are correctly updating the gossip_store file.
2025-02-11 15:11:47 -06:00
Rusty Russell
8156c83e11 gossipd: check that we are always appending.
We had at least one report of overwriting the gossip_store file at
offset 1.  Make sure this doesn't happen.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-02-11 15:11:47 -06:00
Rusty Russell
769ccaa4c3 gossipd: correctly process dying channels.
Found by inspection.  Minor bug, since we'll catch it on the next block,
but annoying.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-02-11 15:11:47 -06:00
Rusty Russell
1df1300cc9 gossip_store: don't need to check for truncated amounts.
That's actually caught by the gossmap load now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-02-11 15:11:47 -06:00
Rusty Russell
9d98740e18 gossmap: stricter checks when gossipd itself loads the gossip_store.
This means we will correctly reset the store if it has redundant
records, for example.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-02-11 15:11:47 -06:00
Rusty Russell
4f2a7039c6 gossipd: put gossip_store pointer inside gossmap_manage.
It's actually the only one that uses it.  We also tweak the way
gossip_store handles failure: gossmap_manage now tells it when to
reset the corrupted store.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-02-11 15:11:47 -06:00
Rusty Russell
e440799b5e devtools: have dump-gossipstore print malformed warnings to stdout.
If they go to stderr, you can't associate them with the record they're
talking about.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-02-11 15:11:47 -06:00
Rusty Russell
05bc4ca5f3 gossmap: use mmap directly to check checksums.
Instead of making a copy.

To measure the performance impact, I timed
tests/test_askrene.py::test_real_biases on my laptop.

	No checksum check: 194.52s
	Copying for checksum check: 202.81s
	Zero-copy checksum check: 194.40s

But these numbers proved noisy.  Still, doesn't hurt.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-02-11 15:11:47 -06:00
Rusty Russell
4b5e5b27ae gossmap: check checksums.
We assume if it's incorrect, we simply need to wait.  If this proves incorrect,
we will see a stream of BROKEN log messages.

To measure the performance impact, I timed
tests/test_askrene.py::test_real_biases on my laptop.

	Before: 194.52s
	After: 202.81s

So it's marginal.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-02-11 15:11:47 -06:00
Rusty Russell
307dbe3e62 tests: put proper checksums into test gossip_store files.
We're about to test them in gossmap.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-02-11 15:11:47 -06:00
Rusty Russell
5e2f6c5028 gossmap: don't stop reading if we hit a redundant channel_announce.
While this shouldn't happen, it does (pending other fixes), and we stop reading the
gossip store until next time.  The result is partial gossip, demonstrated beautifully
by NicolasDorier's report:

```
lightning_gossipd: gossmap: redundant channel_announce for 864063x1306x1, offsets 1272259 and 1784859!"
```

Gossipd stalld there and don't make more progress.  So gossipd itself
doesn't see the entire gossip_store.

Then things get really batshit:

```
2025-02-04T05:53:28.582Z DEBUG   gossipd: Store compact time: 1429910 msec
```

This took 1429 seconds to process.  Why?

Because it hasn't been processing the gossip store fully, gossipd kept adding "new" records to the end:

```
2025-02-04T05:53:28.583Z DEBUG   gossipd: gossip_store: Read 62716143/1739952/5158256/0 cannounce/cupdate/nannounce/delete from store in 31634458462 bytes, now 31634458440 bytes (populated=true)
```

It has 31GB of gossip in there!  No wonder it took so long...

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes: https://github.com/ElementsProject/lightning/issues/8035
Changelog-Fixed: gossipd: corruption in the gossip_store could cause ever-longer startup times and no gossip updates.
2025-02-11 15:11:47 -06:00