Commit graph

15973 commits

Author SHA1 Message Date
Rusty Russell
645b202222 wallet: use htable in wallet_add_onchaind_utxo.
On lookup, we update the htable if any new addresses have been added.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: lightningd: startup time vastly improved for large nodes with pending closes and many bitcoin addresses.
2025-02-13 15:42:51 -06:00
Rusty Russell
44ff4fe5fa wallet: create htable to store scriptpubkeys for issued indexes.
For now we don't actually update it; that's next patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-02-13 15:42:51 -06:00
Rusty Russell
d7ad8af27c wallet: wallet_get_addrtype should tell caller it has no info.
Turns out we don't actually use this, so easy to change the function.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-02-13 15:42:51 -06:00
Rusty Russell
ca785bac06 wallet: re-add ADDR_P2SH_SEGWIT type to enum addrtype
It was long obsoleted, and never appears in the DB, but we do still have
to handle old ones in the code.

We removed it from the enum in
f342630b92 (v24.02) after deprecating it
in 23.02.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-02-13 15:42:51 -06:00
Rusty Russell
c13756d2c5 pytest: test for slow start when closing tx and large number of addresses.
From Whitslack's "startup takes 15 minutes" bug report, we can see
that wallet_can_spend is extremely slow.  We exacerbate this by setting a large
bip32_max_index:

```
    91.29%     0.02%  lightningd       lightningd                 [.] wallet_can_spend
            |          
             --91.27%--wallet_can_spend
                       |          
                       |--47.81%--scriptpubkey_p2tr_derkey
                       |          |          
                       |          |--42.80%--scriptpubkey_p2tr
...
                       |--42.16%--bip32_key_from_parent
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-02-13 15:42:51 -06:00
ShahanaFarooqui
6bcc05ac34 pyln-testing: Fix for disabled schema check
In a previous [commit](https://github.com/ElementsProject/lightning/pull/8065/commits), I mistakenly removed the `lightning-` replacement logic from the base name also. This commit restores that functionality to re-enable schema checks.

Changelog-None.
2025-02-13 15:58:22 +10:30
nicolas.dorier
36b3883215 Fix: getblockfrompeer was always requesting block to the first peer
Changelog-Fixed: getblockfrompeer was always requesting block to the first peer
2025-02-12 11:03:14 -06:00
Lagrang3
dfd9e815ba lightning-cli: fix the access to man pages
Changelog-Fixed: lightning-cli: access to man pages from the installed directory.

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-02-12 14:20:56 +10:30
Lagrang3
37150c1f49 renepay: add test_unannounced
Changelog-None.

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-02-12 13:36:06 +10:30
Lagrang3
552bb1c88c renepay: decode error onion
When paying with injectpaymentonion we need to manually decode the error
from the onion.

Changelog-None.

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-02-12 13:36:06 +10:30
Lagrang3
158fb91062 renepay: resolve self payments with fake node
Always use a fake destination node, the self-payments are no longer a
corner case for the routing problem in this way. Also it is ok for
get_routes to return routes with zero length.

Changelog-None.

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-02-12 13:36:06 +10:30
Lagrang3
ef67fe335d renepay: use injectpaymentonion
Use injectpaymentonion for payments with |routes|=0,
ie. self-payments and blinded paths where our node is
the first node in the blinded path.

Changelog-None.

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-02-12 13:36:06 +10:30
Lagrang3
3fbf73cde5 renepay: simplify rpc error callback functions
Changelog-None.

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-02-12 13:36:06 +10:30
Lagrang3
92863e30a5 renepay: more informative renesendpay error reply
Changelog-None.

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-02-12 13:36:06 +10:30
Lagrang3
3ae8b5e969 renepay: don't fail parsing sendpay error
Changelog-None.

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-02-12 13:36:06 +10:30
Lagrang3
74effa31d2 renepay: make self payments with renesendpay
Changelog-None.

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-02-12 13:36:06 +10:30
Lagrang3
a04010c871 renepay: collect shared secrets
Collect the shared secrets when making a payment request.
We would need this if we use injectpaymentonion instead of sendonion.

Changelog-None.

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-02-12 13:36:06 +10:30
Lagrang3
d7fba2d75b renepay: add test for simple offer
Changelog-None.

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-02-12 13:36:06 +10:30
Lagrang3
6c165e6d9c renepay: refactor create_onion
Refactor create_onion function, now we could use the same function to
build onions for sendonion and injectpaymentonion.

Changelog-None.

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-02-12 13:36:06 +10:30
Lagrang3
843c0b7d8c renepay: add function blinded path to json
Changelog-None.

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-02-12 13:36:06 +10:30
Lagrang3
90789287ff renepay: skip test_previous_sendpays
sendonion RPC does not allow to set the total amount in lightningd's
wallet, therefore it mixing sendpay and sendonion payment parts would
not work. That means for the time being we cannot complete a payment
initialized with sendpay until we add a total_amount parameter to
sendonion.

Changelog-None.

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-02-12 13:36:06 +10:30
Lagrang3
47061f15ce renepay: fix error handling
Fix error handling since we moved from sendpay to sendonion rpc.
With sendonion once a route fails we don't get the scid and node_id that
failed along the route, so we have to deduce those from our own internal
data.

Changelog-None

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-02-12 13:36:06 +10:30
Lagrang3
5672c29952 renepay: use our own sendpay rpc
Use renesendpay to send the payment allowing to pay to BOLT12 invoices
from a higher level interface.

Changelog-Add: renepay: Add support for BOLT12 payments

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-02-12 13:36:06 +10:30
Lagrang3
6ab3bbbac5 renepay: change of variable name for clarity
Changing route.amount to route.amount_deliver
for clarity. This variable hold the value that
the route delivers to destination.

Changelog-None

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-02-12 13:36:06 +10:30
Lagrang3
5b7f0eacf4 renepay: add rpc that replaces sendpay
Add an rpc to renepay that is similar to sendpay that
handles BOLT11 and BOLT12 payments.
This is not the most elegant solution but it is a workaround
until we implement it into lightningd which has more development
friction.

Changelog-None.

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-02-12 13:36:06 +10:30
Lagrang3
af66bf2235 renepay: enable routing through blinded paths
Enable routing through blinded paths using fake channels in local
gossmods.

Changelog-None

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-02-12 13:36:06 +10:30
Lagrang3
1311223da5 renepay: parse bolt12 invoices
A first step towards supporting bolt12 invoices and blinded paths.

Changelog-None

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2025-02-12 13:36:06 +10:30
gudnuf
4a280148f1 single plugin_notification_serialize function 2025-02-12 12:54:10 +10:30
gudnuf
61482e5f45 new notifications: plugin_stopped and plugin_started
[Added version tag into documentation, to show it was added in 25.02 --RR]
Changelog-Added: Plugins: new nofitications `plugin_stopped` and `plugin_started`
2025-02-12 12:54:10 +10:30
Rusty Russell
c3362b057c BOLT12: remove -offers from bolt12 quotes, update them.
Typo fixes and wording changes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-02-11 20:19:01 -06:00
Rusty Russell
c22cc11af5 fetchinvoice: allow user to specify bip353 name corresponding to how they got offer.
Changelog-Added: JSON-RPC: `fetchinvoice` BIP 353 name support (`bip353`).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-02-11 20:19:01 -06:00
Rusty Russell
22a2366901 offers: enforce restrictions in incoming bip353 name fields.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-02-11 20:19:01 -06:00
Rusty Russell
ba3e85bb43 decode: handle new bip353 fields.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-02-11 20:19:01 -06:00
Rusty Russell
3d8238fef9 bolt12: add inv invreq_bip_353_name field to invoice.
All things are supposed to be mirrored from invoice_request into invoice.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-02-11 20:19:01 -06:00
Rusty Russell
7b00e610f6 wire: extract bolt12 by default, now it's merged.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-02-11 20:19:01 -06:00
Rusty Russell
c93ff8e2da wire: update to bolt version which has offers included.
We don't need our patch to add the fields any more.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-02-11 20:19:01 -06:00
Rusty Russell
7d62129dbf wire: update bolts which include sciddir_or_pubkey in blinded path.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-02-11 20:19:01 -06:00
Rusty Russell
77eef3b62e lightningd: update bolts to clarify channel_update timing.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-02-11 20:19:01 -06:00
Rusty Russell
fa188c80ca common: update bolts to include hash value in bolt11 test vectors.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-02-11 20:19:01 -06:00
Rusty Russell
b879cb475f common: update bolt to neaten pubkey descriptions.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-02-11 20:19:01 -06:00
Rusty Russell
f20bea605b wire: fix extracted files.
Changes were made to the wire files manually, but we're supposed to edit the
patches so they can be reproduced.
2025-02-11 20:19:01 -06:00
Alex Myers
c9b01b60a0 unit-tests: cleanup traces after testing
Otherwise it appears to be a leak:

==612637== 11,264 bytes in 1 blocks are still reachable in loss record 1 of 1
==612637==    at 0x484D953: calloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==612637==    by 0x1301F2: trace_init (trace.c:153)
==612637==    by 0x13065D: trace_span_start (trace.c:263)
==612637==    by 0x173968: db_open_ (utils.c:367)
==612637==    by 0x17AE43: create_test_wallet (run-wallet.c:1313)
==612637==    by 0x17C726: test_shachain_crud (run-wallet.c:1548)
==612637==    by 0x18300E: main (run-wallet.c:2329)

Changelog-None
2025-02-11 19:16:16 -06:00
Alex Myers
39753f1fd6 pytest: Address race condition in test_gossip_jsonrpc
In the time it takes connectd to flush the log message for
, gossipd can already have
the announcement sent.

lightningd-1 2025-02-11T15:26:06.745Z DEBUG 022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59-channeld-chan#1: peer_in WIRE_ANNOUNCEMENT_SIGNATURES
lightningd-2 2025-02-11T15:26:06.887Z DEBUG 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-channeld-chan#1: peer_in WIRE_ANNOUNCEMENT_SIGNATURES
lightningd-1 2025-02-11T15:26:06.897Z TRACE gossipd: Received node_announcement for node 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518
lightningd-1 2025-02-11T15:26:06.915Z DEBUG 022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59-connectd: peer_out WIRE_ANNOUNCEMENT_SIGNATURES

Changelog-None
2025-02-11 19:16:16 -06:00
Rusty Russell
1820423cbc common: fix memcpy error in Fischer-Yates shuffle.
Reported by Grubles on ARM64:

```
VALGRIND=1 valgrind -q --error-exitcode=7 --track-origins=yes --leak-check=full --show-reachable=yes --errors-for-leak-kinds=all common/test/run-tal_arr_randomize > /dev/null
==151138== Source and destination overlap in memcpy(0x4d69f08, 0x4d69f08, 8)
==151138==    at 0x48CB68C: __GI_memcpy (vg_replace_strmem.c:1147)
==151138==    by 0x41B50B: tal_arr_randomize_ (pseudorand.c:84)
==151138==    by 0x41BB07: main (run-tal_arr_randomize.c:166)
==151138==
make: *** [Makefile:750: unittest/common/test/run-tal_arr_randomize] Error 7
```

It is correct: you can't overlap src and dst in memcpy.  It *probably* works in
this case, but it's undefined!

Fixes: https://github.com/ElementsProject/lightning/issues/7030
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-02-11 16:54:08 -06:00
Rusty Russell
d554206d7e gossipd: fix bogus message when dying channel is pruned.
```
2025-01-23T12:31:52.528Z DEBUG   gossipd: Pruning channel 839050x1246x0 from network view (ages 1736283379 and 1737600120)
2025-01-27T00:32:01.631Z DEBUG   gossipd: Pruning channel 839050x1246x0 from network view (ages 0 and 1737686520)
2025-01-27T00:50:05.998Z **BROKEN** gossipd: Dying channel 839050x1246x0 already deleted?
```

Easiest not to prune in this case.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-02-11 15:11:47 -06:00
Rusty Russell
2b4b1479ed gossipd: check that gossmap code sees updates from gossip_store writes.
After analyzing various weird cases where we ended up with duplicate
gossip_store entries, it could be explained by us not fully processing
the gossip store.

It's not clear that my assumptions that we would always see our own writes
are true: technically this may require an fsync().  So we now add the
check, and do an fsync and try again.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: gossipd: more sanity checks that we are correctly updating the gossip_store file.
2025-02-11 15:11:47 -06:00
Rusty Russell
8156c83e11 gossipd: check that we are always appending.
We had at least one report of overwriting the gossip_store file at
offset 1.  Make sure this doesn't happen.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-02-11 15:11:47 -06:00
Rusty Russell
769ccaa4c3 gossipd: correctly process dying channels.
Found by inspection.  Minor bug, since we'll catch it on the next block,
but annoying.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-02-11 15:11:47 -06:00
Rusty Russell
1df1300cc9 gossip_store: don't need to check for truncated amounts.
That's actually caught by the gossmap load now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-02-11 15:11:47 -06:00
Rusty Russell
9d98740e18 gossmap: stricter checks when gossipd itself loads the gossip_store.
This means we will correctly reset the store if it has redundant
records, for example.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2025-02-11 15:11:47 -06:00