Commit graph

3384 commits

Author SHA1 Message Date
Christian Decker
603a70e7e2 pytest: Test that we remember learnt channel hints across payments 2024-10-07 14:05:47 +02:00
Aditya Sharma
27b4a03419 tests: Add test_getemergencyrecoverdata to test the RPC getemergencyrecoverdata. 2024-10-03 18:59:10 -07:00
Rusty Russell
b8acd3b37c askrene: more code tweaks on feedback from Lagrang3.
1. describe_disabled should point out if node itself is disabled.
2. Hoist constraint check for neater if branching.
3. Use amount_msat_max/min for greater clarity.
4. Simply disable channels, don't zero htlc_min/max when node disabled.

I also fixed the diagnostic of htlc_max correctly, which removes a FIXME.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-10-04 11:27:53 +09:30
Rusty Russell
630ec6a566 askrene: give better feedback when we can't find a suitable route.
This turns out to be critical for users: also stops them from
bothering us when their node is offline or has insufficient capacity!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-10-04 11:27:53 +09:30
Rusty Russell
c307b77d2f askrene: split askrene-create-channel into create-channel and update-channel.
This allows for explicit partial updates to channels (e.g. just change
fees, or just disable) without haveing to set the other fields.

This generalizes askrene-disable-channel, which is removed.

We also take the chance to use the proper BOLT 7 terms in the API:

- htlc_minimum_msat
- htlc_maximum_msat
- cltv_expiry_delta
- fee_base_msat
- fee_proportional_millionths

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-10-04 11:27:53 +09:30
Rusty Russell
154ad585ec pytest: enhance test_getroutes_auto_sourcefree with same case *without* auto.sourcefree.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-10-04 11:27:53 +09:30
Rusty Russell
321ec0875f askrene: rework constraints to exist in pairs.
This is a bit more efficient, but moreover the JSONRPC API is more
logical this way.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-10-04 11:27:53 +09:30
Rusty Russell
3c5c22b17a askrene: change inform interface, take into account reserve.
Lagrang3 points out that if we hit a maximum, we should take into account
the reserve.  This is true, but it's hard for the caller to do, so change
the API to be slightly higher level.

Tell "inform" what happened, and it adjust the constraints appropriately.
This makes the least assumptions possible (a reserve does *not* mean that
the capacity was actually used at that time).

We also add a mode to say "this succeeded": for now this does nothing,
but it could reduce both min/max capacities, and add capacity in the
other direction.  This is useful for future payments, but not as useful
for the current one.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-10-04 11:27:53 +09:30
Rusty Russell
d50838b60f askrene: implement listreservations
And actually write tests!

Suggested-by: Lagrang3 <lagrang3@protonmail.com>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-10-04 11:27:53 +09:30
Rusty Russell
b88f4cb854 askrene: askrene-create-layer and askrene-remove-layer.
It's generally better to be explicit with these things: currently typos
would be ignored.  But it's also much easier to clean up entire layers
as we use them for temporary (per-payment) effects.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-10-04 11:27:53 +09:30
Rusty Russell
29cc227a53 askrene: use short_channel_id_dir in API.
It's generally much more convenient, and it's already present in
other APIs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-10-04 11:27:53 +09:30
Lagrang3
33404b03a0 add askrene-disable-channel
Changelog-EXPERIMENTAL: askrene: add askrene-disable-channel RPC
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2024-10-04 11:27:53 +09:30
Rusty Russell
79fa3d3d7a tests/data: add compressd canned gossip from mainnet.
We can run a lot of tests on this.

Look at me, I'm number 3301!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-10-04 08:47:53 +09:30
Rusty Russell
00e0b99d3a hsmtool: provide nodeid from hsm secret.
This allows tools to validate that it is accessing the correct hsm_secret for this node!

This is extremely important for backups: if they are using VLS, they need to back *that*
up instead, for example.

Changelog-Added: `hsmtool`: `getnodeid` command derives the node id from the hsm_secret, to verify it's the correct secret.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-25 12:07:40 -07:00
Rusty Russell
9b60f6cc6d askrene: re-check min_htlc violations after correcting for MCF rounding.
Thanks to @Lagrang3 for spotting this!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-19 12:16:53 +09:30
Rusty Russell
9d966b46a7 askrene: take into account the reduction in "spendable" with additional HTLCs.
"spendable" is for a single HTLC: if we own the channel, this amount
decreases with every HTLC, as we have to pay fees.  We have access to this since
we call listpeerchannels anyway, so we can calculate the additional costs and
use it in the refining phase.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-19 12:16:53 +09:30
Rusty Russell
3e96955c72 pytest: test askrene treats htlc_maximum_msat and htlc_minimum_msat restrictions properly.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-19 12:16:53 +09:30
Rusty Russell
f0331cd82e askrene: add a "refining" step to add fees and handle corner cases.
This is the root cause of the problem worked around in 50949b7b9c
"askrene: hack in some padding so we don't overflow capacities."

When adding fees to flows, we didn't recheck the boundary conditions: in
renepay this is done by routebuilder.

Fortunately, we can use our "reservations" infrastructure to temporarily
use capacity as we process flows, so we handle the cases where they are
not independent correclty.

My assumption is that the resulting errors are small, so we divide
them between the remaining flows based on highest-to-least
probability.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-19 12:16:53 +09:30
Rusty Russell
1b8a64f5e6 askrene: apply "auto.sourcefree" to channels created in layers, too.
We had a workaround for channels added by "auto.local", but instead we
should make it work properly.

I didn't do this before because we can't manipulate the localmods while
they're applied, but it's simple to do it in two stages.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-19 12:16:53 +09:30
Rusty Russell
06957dc832 pytest: separate out routine which checks only some fields of getroutes.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-19 12:16:53 +09:30
Rusty Russell
e90be8d957 pyln-testing: add gossip_store_file arg to get_node()
This makes it much easier to use generated gossip_store files.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-19 12:16:53 +09:30
Rusty Russell
5ea049ea12 pytest: duplicate test_live_spendable as a canned gossmap test.
Not quite the same, as it doesn't have the "auto.local" layer, but it exhibits
the same problem if we revert the fix for test_live_spendable.

And it's much faster!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-19 12:16:53 +09:30
Rusty Russell
b11642538e pytest: ensure there are no duplicate paths in test_live_spendable, and check total.
There aren't, but make sure.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-19 12:16:53 +09:30
Rusty Russell
6150d1b3fa patch general-node-id-assign.patch 2024-09-19 12:16:53 +09:30
ShahanaFarooqui
3d3e86e22c test: CI error fixes
- Removes CI value error for Broken logs
- Fixes CI errors due to deprecated listconfigs 'important-plugins'
- Removed listchannels deprecated local test

Changelog-None.
2024-09-18 16:59:27 +09:30
Michael Schmoock
02176f7004 pytests: extend the offline mode testcase 2024-09-17 19:53:25 +02:00
Rusty Russell
f0cf088489 lightningd: don't crash if plugin dies during initial plugins_init.
Thanks to Michael Schmook for the excellent bug report.

Fixes: https://github.com/ElementsProject/lightning/issues/7671
Changelog-Fixed: lightningd: no longer crash if a plugin dies during lightningd startup.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-17 10:24:01 -07:00
Rusty Russell
f4f6279b17 pytest: thoroughly test plugin death during startup.
We modify broken.py to be able to fail at different points,
and test that during startup.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-17 10:24:01 -07:00
Rusty Russell
bfe0557f0b topology: fix overzealous deprecation check in listpeerchannels.
It's an output field (which we don't complain about), not an input field!

Fixes: https://github.com/ElementsProject/lightning/issues/7652
Changelog-Fixed: Logging: removed bogus "**BROKEN** plugin-topology: DEPRECATED API USED: listchannels.include_private" message.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-16 16:03:48 +09:30
Rusty Russell
7e2ea3ed04 pytest: add test for bogus deprecation BROKEN log.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-09-16 16:03:48 +09:30
evansmj
873e8b4ef8 docs: clarify writing-json-schemas.md and autogenerate-rpc-examples.py
Spelling and clarity on how to update schemas when adding RPCs.
2024-09-05 17:01:04 -07:00
ShahanaFarooqui
84b3e80d16 test: testplugpass requirements.txt updated to install local pyln-client
Python was installing `pyln-client` directly from the server for testplugpass plugin. This commit is updating the requirements.txt file to install pyln-client with absolute local path.
2024-09-03 23:24:12 -07:00
Rusty Russell
9d88ce3b59 pytest: suppress failing renepay test for Liquid.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-23 18:52:15 +09:30
Rusty Russell
50949b7b9c askrene: hack in some padding so we don't overflow capacities.
Of course, we still will, since spendable is for a single HTLC, but
this also shows why we should treat *minimum* as the incorrect answer
if they cross, too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes: https://github.com/ElementsProject/lightning/issues/7563
2024-08-23 18:52:15 +09:30
Rusty Russell
af011f9dac pytest: make spendable test for askrene work properly, give a better name.
Based on great test case from https://github.com/daywalker90

```
E       AssertionError: assert {'107x2x0/1': 'Path total 285720859 > spendable 285718000', '108x1x0/1': 'Path total 384721849 > spendable 384718000'} == {}
E         Left contains 2 more items:
E         {'107x2x0/1': 'Path total 285720859 > spendable 285718000',
E          '108x1x0/1': 'Path total 384721849 > spendable 384718000'}
E         Full diff:
E           {
E         -  ,
E         +  '107x2x0/1': 'Path total 285720859 > spendable 285718000',
E         +  '108x1x0/1': 'Path total 384721849 > spendable 384718000',
E           }
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-23 18:52:15 +09:30
Rusty Russell
b99fd02393 pytest: add spendable tests for askrene.
Make sure we're not exceeding the spendable amount of a local channel.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-19 10:12:51 -07:00
Rusty Russell
7fb7234da1 askrene: change finalcltv to final_cltv, and return it in response.
You need to know it to make an onion, and in theory if we decided to
fuzz it could be different for different paths.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-19 10:12:51 -07:00
Lagrang3
62a86cc47f renepay: add daywalker90's test case
Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2024-08-19 10:11:35 -07:00
Rusty Russell
054eeb8aa2 plugins/sql: fix crash when subobject is missing which has subobject inside it.
There's only one place (added in v24.02) where we have nested subobjects, and
if the outer sub-object is not there it crashes.

```
sql: FATAL SIGNAL 11 (version v24.08rc1-2-ge134f2f-modded)
0x5781d447239b send_backtrace
        common/daemon.c:33
0x5781d4472432 crashdump
        common/daemon.c:75
0x787bcd64531f ???
        ./signal/../sysdeps/unix/sysv/linux/x86_64/libc_sigaction.c:0
0x5781d447782a json_get_membern
        common/json_parse_simple.c:210
0x5781d44778cc json_get_member
        common/json_parse_simple.c:223
0x5781d445d0e2 process_json_obj
        plugins/sql.c:537
0x5781d445d10d process_json_obj
        plugins/sql.c:538
0x5781d445d598 process_json_list
        plugins/sql.c:684
0x5781d445d65a process_json_result
        plugins/sql.c:699
0x5781d445d7cb default_list_done
        plugins/sql.c:722
0x5781d446349d handle_rpc_reply
        plugins/libplugin.c:1016
0x5781d4463640 rpc_read_response_one
        plugins/libplugin.c:1202
0x5781d44636f1 rpc_conn_read_response
        plugins/libplugin.c:1226
0x5781d459e56c next_plan
        ccan/ccan/io/io.c:60
0x5781d459ea3d do_plan
        ccan/ccan/io/io.c:422
0x5781d459eafa io_ready
        ccan/ccan/io/io.c:439
0x5781d45a0469 io_loop
        ccan/ccan/io/poll.c:455
0x5781d4463dcb plugin_main
        plugins/libplugin.c:2230
```

Changelog-Fixed: Plugins: `sql` crash on querying `listpeerchannels` during channel establishment.
Fixes: https://github.com/ElementsProject/lightning/issues/7505
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-16 07:19:52 +09:30
Rusty Russell
a6c2f183d6 bookkeeper: fix up out-of-order migrations in rc1
This was introduced in rc1, resulting in:

```
2024-08-15T01:33:48.343Z **BROKEN** plugin-bookkeeper: query failed: plugins/bkpr/recorder.c:738: SELECT  e.id, e.account_id, a.name, e.origin, e.tag, e.credit, e.debit, e.output_value, e.currency, e.timestamp, e.blockheight, e.utxo_txid, e.outnum, e.spending_txid, e.payment_id, e.ignored, e.stealable, e.ev_desc, e.spliced FROM chain_events e LEFT OUTER JOIN accounts a ON e.account_id = a.id WHERE  e.spending_txid = ? AND e.account_id = ? AND e.utxo_txid = ? AND e.outnum = ?
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-16 07:18:29 +09:30
Rusty Russell
e2ec60a369 pytest: handle more expected broken messages.
Once we fix our broken log detection, we find some we were missing.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-15 16:38:12 +09:30
daywalker90
7ffd0a3936 cln-plugin: add multi options for String and i64
Changelog-Added: cln-plugin: add multi options for String and i64
2024-08-13 12:24:45 +09:30
Rusty Russell
48259afb70 lightningd: always broadcast our own gossip when it changes.
When a peer connects, we always send all our own gossip (even if they
had set the timestamp filters to filter it out).  But we weren't
forcing it out to them when it changed, so this logic only applied to
unstable or frequently-restarting nodes.

So now, we tell all the peers whenever we tell gossipd about our new
gossip.

Fixes: #7276
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: We now send current peers our changed gossip (even if they set timestamp_filter otherwise), not just on reconnect.
2024-08-12 16:30:29 +09:30
Rusty Russell
4886d228e7 pytest: test that we inform all peers about gossip changes.
In particular, those who've filtered it out.

This currently fails, as expected.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-12 16:30:29 +09:30
Rusty Russell
2d129e79ab pytest: enhance test_gossip_pruning
We didn't actually check that we *send* the refreshed gossip, just
that we print the message saying we're going to.

So check that everyone received updated gossip when this happens.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-12 16:30:29 +09:30
Rusty Russell
975dd76086 lightningd: create new return code for enableoffer.
Suggested-by: https://github.com/Lagrang3
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-08-11 11:27:10 +09:30
Vincenzo Palazzo
1e1edfd073 offer: allow re-enabling a previously disabled offer
Sometimes, for various reasons, a user disables an offer
and then wants to re-enable it. This should be allowed because,
from the CLN point of view, it is just an internal state.

If a user has constraints on the description of the invoice
because they are using services that link some sort of user ID
to an offer, it is important for the user to be able to re-enable the
offer, not create a new one. Creating a new offer would
require a different description.

Link: https://github.com/ElementsProject/lightning/issues/7360
Co-Developed-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2024-08-11 11:27:10 +09:30
ShahanaFarooqui
50faa7c122 script: Auto generate rpc examples for documentation
I will commit doc/schemas/lightning-*.json files separately to keep this commit easier to review.

- test-autogenerate-rpc-examples updates all example request & responses in doc/schemas/lightning-*.json files.

- Updated tools/fromschema.py to accommodate the sql JSON example requirement where it does not accept -o in the query but shell does (for queries containing the = sign).

Changelog-None.
2024-08-09 23:56:45 -07:00
ShahanaFarooqui
9c0f073a44 tests: Removed non-essential hsm copy step 2024-08-09 23:56:45 -07:00
Max Rantil
3e65ef4b12 lightningd: trim whitespaces from end of config parameters
Signed-off-by: Max Rantil <rantil@pm.me>
2024-08-10 15:20:59 +09:30