Commit Graph

2936 Commits

Author SHA1 Message Date
Rusty Russell
bfa81f30e1 pytest: ignore private updates in test_addgossip.
This was triggering on private updates after gossip changes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

1diff --git a/tests/test_gossip.py b/tests/test_gossip.py
index 677ec4825..285290b71 100644
2024-01-31 09:15:46 +10:30
Rusty Russell
0a8003a4d2 pytest: fix psbt db test.
When we rewrite, we actually use the remote channel_announcement signatures from the db, and notice that l1 here is actually l2:

```
0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-chan#1: gossipd rejected our channel announcement: WARNING: Bad node_signature_1 304402204ad9f0ca9b5f1f8eec61290b6008bf2c15f011e35543d25c35c5b89d2e1a183402201aaca5883714dbe11d6ae00a146fac7302352e1b9e987eee5ce98e8eb6a82909 hash e713698e14d3d8f973210e8d93c96d1a7c117c58340ccb195785a672fd0b5ef1 on channel_announcement 01004ad9f0ca9b5f1f8eec61290b6008bf2c15f011e35543d25c35c5b89d2e1a18341aaca5883714dbe11d6ae00a146fac7302352e1b9e987eee5ce98e8eb6a829095c05509264fcf72e19180f0285cf0ca0d786f280a75d79e46e5627be20c9dd9a4e9105aa61d7faf3a375454db94376b9b9df29cf078130ed827836b0849f119014ec750ab163baef701e7889f430ee37ea7e38857aa2b2745440600999c85f2903abefde5210dd06ae12f24b8801416800db84f5c73d35594af594f9001800025dee15519d29eb81e9d1ac761deebdad91ba1d08a05c027c16ea5fc08f3a9b663f8433209392395f0107f6b4d266c46b0946bd098f88c0693817fb560e0526a8000006226e46111a0b59caaf126043eb5bbf28c34f3a5e332a1fc7b2b73cf188910f00006900000100000266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c035180266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c0351802324266de8403b3ab157a09f1f784d587af61831c998c151bcc21bb74c2b2314b0206540493a98599967a61c45172df49681e398a6cb0890a506e74f76c2d2be242
```

Explicitly remove those messages.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-31 09:15:46 +10:30
Christian Decker
948ee4dbc5 pyln: Consolidate on pyln.testing.db
We had a complete copy of the `db_provider` and  associated classes in
teh `tests/` directory, causing them to drift apart. Consolidating on
one version makes this more maintainable.

Changelog-None
2024-01-29 19:13:12 +01:00
Tushar Vyas
464dc02b94 wipe_db() implementation 2024-01-29 19:13:12 +01:00
Rusty Russell
af4b244f2f pytest: fix flake in test_peer_anchor_push
If l1 is one block behind, l2 goes onchain one block early:

```
        # Drops to chain
>       wait_for(lambda: only_one(l2.rpc.listpeerchannels(l3.info['id'])['channels'])['state'] == 'AWAITING_UNILATERAL')

tests/test_closing.py:3951:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

success = <function test_peer_anchor_push.<locals>.<lambda> at 0x7f799f9cd1f0>
timeout = 180

    def wait_for(success, timeout=TIMEOUT):
        start_time = time.time()
        interval = 0.25
        while not success():
            time_left = start_time + timeout - time.time()
            if time_left <= 0:
>               raise ValueError("Timeout while waiting for {}".format(success))
E               ValueError: Timeout while waiting for <function test_peer_anchor_push.<locals>.<lambda> at 0x7f799f9cd1f0>
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-29 16:47:33 +10:30
Rusty Russell
bf9cdc597e pytest: fix flake in test_custommsg_triggers_notification
If listpeers on l2 is called too fast, then it won't return any peers, and
we will fail:

```
        # Connect l1 to l2
        l1.connect(l2)
>       wait_for(lambda: l2.rpc.listpeers(l1.info['id'])['peers'][0]['connected'])

tests/test_misc.py:2690:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
contrib/pyln-testing/pyln/testing/utils.py:88: in wait_for
    while not success():
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

>   wait_for(lambda: l2.rpc.listpeers(l1.info['id'])['peers'][0]['connected'])
E   IndexError: list index out of range
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-29 16:47:33 +10:30
Rusty Russell
6778f320d2 openingd: make channel-type rejection message clearer.
For example, lnprototest got the error 'You gave bad parameters: Did not support channel_type ' which doesn't make it clear that it's rejecting the empty channel type.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-29 13:40:34 +10:30
Rusty Russell
b805899b06 pytest: check that listpeerchannels gives the same channel type as returned from fund/openchannel
Could have done this in an earlier commit, but that would be a messy rebase.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-29 13:40:34 +10:30
Rusty Russell
17d0d006d2 fundchannel_start & multifundchannel: add channel_type.
Let's tell the caller what channel_type they got!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `fundchannel`, `multifundchannel`, `fundchannel_start` and `openchannel_init`: new field `channel_type`.
2024-01-29 13:40:34 +10:30
Rusty Russell
08e29ab47f lightningd: make option_static_remotekey compulsory.
As suggested in https://github.com/lightning/bolts/pull/1092.

We still support channels opened without it, but you can no longer open new ones without it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: `option_gossip_queries` is now required (advertized by all but 16 nodes)
2024-01-29 13:40:34 +10:30
Rusty Russell
e531ff83b3 lightningd: add dev-any-channel-type to allow obsolete channel types.
We're about to make static_remotekey compulsory, but we still want to
do tests for pre-existing channels.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-29 13:40:34 +10:30
Rusty Russell
3c2a57598f spenderp: add channel_type parameter to fundchannel / multifundchannel.
And add a request schema for multifundchannel.

Changelog-Added: JSON-RPC: `fundchannel` and `multifundchannel` now take an optional `channel_type` parameter.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-29 13:40:34 +10:30
Rusty Russell
e749aebbff fundchannel_start / openchannel_init: add a channel_type parameter to force channel type.
And add request schemas for openchannel_init and fundchannel_start.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `fundchannel_start` and `openchannel_init` now take an optional `channel_type` parameter.
2024-01-29 13:40:34 +10:30
Rusty Russell
f450dfeb55 lightningd: make option_gossip_queries compulsory.
As suggested in https://github.com/lightning/bolts/pull/1092.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: `option_gossip_queries` is now required (advertized by all but 11 nodes)
2024-01-29 13:40:34 +10:30
Rusty Russell
88a2146664 lightningd: make option_data_loss_protect compulsory.
As suggested in https://github.com/lightning/bolts/pull/1092.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: `option_data_loss_protect` is now required (advertized by all but 11 nodes)
2024-01-29 13:40:34 +10:30
Lagrang3
b0054aad46 renepay: accomodate fees in the payment flow
Min. Cost Flow does not take into account fees when computing a flow
    with liquidity constraints.
    This is a work-around solution that reduces the amount on every route to
    respect the liquidity bound. The deficity in the delivered amount is
    solved by running MCF once again.

    Changes:

    1. the function `flow_complete` allocates amounts to send over the set of routes
       computed by the MCF algorithm, but it does not allocate more than liquidity
       bound of the route. For this reason `minflow` returns a set of routes that
       satisfy the liquidity bounds but it is not guaranteed that the total payment
       reaches the destination therefore there could a deficit in the delivery:
       `deficit = amount_to_deliver - delivering`.

    2. in the function `add_payflows` after `minflow` returns a set of routes we
       call `flows_fit_amount` that tries to a allocate the `deficit` in the routes
       that the MCF have computed.

    3. if the resulting flows pass all payment constraints then we update
        `amount_to_deliver = amount_to_deliver - delivering`, and the loop
        repeats as long as `amount_to_deliver` is not zero.

    In other words, the excess amount, beyond the liquidity bound,
    in the routes is removed and then we try to allocate it
    into known routes, otherwise we do a whole MCF again just for the
    remaining amount.

    Fixes issue #6599
2024-01-29 10:48:24 +10:30
niftynei
a4f92eac81 bkpr: now that we're not doing empty acct logging, we dont need this bool
We stopped doing empty journal logs, so we no longer need to switch
our log severity based on whether or not an account exists.

Should make bookkeeper less chatty and remove noisy logs

Changelog-None
2024-01-29 10:05:03 +10:30
niftynei
5484aaee33 bkpr: dont log unknown accounts with zero balances
We were putting out a lot of empty journal entries. Let's
stop doing that.

Now the wallet balance stays uninitialized until/unless you have
funds in it.

Fixes #5672
2024-01-29 10:05:03 +10:30
Rusty Russell
277ed2ccbf libplugin: support version strings for deprecations.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-26 10:30:22 +10:30
Rusty Russell
4764ec794c pytest: note where we use very deprecated features, so tests don't rbeak.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-26 10:30:22 +10:30
Rusty Russell
5ef4779edc lightningd: remove msatoshi alias for amount_msat.
Changelog-Removed: JSON-RPC: `invoice`, `sendonion`, `sendpay`, `pay`, `keysend`, `fetchinvoice`, `sendinvoice`: `msatoshi` argument (deprecated 0.12.0). Use `amount_msat`.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-26 10:30:22 +10:30
Lagrang3
7c7d13d6dc renepay: add self-pay feature 2024-01-24 14:42:45 +10:30
Matt Morehouse
3260d7dd5e fuzz: initial fuzz-bolt12-invoice-decode corpus 2024-01-23 20:21:48 +10:30
Matt Morehouse
1c18269cd1 fuzz: initial fuzz-bolt12-invrequest-decode corpus 2024-01-23 20:21:48 +10:30
Matt Morehouse
2073d9771d fuzz: initial fuzz-bolt12-offer-decode corpus 2024-01-23 20:21:48 +10:30
Matt Morehouse
3af61e087c fuzz: initial fuzz-bolt12-bech32-decode corpus 2024-01-23 20:21:48 +10:30
Matt Morehouse
7d05f07a2b fuzz: test bolt12 decoding
Add fuzz targets for decoding of offers, invoice requests, and invoices.
Mutators are shared in bolt12.h.
2024-01-23 20:21:48 +10:30
Matt Morehouse
b73f921a91 fuzz: test bolt12 bech32 decoding 2024-01-23 20:21:48 +10:30
Rusty Russell
09c1cfde39 pyln-client: don't automatically turn JSON into Millisatoshi class.
Now _msat fields are all integers (last conversion 23.08) we can simply
leave them alone, rather than trying to convert them.

And for turning Millisatoshi into JSON, we simply globally replace the
default encoding function to try ".to_json()" on items, which allows
anything to be marshalled.

The global replacement was interfering with other uses of JSON, such
as the clnrest plugin.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: pyln-client: no longer autoconverts _msat field to Millisatoshi class (leaves as ints).
2023-12-16 10:22:01 +01:00
Rusty Russell
8b224b6ec2 pytest: don't assume that msat fields get turned into Millisatoshi.
Since the class interacts with int very well now, we don't have to
make explicit accesses, so it's easy to write code which works with
Millisatoshi or int.

Also, don't access rpc.decoder in one test, it's unnecessary.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-12-16 10:22:01 +01:00
Erik De Smedt
375985bf6c Implement tests for notifications on custommsg 2023-12-16 11:36:42 +10:30
Dusty Daemon
6f4a3c86e9 splice: Agressive restart testing during splices
Test node restarts in lots of phases of splice.

Changelog-None
2023-12-15 09:42:45 +10:30
Erik De Smedt
557b5fee46 Allow dynamic option in python plugin 2023-12-14 16:48:02 +01:00
Rusty Russell
36b631699b plugins: re-enable listchannels local info in deprecated mode.
We only deprecated this, we didn't actually remove it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-12-14 09:16:56 +10:30
Rusty Russell
58a1c4c012 topology: don't show private nodes in listchannels.
This alters a few remaining tests, as well.

[ Inclused even more test fixes from Alex Myers <alex@endothermic.dev>! ]

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: JSON-RPC: `listchannels` listing private channels: use listpeerchannels
2023-12-14 09:16:56 +10:30
Alex Myers
3a6b9f2ec6 pytest: skip test_hook_crash on liquid CI 2023-12-14 09:16:56 +10:30
Rusty Russell
b318811a2d pytest: fix tests in assumption that listchannels will no longer show private gossip.
Some can only be changed once that is true, but some can be removed/amended already.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-12-14 09:16:56 +10:30
Rusty Russell
ea0d00e677 topology: don't show private nodes in listnodes.
This breaks our tests a bit, which assumed we can always see ourselves
even if we don't have a proper channel.

Usually we would deprecate this first, but it's unlikely to break
anyone since it's a bit obscure that this worked at all.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: `listnodes` no longer shows private (peer) nodes: use listpeers
2023-12-14 09:16:56 +10:30
Rusty Russell
f61ba7b332 pytest: remove test waits for routehints.
Now listincoming sees local channels in listpeerchannels, we don't need these waits.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-12-14 09:16:56 +10:30
Rusty Russell
f3b4821d36 pytest: fix test_zeroconf_open()
l2 gave us a routehint, but it should have seen l1 as a dead-end.  It
didn't due to the presence of a channel alias, which was a bug.  We're
about to fix this, which breaks the test.

Add a dummy node off l1.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-12-14 09:16:56 +10:30
Rusty Russell
c3f33eb6dd listpeerchannels: show gossip updates.
This is redundant if it's a public channel, but vital if it's not.  Publishing unconditionally makes
it easier for gossmap: we create a local modification all the time, even if redundant (and we can
have the actual capacity ceiling accurate in this case, since we know it for local channels).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Changelog-Added: JSON-RPC: `listpeerchannels` now shows gossip update contents (even if channel unannounced).
2023-12-14 09:16:56 +10:30
Alex Myers
a8995caa8b pytest: use wait_local_channel_active 2023-12-14 09:16:56 +10:30
Rusty Russell
0b23133ab2 lightningd: don't print out notification msat fields as strings.
Reported-by: Shahana Farooqui
Changelog-Fixed: JSON-RPC: Plugin notification `msat` fields in `invoice_payment` and `invoice_created` hooks now a number, not a string with "msat" suffix.
Changelog-Fixed: JSON-RPC: Plugin hook `payment` `msat` field is now a number, not a string with "msat" suffix.
2023-11-21 08:23:39 +01:00
ShahanaFarooqui
fdc7fcbf34 tests/clnrest: Test to confirm that msat suffix has been removed from notifications 2023-11-21 08:23:39 +01:00
ShahanaFarooqui
2a7a3d2371 tests/clnrest: Added rpc method name and params to use the same notifications_received_via_websocket function for other rpc requests too 2023-11-21 08:23:39 +01:00
Dusty Daemon
a6a9e5b1e3 splice: Reestablish when commit or sig sends fail
Adds tests for when the connection fails during
1) splice tx_signature
2) splice commitment_signed

Fleshed out the reestablish flow for these two cases and implemented the fixes to make these reestablish flows work.

Part of this work required changing commit process for splices: Now we send a single commit_part for the splice where previously we sent all commits, and accordingly, we no longer revoke in response.

Changelog-Fixed: Implemented splicing restart logic for tx_signature and commitment_signed. Splice commitments are reworked in a manner incompatible with the last version.
2023-11-20 07:35:22 +01:00
Rusty Russell
eae0d1da1e lightningd: fix up deprecated rest-port, rest-protocol, rest-host and rest-certs option if we would otherwise fail.
Since these worked in v23.08, we can't just rename them.  So if they are
used and unclaimed, we should rename them internally (if they're claimed,
it's probably clightning-rest, and we should *NOT* touch them!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: Plugins: `clnrest` parameters `rest-port`, `rest-protocol`, `rest-host` and `rest-certs`: prefix `cln` to them
2023-11-17 10:51:41 +01:00
ShahanaFarooqui
cc2665da7f clnrest: prefixing all rest config options with cln
This will allow users to use clnrest with c-lightning-REST without conflicts.
It was required for applications to have enough time for migrating from c-lightning-REST to clnrest.

Changelog-Changed:
config option `rest-certs` changed to `clnrest-certs`
config option `rest-protocol` changed to `clnrest-protocol`
config option `rest-host` changed to `clnrest-host`
config option `rest-port` changed to `clnrest-port`
config option `rest-cors-origins` changed to `clnrest-cors-origins`
config option `rest-csp` changed to `clnrest-csp`
2023-11-16 11:21:04 +01:00
Rusty Russell
8b84c8945b clnrest: don't convert *_msat fields to strings.
We have a global JSON encoder hack, which means that any field ending in msat gets special treatment (so we can safely talk to lightningd, even if a field expects satoshi amounts, we are explicit).  However, requests uses the JSON parser and DOES NOT want this conversion when sending it out as an HTTP response!

The simplest local fix we could find was Shahana's suggestion to iterate and covert away from Millisatoshi(): the reverse of what our JSON encoder does.

Fixes: https://github.com/ElementsProject/lightning/issues/6848
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-11-16 09:07:07 +01:00
Rusty Russell
f2f6c73977 clnrest: don't try to handle plugin.rpc.call results.
It already throws an exception or error, or decodes the JSON response:
we can simply pass it through.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-11-16 09:07:07 +01:00