Commit Graph

391 Commits

Author SHA1 Message Date
Rusty Russell
08f0a54fdc connectd: don't disconnect automatically on sending warning or error.
This changes some tests which expected us to disconnect.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-23 15:48:50 +10:30
Rusty Russell
176a58f9e0 pytest: wean many tests off the assumption that listchannels shows private channels.
We will be changing this, or at least deprecating it, so get our
tests ready.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-04 08:02:33 +10:30
Rusty Russell
d61dec8925 pytest: trim mostly-reundant commando-rune test.
We test this in test_runes, so make this test more cursory (otherwise
next patch breaks it).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-03 06:57:54 +10:30
Rusty Russell
acc30c0b3f lightningd: split DUALOPEND_OPEN_INIT into DUALOPEND_OPEN_INIT and DUALOPEND_OPEN_COMMITTED.
The latter is used when we're put in the db, the former is the uncommitted state.
Currently dbid == 0 is used in addition to the state, which is unwieldy.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Experimental: JSON-RPC: added new dual-funding state `DUALOPEND_OPEN_COMMITTED`
2023-10-02 11:41:19 +10:30
Rusty Russell
6c15ea44dd pytest: use --developer instead of environment variable.
And we always enable it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-21 20:08:24 +09:30
Rusty Russell
f725edad62 plugins: remove #if DEVELOPER.
And rename dev-only-option `use_shadow` to `dev_use_shadow`.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-21 20:08:24 +09:30
Rusty Russell
de81a59b1e lightningd: mark explicitly what commands are dev-only.
And require --developer to use them.

Also refuse redirection to deprecated APIs if deprecated APIs are disabled! 

Changelog-Removed: `dev-sendcustommsg` (use `sendcustommsg`, which was added in v0.10.1)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-21 20:08:24 +09:30
Rusty Russell
48522f3e9e common: don't hang up when we receive a warning.
We were allowed to, but the spec removed that.  So we handle warnings
differently from errors now.

This also means the LND "internal error" workaround is done in
lightningd (we still disconnect, but we don't want to close channel).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: we no longer disconnect every time we receive a warning message.
2023-09-20 13:56:46 +09:30
Rusty Russell
138d178dff pytest: fix plugin_connected_hook_chaining flake.
The code to workaround the intermittant error didn't work,
and we finally hit it again:

```
        # If reject happens fast enough, connect fails with "disconnected
        # during connection"
        try:
            l3.connect(l1)
        except RpcError as err:
>           assert "disconnected during connection" in err.error
E           assert 'disconnected during connection' in {'code': 402, 'message': 'disconnected during connection'}
E            +  where {'code': 402, 'message': 'disconnected during connection'} = RpcError("RPC call failed: method: connect, payload: {'id': '0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518', 'host': '127.0.0.1', 'port': 41865}, error: {'code': 402, 'message': 'disconnected during connection'}").error
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-21 21:14:01 +09:30
Rusty Russell
dec8ad3b8d pytest: fix autoclean test flake
```
        l3.rpc.setconfig('autoclean-cycle', 10)
    
        # First it expires.
>       wait_for(lambda: only_one(l3.rpc.listinvoices('inv1')['invoices'])['status'] == 'expired')

```

If we're slow enough, the invoice is cleaned before we see it expire!
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-21 21:14:01 +09:30
Rusty Russell
b5d1708900 plugins/sql: make testing-only sqlfilename option dev-sqlfilename.
dev options don't need to be documented, and this is only for debugging
the plugin itself.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-18 20:29:34 +09:30
Rusty Russell
f4e2d9a0ae lightningd: clean up properly if we fail to exec plugin.
Reported-by: @niftynei
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Plugins: we clean up properly if a plugin fails to start, and we don't kill all processes if it's from `plugin startdir`.
2023-08-18 13:18:49 +09:30
Rusty Russell
04ddb4af8d pytest: test that failing to start a plugin due to failure to exec does not put it in plugins list.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-18 13:18:49 +09:30
Rusty Russell
bc36a4580c pytest: test (failing) for not crashing if plugin fails to exec via plugin startdir
It hits the new assert(): before that (tested!) it actually kills everything and logs you out.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-18 13:18:49 +09:30
Rusty Russell
f556be5d82 renepay: allow it to die gracefully without crashing lightningd.
Suggested-by: @Lagrang3
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-08-07 17:13:29 +09:30
Dusty Daemon
4628e3ace8 channeld: Code to implement splicing
Update the lightningd <-> channeld interface with lots of new commands to needed to facilitate spicing.

Implement the channeld splicing protocol leveraging the interactivetx protocol.

Implement lightningd’s channel_control to support channeld in its splicing efforts.

Changelog-Added: Added the features to enable splicing & resizing of active channels.
2023-07-31 21:00:22 +09:30
Rusty Russell
07e00d50c6 invoice: return error string from invoice_check_payment.
Clean these up: they were debug logs, but we want to pass this information
back for self-payments.

Also fixes "Attept" typo which altered tests!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-07-26 06:55:48 +09:30
Rusty Russell
a219c99691 commando: pass through commands to lightningd.
The in-core commands are much more helpful with details on why runes failed, too!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-07-25 14:47:12 +09:30
Rusty Russell
38075a95d4 commando: use CLN's checkrune() instead of our own for for rune validation.
This means (temporarily) that blacklisting won't work (fix later), and
means that old-style (commando.py) master-secret-override doesn't work.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: Plugins: `commando` no longer allows datastore ['commando', 'secret'] to override master secret (re-issue runes if you were using that!).
2023-07-25 14:47:12 +09:30
Rusty Russell
cd16690af8 commando: always active.
We used to activate on the first rune creation, but we're no longer in charge
of runes, so we can't make that call.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-07-25 14:47:12 +09:30
Rusty Russell
8d64e6e277 pytest: use native l1 commando secret.
We're going to remove the ability to override the master secret, so fix
up our tests.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-07-25 14:47:12 +09:30
Rusty Russell
d25a8ca0fd lightningd: expose created_index and updated_index fields.
If you miss a wait event, you can catch up by doing listinvoices and
getting the max of these fields.  It's also a good debugging clue.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-07-23 13:48:37 +09:30
Rusty Russell
8910529e6a lightningd: expose ignore_fee_limits field in listpeerchannels.
Changelog-Added: JSON-RPC: `listpeerchannels` has a new field `ignore_fee_limits`
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-07-22 13:11:08 +09:30
Rusty Russell
0d34f4916e pytest: fix timeout in test_sql
The logs show we're not waiting for the right transaction, so be explicit.

```
        # Check that channels gets refreshed!
        scid = l1.get_channel_scid(l2)
        l1.rpc.setchannel(scid, feebase=123)
        wait_for(lambda: l3.rpc.sql("SELECT short_channel_id FROM channels WHERE base_fee_millisatoshi = 123;")['rows'] == [[scid]])
        l3.daemon.wait_for_log("Refreshing channels...")
        l3.daemon.wait_for_log("Refreshing channel: {}".format(scid))
    
        # This has to wait for the hold_invoice plugin to let go!
        l1.rpc.close(l2.info['id'])
        bitcoind.generate_block(13, wait_for_mempool=1)
>       wait_for(lambda: len(l3.rpc.listchannels()['channels']) == 2)

tests/test_plugin.py:4143: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

success = <function test_sql.<locals>.<lambda> at 0x7fde3b9cd3a0>, timeout = 180

    def wait_for(success, timeout=TIMEOUT):
        start_time = time.time()
        interval = 0.25
        while not success():
            time_left = start_time + timeout - time.time()
            if time_left <= 0:
>               raise ValueError("Timeout while waiting for {}".format(success))
E               ValueError: Timeout while waiting for <function test_sql.<locals>.<lambda> at 0x7fde3b9cd3a0>
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-07-22 08:52:55 +09:30
Christian Decker
c3a983375c pytest: Remove tests that rely on presplit behavior
These tests make assumptions about the presplitter behavior which
we'll remove in the next commit. We remove them here so we don't cause
temporary breaks in the git history.
2023-07-21 07:13:48 +09:30
Shahana Farooqui
a4835cf266 commando: changed listrunes schema for restrictions english key
Changelog: None
2023-07-19 05:40:30 +09:30
Rusty Russell
8de3b89aac pytest: test for bad estimatefees results.
See: #6183
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-07-16 12:57:27 +09:30
Rusty Russell
ad592d8b0d plugins: fix shutdown notification to contain object.
We fixed the others.  There are no fields, but this keeps it consistent.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `shutdown` notification contains `shutdown` object (notification consistency)
2023-07-13 18:59:37 +09:30
Rusty Russell
4d507065f3 libplugin: support wildcard subscriptions.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-07-13 18:59:37 +09:30
Rusty Russell
279b3aa7e8 pyln: add ability to subscribe to all notifications.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-07-13 18:59:37 +09:30
Rusty Russell
71adeccfbf pytest: fix reconnect flake in test_plugin_connected_hook_chaining
```
         l1.rpc.reject(l3.info['id'])
     
         l2.connect(l1)
         l1.daemon.wait_for_logs([
             f"peer_connected_logger_a {l2id}",
             f"{l2id} is allowed",
             f"peer_connected_logger_b {l2id}"
         ])
         assert len(l1.rpc.listpeers(l2id)['peers']) == 1
     
 >       l3.connect(l1)
 tests/test_plugin.py:468: 
... 
 >           raise RpcError(method, payload, resp['error'])
 E           pyln.client.lightning.RpcError: RPC call failed: method: connect, payload: {'id': '0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518', 'host': '127.0.0.1', 'port': 42391}, error: {'code': 402, 'message': 'disconnected during connection'}
 
 contrib/pyln-client/pyln/client/lightning.py:422: RpcError
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-23 14:36:23 +09:30
Rusty Russell
263625872e autoclean: allow dynamic changes to autoclean-cycle.
Slightly less trivial: reset timer unless it's currently running callback.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-20 20:08:25 +09:30
Rusty Russell
f8f8045a06 autoclean: various configuration options now dynamic.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Plugins: `autoclean` configuration variables now settable with `setconfig`.
2023-06-20 20:08:25 +09:30
Vincenzo Palazzo
85992e6e48 feat(bitcoind): pass the current known block height
When core lightning is asking the information about
the blockchain with `getchaininfo` command lightningd
know already the information about the min and max block height.

the problem is when we have a smarter Bitcoin backend that is able
to switch between different clients in some cases is helpful
give the information about current known height by lightningd and
pass it down to the plugin.

In this way, the plugin knows what is the correct known height from lightnind, and can
try to fix some problems if any exit.

This is particularly useful when you are syncing a new backend from scratch
like https://github.com/cloudhead/nakamoto and we avoid returning the
lower height from the known, and avoid the crash of core lightning.

With this information, the plugin can start to sync the chain and return
the answer back only when the chain is in sync with the current status of
lightningd.

Another reason to add this field and not wait the correct block in core
lightning itself is because Bitcoin Core is extremely slow to sync up,
so the question here is, how long should we wait? The time depends
on various factors.

With this approach of informing the plugin about the height, in some cases,
you can start the syncing but move the execution to another backend until
the previous one is ready.

The problem I want to solve is that I don't want to be left in the dark when
we run `getchaininfo`, and I want to have the opportunity to wait for
the blockchain sync or decide to dispatch the request elsewhere.

Changelog-Added: Pass the current known block height down to the getchaininfo call.
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2023-06-13 16:27:10 +02:00
Rusty Russell
55119e6ca0 pytest: mark test_commando_stress as slow.
It sometimes takes > 1800 seconds under valgrind.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-06 19:47:08 +09:30
Rusty Russell
33be5c8d80 commando: integer command parameters can be compared with < and >.
Previously any attempt would result in "is not an integer field"; we
now recognize valid JSON integers as integer fields.

Changelog-Fixed: Plugins: `commando` runes can now compare integer parameters using '<' and '>' as expected.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-06 19:47:08 +09:30
Rusty Russell
0f72542967 pytest: test for amount comparison.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-06 19:47:08 +09:30
Rusty Russell
3ac949d4c3 listconfigs: add plugin field if config is for a plugin.
I chose the full path name, not just the basename.

Suggested-by: @SimonVrouwe
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-03 10:50:29 +09:30
Rusty Russell
ea928bfca1 pytest: use modern listconfigs.
Use the configs object, as the others are about to be deprecated.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-03 10:50:29 +09:30
Rusty Russell
9cb2b2f13a listconfigs: show plugin options in 'configs' with normal options.
This integrates them with configvars properly: they almost "just work"
in listconfigs now, and we don't put them in a special sub-object
under their plugin.

Unfortunately, this means `listconfigs` now has a loose schema: any
plugin can add something to it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Plugins: reloaded plugins get passed any vars from configuration files.
Changelog-Deprecated: Config: boolean plugin options set to `1` or `0` (use `true` and `false` like non-plugin options).
2023-06-03 10:50:29 +09:30
Rusty Russell
edbaa944da lightningd: switch parsing to common/configvar
Now we wire in the code which gathers configvars and parses from there;
lightningd keeps the array of configuration variables for future use.

Note that lightning-cli also needs to read the config, but it has its
own options (including short ones!) and doesn't want to use this
configvar mechanism, so we have a different API for that now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-03 10:50:29 +09:30
Rusty Russell
b8aa3a579e listtransactions: remove annotations, which were only in EXPERIMENTAL_FEATURES.
I don't know if anyone was using them, they seem half-hearted.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-05-23 09:34:08 +09:30
Rusty Russell
a000ee015a lightningd: do RBF again for all the txs.
Now we've set everything up, the replacement code is quite simple.

Some tests now have to deal with RBF though, and our rbf tests need work
since they look for the old onchaind messages.

In particular, when we can't afford the fee we want, we back off to
the next blockcount estimate, rather than spending all on fees
(necessarily).  So test_penalty_rbf_burn no longer applies.

Changelog-Changed: Protocol: spending unilateral close transactions now use dynamic fees based on deadlines (and RBF), instead of fixed fees.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-10 07:31:12 +09:30
Rusty Russell
812a5a14c0 plugins/bcli: use the new feerate levels, and the floor.
Fixes: #4473
Changelog-Deprecated: Plugins: `estimatefees` returning feerates by name (e.g. "opening"); use `fee_floor` and `feerates`.
Changelog-Fixed: Plugins: `bcli` now tells us the minimal possible feerate, such as with mempool congestion, rather than assuming 1 sat/vbyte.
2023-04-10 07:31:12 +09:30
Rusty Russell
c5b7dbcd98 pytest: clean up wait_for_onchaind_tx interface, remove wait_for_onchaind_broadcast
Using single tuples in Python is ugly, so:
1. Rename wait_for_onchaind_tx to wait_for_onchaind_txs.
2. Make it take tuples explicitly.
3. Make wait_for_onchaind_tx a simpler wrapper/unwrapper.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-07 11:49:09 +09:30
Rusty Russell
0c27acc705 onchaind: use lightningd to sign and broadcast htlc expired txs.
This is when they closed the channel, we can simply make our own tx to
expire the HTLC.  (The other case is where we closed the channel, and
we have a special htlc_timeout tx which we have their signature for).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-07 11:49:09 +09:30
Shahana Farooqui
61b063440c tests: commando-blacklist 2023-04-01 14:07:23 +10:30
Shahana Farooqui
9d7afba357 tests: commando-listrunes 2023-04-01 14:07:23 +10:30
Rusty Russell
b8519a6a1a plugins/sql: add listclosedchannels
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `sql` now includes `listclosedchannels`.
2023-03-25 15:28:02 +10:30
Rusty Russell
e75cf2e7fb listpeerchannels: add channel_type, both in hex and as array of names.
Changelog-Added: JSON-RPC: `listpeerchannels` now has `channel_type` field.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-03-25 15:28:02 +10:30