Commit Graph

436 Commits

Author SHA1 Message Date
Rusty Russell
f33c5188ef common: don't mention the contents of invalid parameters when in non-dev mode.
Shahana decided this was the optimal UX path, though I insisted that we still
report the actual problem directly when in dev mode, as a compromise.

Suggested-by: https://github.com/Amperstrand
Changelog-Changed: JSON-RPC: Do not return the contents of invalid parameters in error messages, refer to logs (use 'check' to get full error messages)
Fixes: https://github.com/ElementsProject/lightning/issues/7338
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-25 08:47:50 +09:30
Rusty Russell
8288603bab pytest: fix flake in test_emergencyrecover
```
2024-06-24T05:09:32.5233603Z         assert l1.rpc.listfunds()["channels"][0]["state"] == "ONCHAIN"
2024-06-24T05:09:32.5234187Z >       assert l2.rpc.listfunds()["channels"][0]["state"] == "ONCHAIN"
2024-06-24T05:09:32.5234917Z E       AssertionError: assert 'FUNDING_SPEND_SEEN' == 'ONCHAIN'
2024-06-24T05:09:32.5235464Z E         - ONCHAIN
2024-06-24T05:09:32.5235773Z E         + FUNDING_SPEND_SEEN
2024-06-24T05:09:32.5236096Z 
2024-06-24T05:09:32.5236242Z tests/test_misc.py:2907: AssertionError
```

It's possible that l2 hasn't seen the onchain tx yet.  What we really want to do is
wait for it, then test that l1 really can spend the output it has recovered.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-06-24 21:12:43 +09:30
Rusty Russell
7f2aedc76f common: BOLT update: option_anchors_zero_fee_htlc_tx is now simply "option_anchors".
This is a difficult transition for us: this string appears in channel
types.  We make the transition now in the understanding that it will
be more difficult in future.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Changelog-Deprecated: JSON-RPC: `listpeers` `features` array string "option_anchors_zero_fee_htlc_tx": use "option_anchors" (spec renamed it).
Changelog-Added: JSON-RPC: `listpeers` `features` array string uses "option_anchors" for feature 22/23, following renaming in BOLT 9.
Changelog-Changed: JSON-RPC: `listclosedchannels`, `listpeerchannels`, `openchannel_update`, `openchannel_init`, `fundchannel`, `fundchannel_start` and `multifundchannel`: `channel_type` array `names` now contains "anchors" instead of "anchors_zero_fee_htlc_tx".
Changelog-Changed: lightningd: `--list-features-only` now lists "option_anchors" instead of "option_anchors_zero_fee_htlc_tx".
2024-06-19 15:54:24 +09:30
ShahanaFarooqui
35101c072a commando: update removal version for rpcs
We documented them as deprecated in v23.08 but unintentionally didn't deprecate them in the code. Thus we are starting their actual deprecation cycle from v24.08 release.

Updating removal version for commands `commando-rune`, `commando-listrunes`, and `commando-blacklist`

Changelog-None.
2024-06-18 14:06:06 +09:30
daywalker90
a9ff3cb039 pytests: use reserve_unused_port() everywhere
Changelog-None
2024-05-16 17:31:02 +02:00
Rusty Russell
f0d9fc6b2e pytest: relax test_low_fd_limit for Github CI.
We expect:
	UNUSUAL.*WARNING: we have 1 channels but can file descriptors limited to 65536

We get:
	lightningd: WARNING: we have 1 channels but can file descriptors limited to 32768!

This is strange, since the first line is from Python's hard limit.  Presumably something is restricting the fd limit of children

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-05-09 01:23:46 -05:00
Rusty Russell
80f1063dee lightningd: increase fd limit if we can, to double number of starting channels.
1024 is a common limit, and people are starting to hit that many channels, so we should increase it: twice the number of channels seems reasonable, though we only do this at restart time.

Changelog-Changed: lightningd: we now try to increase the number of file descriptors, if it's less than twice the number of channels at startup (and log if we cannot!).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-05-09 01:23:46 -05:00
Rusty Russell
153567d699 pytest: test that the preapprove hooks get called by check.
We add a dev flag so we can decline them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-05-06 20:51:19 -05:00
Rusty Russell
c7339ea310 pytest: test various preapprove scenarios.
We wire through --dev options into the hsmd, and test preapprove accept and deby
with both old and new protocols.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-05-06 20:51:19 -05:00
Rusty Russell
e99f061909 setconfig: more thorough check for internal config options.
Now we can check that the setconfig would actually parse.  We do this
by handing NULL to the calback to indicate it's a test, which may not
work in general but works for now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: `check` `setconfig` now checks that the new config setting would be valid.
2024-05-06 20:51:19 -05:00
Peter Neuroth
9ae5c04583 bcli: change iteration order on peerlist
It is simpler to just iterate through the peerlist backwards.

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2024-04-24 05:50:54 +09:30
Peter Neuroth
6e3ea36976 tests: add tests for getblockfrompeer
Changelog-Changed: Plugins: bcli: Add a path that tries to fetch blocks
from a peer if we can not find them locally.

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2024-04-24 05:50:54 +09:30
Rusty Russell
901342b50d pyln-testing: require explicit pattern for BROKEN messages.
Rather than `allow_broken_log`, we have `broken_log` which is a regex
indicating what log lines are expected.  This tightens our tests
significantly, as it will catch *unexpected* brokenness.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-04-20 16:36:57 +09:30
daywalker90
507c2de9c1 newaddr: various fixes for msggen and docs
Changelog-None
2024-04-17 14:47:52 +02:00
Rusty Russell
06136ed3e3 fundchannel_start: delay, don't refuse, if node not synced.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-04-01 13:20:53 +02:00
Rusty Russell
b9b86ca526 openchannel: delay, don't refuse, if node not synced.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-04-01 13:20:53 +02:00
Rusty Russell
ef45eacc95 lightningd: allow *outgoing* HTLCs before full bitcoind sync.
This allows for faster startup for Greenlight.  We still require full sync
before *incoming* htlcs, and onchain operations.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-04-01 13:20:53 +02:00
Rusty Russell
0d7d3f5c00 lightningd: revert f450dfeb55 to allow non-gossip_query nodes.
LDK doesn't set this feature if they don't have any useful gossip (mobile nodes)
and it was agreed at the spec meeting that we should repurpose this feature
to mean "I don't have any useful gossip".

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-03-31 12:32:05 +02:00
Christian Decker
052542ea28 pytest: Stabilize the test_recover_plugin test 2024-02-20 17:59:06 +01:00
Aditya Sharma
cdb0001ce6 tests/test_misc.py: Add test_recover_plugin to check if the funds are being recovered when we lose some state and enter recover mode. 2024-02-16 22:17:46 +01:00
Rusty Russell
a005ec1e84 pytest: fix flake in test_htlc_in_timeout with anchors.
If we try to reuse the same UTXO for the HTLC as the anchor, it will clash.
One block later we can spend the anchor change, all good.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-02-08 06:32:01 +10:30
Rusty Russell
689e596036 options: make anchors enabled by default, ignore experimental-anchors.
We still want to test non-anchor channels, as we still support them, but
we've made it non-experimental.  To test non-anchor channels, we
use dev-force-features: -23.

Changelog-Added: Protocol: `option_anchors_zero_fee_htlc_tx` enabled, no longer experimental.
Changelog-Changed: Config: `experimental-anchors` now does nothing (it's enabled by default).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>


Header from folded patch 'fixup!_options__make_anchors_enabled_by_default,_ignore_experimental-anchors.patch':

fixup! options: make anchors enabled by default, ignore experimental-anchors.
2024-02-08 06:32:01 +10:30
Rusty Russell
0a2f03ac1e pytest: changed tests if we're using experimental-anchors.
This is in anticipation of changing the defaults for non-elements.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-02-08 06:32:01 +10:30
Rusty Russell
97ccf05117 lightningd: ignore gossip messages from channeld, switch to our own.
This commit is a bit messy, but it tries to do the minimal switchover.

Some tests change, so those are included here. 

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-31 14:47:33 +10:30
Rusty Russell
e30eadd6a9 pytest: check for short_channel_id, not status.
Status actually changes in this series.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-31 09:15:46 +10:30
Rusty Russell
bf9cdc597e pytest: fix flake in test_custommsg_triggers_notification
If listpeers on l2 is called too fast, then it won't return any peers, and
we will fail:

```
        # Connect l1 to l2
        l1.connect(l2)
>       wait_for(lambda: l2.rpc.listpeers(l1.info['id'])['peers'][0]['connected'])

tests/test_misc.py:2690:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
contrib/pyln-testing/pyln/testing/utils.py:88: in wait_for
    while not success():
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

>   wait_for(lambda: l2.rpc.listpeers(l1.info['id'])['peers'][0]['connected'])
E   IndexError: list index out of range
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-29 16:47:33 +10:30
Rusty Russell
08e29ab47f lightningd: make option_static_remotekey compulsory.
As suggested in https://github.com/lightning/bolts/pull/1092.

We still support channels opened without it, but you can no longer open new ones without it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: `option_gossip_queries` is now required (advertized by all but 16 nodes)
2024-01-29 13:40:34 +10:30
Rusty Russell
f450dfeb55 lightningd: make option_gossip_queries compulsory.
As suggested in https://github.com/lightning/bolts/pull/1092.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: `option_gossip_queries` is now required (advertized by all but 11 nodes)
2024-01-29 13:40:34 +10:30
Rusty Russell
88a2146664 lightningd: make option_data_loss_protect compulsory.
As suggested in https://github.com/lightning/bolts/pull/1092.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: `option_data_loss_protect` is now required (advertized by all but 11 nodes)
2024-01-29 13:40:34 +10:30
Rusty Russell
8b224b6ec2 pytest: don't assume that msat fields get turned into Millisatoshi.
Since the class interacts with int very well now, we don't have to
make explicit accesses, so it's easy to write code which works with
Millisatoshi or int.

Also, don't access rpc.decoder in one test, it's unnecessary.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-12-16 10:22:01 +01:00
Erik De Smedt
375985bf6c Implement tests for notifications on custommsg 2023-12-16 11:36:42 +10:30
Rusty Russell
f004952442 lightningd: wumbo is now the default, setting has no effect.
"Patrick, I'm sorry I doubted you."

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Config: `large-channels` is now the default, wumbology for all.
2023-11-02 08:16:51 +01:00
Alex Myers
10bac49dac ld: add commit-fee-offset option, update config schema
Changelog-Added: Added option --commit-fee-offset to potentially reduce feerate update disagreements
2023-11-02 09:49:59 +10:30
Rusty Russell
af971fd025 wait forwards: add in_htlc_id
Without this, we have no unique identifier for which forward happened.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-28 15:48:26 +10:30
Rusty Russell
0317348479 pytest: add test for wait forwards.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-28 15:48:26 +10:30
Aditya Sharma
34692ec3c8 tests/test_misc.py: Add test_decode to test the decryption of bech32 encoding of emergency.recover 2023-10-27 16:05:28 +10:30
Rusty Russell
a50b795bc2 lightningd: implement recover command.
Changelog-Added: JSON-RPC: `recover` command to force (unused) lightningd node to restart with `--recover` flag.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-26 12:59:55 +10:30
Rusty Russell
7f18ed7743 lightningd: allow --recover to take a 64-char hex string.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Config: `--recover` can take a 32-byte hex string, as well as codex32.
2023-10-26 12:59:55 +10:30
Rusty Russell
3b622f06c3 lightningd: use param_check on all commands which do extra checks.
This makes `check` much more thorough, and useful.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `check` now does much more checking on every command (not just basic parameter types).
2023-10-26 12:59:55 +10:30
Peter Neuroth
747725dabd test: Add test for datastoreusage
Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2023-10-26 12:58:04 +10:30
Rusty Russell
b0ab350927 pytest: fix flake in test_listforwards_and_listhtlcs
Make sure l1 sees chanell to l4.

```
   def test_listforwards_and_listhtlcs(node_factory, bitcoind):
        """Test listforwards and listhtlcs commands."""
        l1, l2, l3, l4 = node_factory.get_nodes(4, opts=[{}, {}, {}, {}])
    
        l1.rpc.connect(l2.info['id'], 'localhost', l2.port)
        l2.rpc.connect(l3.info['id'], 'localhost', l3.port)
        l2.rpc.connect(l4.info['id'], 'localhost', l4.port)
    
        c12, c12res = l1.fundchannel(l2, 10**5)
        c23, _ = l2.fundchannel(l3, 10**5)
        c24, _ = l2.fundchannel(l4, 10**5)
    
        # Wait until channels are active
        mine_funding_to_announce(bitcoind, [l1, l2, l3, l4])
        l1.wait_channel_active(c23)
    
        # successful payments
        i31 = l3.rpc.invoice(1000, 'i31', 'desc')
        l1.rpc.pay(i31['bolt11'])
    
        i41 = l4.rpc.invoice(2000, 'i41', 'desc')
>       l1.rpc.pay(i41['bolt11'])

tests/test_misc.py:2871: 
...
>           raise RpcError(method, payload, resp['error'])
E           pyln.client.lightning.RpcError: RPC call failed: method: pay, payload: {'bolt11': 'lnert20n1pjnd7xlsp5lcwwuxw90nanl7cpfzp4jf7w5tats5aalkksswpps6cqrwpu0l3spp5pzng94efstxjhdc2ya8hhgrch33qavzgv4gahga3y6dxqshezuqsdq8v3jhxccxqyjw5qcqp99qxpqysgqys70jhf8aj4zpwrvfrse54c5kfaqxj98xl4ddxp7jkm75e4p59gy6l2ryvz4fejg5reqkxw44k90rp0l7c8vxut424m4ek743k0lpwcqzg2a60'}, error: {'code': 210, 'message': 'Destination 0382ce59ebf18be7d84677c2e35f23294b9992ceca95491fcf8a56c6cb2d9de199 is not reachable directly and all routehints were unusable.', 'attempts': [{'status': 'failed', 'failreason': 'Destination 0382ce59ebf18be7d84677c2e35f23294b9992ceca95491fcf8a56c6cb2d9de199 is not reachable directly and all routehints were unusable.', 'partid': 0, 'amount_msat': 2000msat}]}
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-24 10:02:52 +02:00
Rusty Russell
043c4ae5eb pytest: fix flake in test_even_sendcustommsg
Make sure plugin has got message to connectd before sending!

```
    def test_even_sendcustommsg(node_factory):
        l1, l2 = node_factory.get_nodes(2, opts={'log-level': 'io',
                                                 'allow_warning': True})
        l1.connect(l2)
    
        # Even-numbered message
        msg = hex(43690)[2:] + ('ff' * 30) + 'bb'
    
        # l2 will hang up when it gets this.
        l1.rpc.sendcustommsg(l2.info['id'], msg)
        l2.daemon.wait_for_log(r'\[IN\] {}'.format(msg))
        l1.daemon.wait_for_log('Invalid unknown even msg')
        wait_for(lambda: l1.rpc.listpeers(l2.info['id'])['peers'] == [])
    
        # Now with a plugin which allows it
        l1.connect(l2)
        l2.rpc.plugin_start(os.path.join(os.getcwd(), "tests/plugins/allow_even_msgs.py"))
    
        l1.rpc.sendcustommsg(l2.info['id'], msg)
        l2.daemon.wait_for_log(r'\[IN\] {}'.format(msg))
>       l2.daemon.wait_for_log(r'allow_even_msgs.*Got message 43690')

tests/test_misc.py:3623: 
...
>                   raise TimeoutError('Unable to find "{}" in logs.'.format(exs))
E                   TimeoutError: Unable to find "[re.compile('allow_even_msgs.*Got message 43690')]" in logs.

contrib/pyln-testing/pyln/testing/utils.py:327: TimeoutError
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-24 09:51:43 +02:00
Rusty Russell
dfe207d503 pytest: test for accepting even unknown messages.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-24 11:50:57 +10:30
Rusty Russell
bd80af5295 lightningd: allow sending of even messages.
It's your funeral!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-24 11:50:57 +10:30
Vincenzo Palazzo
9b69c56cfb lightningd: print the version of cln early
As a node matures and is no longer new, it can take some time
to determine which version of `cln` it's running.

To address this, we now display the version earlier. This ensures
that even in the event of a crash, we're aware of the running version.

(cln-meta-project-py3.11) ➜  lightning git:(macros/log-version) ✗ ./lightningd/lightningd
2023-10-12T19:21:00.899Z INFO    lightningd: v23.08.1-209-gae94be4-modded
2023-10-12T19:21:00.994Z INFO    lightningd: Creating configuration directory /home/vincent/.lightning/bitcoin
2023-10-12T19:21:01.235Z INFO    lightningd: Creating database
2023-10-12T19:21:01.279Z UNUSUAL hsmd: HSM: created new hsm_secret file

Could not connect to bitcoind using bitcoin-cli. Is bitcoind running?

Make sure you have bitcoind running and that bitcoin-cli is able to connect to bitcoind.

You can verify that your Bitcoin Core installation is ready for use by running:

    $ bitcoin-cli echo 'hello world'
2023-10-12T19:21:01.349Z **BROKEN** plugin-bcli: \nCould not connect to bitcoind using bitcoin-cli. Is bitcoind running?\n\nMake sure you have bitcoind running and that bitcoin-cli is able to connect to bitcoind.\n\nYou can verify that your Bitcoin Core installation is ready for use by running:\n\n    $ bitcoin-cli echo 'hello world'\n
2023-10-12T19:21:01.349Z INFO    plugin-bcli: Killing plugin: exited before we sent init
The Bitcoin backend died.

Link: https://github.com/ElementsProject/lightning/issues/6374
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2023-10-14 17:52:15 +02:00
Rusty Russell
176a58f9e0 pytest: wean many tests off the assumption that listchannels shows private channels.
We will be changing this, or at least deprecating it, so get our
tests ready.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-04 08:02:33 +10:30
Rusty Russell
2de7171286 lightningd: fold funding tx depth into a single function.
Currently it's half done in funding_depth_cb, and half in
channeld_tell_depth.  It's very confusing as a result,
with splicing, dual-funding and zeroconf.

This does introduce a behaviour change: if a channel is NORMAL and
it gets reorganized, we force close (unless we were the one who funded
it, or it's zeroconf anyway).  This is safer than continuing to use
the channel in this case!

Some tests are changed to zeroconf to make them work, but v2 doesn't
support zeroconf, so that's removed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-02 11:41:19 +10:30
Rusty Russell
6c15ea44dd pytest: use --developer instead of environment variable.
And we always enable it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-21 20:08:24 +09:30
Rusty Russell
f725edad62 plugins: remove #if DEVELOPER.
And rename dev-only-option `use_shadow` to `dev_use_shadow`.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-21 20:08:24 +09:30
Rusty Russell
c5f6893e28 common: remove #if DEVELOPER.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-21 20:08:24 +09:30