Commit graph

3278 commits

Author SHA1 Message Date
Dusty Daemon
2e92fdc507 interactivetx: Add support for shared prevtx
It is possible for prevtx to be larger than max packet size, so for shared outputs (currently only the funding tx) we add support for sending the `txid` only across the wire and filling in the prevtx locally.

Changelog-None
2024-11-21 14:15:36 +10:30
Rusty Russell
2bf1053cdb offers: update block height correctly.
As we can see from the previous test, l3 tells us why it rejected the payment:

```
lightningd-3 2024-11-19T03:56:27.151Z DEBUG   022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59-chan#1: Failing HTLC because of an invalid payload (TLV 10 pos 104): cltv_expiry 136 > payment_constraint 130
```

It turns out, we were not updating the block height in the plugin!

Without this, when we create a (non-dummy) blinded path we set a
too-low CLTV restriction, and it doesn't work after a few blocks!

Note we were actually triggering this error in the xpay tests!

Reported-by: Vincenzo Palazzo
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Offers: Receiving bolt12 payments where we have no public channels would fail a few blocks after startup.
2024-11-20 12:29:27 +01:00
Vincenzo Palazzo
6f4f33d06b test: reproduce WIRE_INVALID_ONION_PAYLOAD
```
>           raise RpcError(method, payload, resp['error'])
E           pyln.client.lightning.RpcError: RPC call failed: method: pay, payload: {'bolt11': 'lni1qqgvsykv6pslpmzq73597z0ws2qv5q3qqc3xu3s3rg94nj40zfsy866mhu5vxne6tcej5878k2mneuvgjy8ssqgzpg48getnw30k7enxv4e97amfw3597urjd9mxzar9ta3ksctwdejkcu6ld46kcaredphhqvssacp462c3jt0m5y6wzrj5pp6axehtez7r20265antsrqfpvuu8fwcshgzqjushll8xx9x356tn40gk9mxzkyua9ajtrdpyhm3uaj9nvj0fm9qyqnjp20gp6gr2qsmfas7j086jvkmszqgyys3uht6jq7g4p6vsg7fyyqrx76aulp40m9uxejn57eyczy6v6hqmxr8xx273l480kd5zcl0g9hqp3d9qnsrj40gmeshx0w7fu6j9cfthksz2xv78wxr4ae4wrc3lht8lryc2kxxdpxvs3tpdepm0asuvp0l25fqqvjumjneecjg9etepcu426t2ueu6p3escfrxl9ggnkh5k2vm929tnt26dx66nt67kfy5lgx99py2jhqalaqkyypjeu2artvufgydym4tryv0wvkca78ac64mjeqt70d3wsmjcjgnqnjsyqrzymjxzydqkkw24ufxqslttwlj3s608f0rx2slc7etw0833zgs7kppqd350d9wur2l6mkanmpsswv4xrc49kaq6ey9sfn3rg3z8afgng8fdg8aqr7sxhftzxfdlwsnfcgw2sy8t5mxa0ytcdfat2nkdwqvpy9nnsa9mzzaqtyu8wy0yul9hk026znqy6pn6xd2fpxva7jjcpmvqugeewk7emufyqsru03er082j6ya0p694k6qu858hl0g9rt7g2y042ppzyhdv4qdv99qqs3scf49sff7vg27zlmx6n3kgywrh3s82rwgpza2s8jmqqx72ah0kurp94rj7dlxq438nnm34w78kq7hwu53chx0aqh824eqcsgmq9j2fyvsqttg4yksstnk7h7ga5as69fhemltg0m9hqnn2yxr0lxv70293l7ryqpjfamk2k4mzgax8txef7zcxdzjn0wg7te2cx98ft9cyhk0hquzypasww8m40kyzyqvahtzamflylsygnny5gwqqqqqqyqqqqq2qq9sqqqqqqqqqqqqqqqqqqp6tqqqqqqq5szxdnxnuk5zpctsclfcs4yx0kcvz8nckast2ueswxftuelh49su6sqs2vlzs8ha4gqs9vppqvk0zhg6m8z2prfxa2cerrmn9k803lwx4wukgzlnmvt5xukyjycyauzqwvm6pxlfpfffgktvj3wkurcwrqcp0537hnkd8pnm7tsa0zcklua9zv338cjuphz38wml6tlr8xgdzxdsqh0ks2pns2zkn3c52crfcfs'}, error: {'code': 203, 'message': 'failed: WIRE_INVALID_ONION_PAYLOAD (reply from remote)', 'id': 1, 'failcode': 16406, 'failcodename': 'WIRE_INVALID_ONION_PAYLOAD', 'bolt12': 'lni1qqgvsykv6pslpmzq73597z0ws2qv5q3qqc3xu3s3rg94nj40zfsy866mhu5vxne6tcej5878k2mneuvgjy8ssqgzpg48getnw30k7enxv4e97amfw3597urjd9mxzar9ta3ksctwdejkcu6ld46kcaredphhqvssacp462c3jt0m5y6wzrj5pp6axehtez7r20265antsrqfpvuu8fwcshgzqjushll8xx9x356tn40gk9mxzkyua9ajtrdpyhm3uaj9nvj0fm9qyqnjp20gp6gr2qsmfas7j086jvkmszqgyys3uht6jq7g4p6vsg7fyyqrx76aulp40m9uxejn57eyczy6v6hqmxr8xx273l480kd5zcl0g9hqp3d9qnsrj40gmeshx0w7fu6j9cfthksz2xv78wxr4ae4wrc3lht8lryc2kxxdpxvs3tpdepm0asuvp0l25fqqvjumjneecjg9etepcu426t2ueu6p3escfrxl9ggnkh5k2vm929tnt26dx66nt67kfy5lgx99py2jhqalaqkyypjeu2artvufgydym4tryv0wvkca78ac64mjeqt70d3wsmjcjgnqnjsyqrzymjxzydqkkw24ufxqslttwlj3s608f0rx2slc7etw0833zgs7kppqd350d9wur2l6mkanmpsswv4xrc49kaq6ey9sfn3rg3z8afgng8fdg8aqr7sxhftzxfdlwsnfcgw2sy8t5mxa0ytcdfat2nkdwqvpy9nnsa9mzzaqtyu8wy0yul9hk026znqy6pn6xd2fpxva7jjcpmvqugeewk7emufyqsru03er082j6ya0p694k6qu858hl0g9rt7g2y042ppzyhdv4qdv99qqs3scf49sff7vg27zlmx6n3kgywrh3s82rwgpza2s8jmqqx72ah0kurp94rj7dlxq438nnm34w78kq7hwu53chx0aqh824eqcsgmq9j2fyvsqttg4yksstnk7h7ga5as69fhemltg0m9hqnn2yxr0lxv70293l7ryqpjfamk2k4mzgax8txef7zcxdzjn0wg7te2cx98ft9cyhk0hquzypasww8m40kyzyqvahtzamflylsygnny5gwqqqqqqyqqqqq2qq9sqqqqqqqqqqqqqqqqqqp6tqqqqqqq5szxdnxnuk5zpctsclfcs4yx0kcvz8nckast2ueswxftuelh49su6sqs2vlzs8ha4gqs9vppqvk0zhg6m8z2prfxa2cerrmn9k803lwx4wukgzlnmvt5xukyjycyauzqwvm6pxlfpfffgktvj3wkurcwrqcp0537hnkd8pnm7tsa0zcklua9zv338cjuphz38wml6tlr8xgdzxdsqh0ks2pns2zkn3c52crfcfs', 'raw_message': '40160a0068', 'created_at': 1724699621, 'destination': '032cf15d1ad9c4a08d26eab1918f732d8ef8fdc6abb9640bf3db174372c491304e', 'payment_hash': 'e170c7d38854867db0c11e78b760b573307192be67f7a961cd4010533e281efd', 'status': 'failed', 'amount_msat': 3, 'amount_sent_msat': 0, 'erring_index': 2, 'erring_node': '035d2b1192dfba134e10e540875d366ebc8bc353d5aa766b80c090b39c3a5d885d'}

contrib/pyln-client/pyln/client/lightning.py:416: RpcError
```

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2024-11-20 12:29:27 +01:00
Rusty Russell
1160e35669 lightningd: send errors inside blinded paths correctly.
Don't reply with update_fail_malformed_htlc, even though WIRE_INVALID_ONION_BLINDING
has BADONION set.  Fail it with a normal error message.

This fixes a known FIXME.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Protocol: entry to blinded paths return more useful errors (e.g if it's the final node, you get a real error, otherwise you get invalid_onion_blinding).
2024-11-20 12:29:27 +01:00
Rusty Russell
e38f5d8c27 common: provide readable explanation when onion payload is invalid.
I had to use fprintf, which is terrible.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-20 12:29:27 +01:00
Rusty Russell
9ed7260328 pytest: fix test_pay tests now we've deprecated experimental-offers.
The test was merged after it was deprecated, and autogenerate-rpc-examples isn't
run by CI.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-20 17:34:24 +10:30
Vincenzo Palazzo
e76a334609 tests: add the test for fetching invoice with metadata
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2024-11-19 22:54:22 +01:00
Rusty Russell
6c347a4050 pytest: fix flake in test_gossip_pruning
We actually pruned before we got all the channels.  Extend the pruning time,
which unfortunately makes the test slower.

```
2024-11-18T02:13:11.7013278Z node_factory = <pyln.testing.utils.NodeFactory object at 0x7ff72969e820>
2024-11-18T02:13:11.7014386Z bitcoind = <pyln.testing.utils.BitcoinD object at 0x7ff72968fe20>
2024-11-18T02:13:11.7014996Z 
2024-11-18T02:13:11.7015271Z     def test_gossip_pruning(node_factory, bitcoind):
2024-11-18T02:13:11.7016222Z         """ Create channel and see it being updated in time before pruning
2024-11-18T02:13:11.7017037Z         """
2024-11-18T02:13:11.7017871Z         l1, l2, l3 = node_factory.get_nodes(3, opts={'dev-fast-gossip-prune': None,
2024-11-18T02:13:11.7018971Z                                                      'allow_bad_gossip': True})
2024-11-18T02:13:11.7019634Z     
2024-11-18T02:13:11.7020236Z         l1.rpc.connect(l2.info['id'], 'localhost', l2.port)
2024-11-18T02:13:11.7021153Z         l2.rpc.connect(l3.info['id'], 'localhost', l3.port)
2024-11-18T02:13:11.7021806Z     
2024-11-18T02:13:11.7022226Z         scid1, _ = l1.fundchannel(l2, 10**6)
2024-11-18T02:13:11.7022886Z         scid2, _ = l2.fundchannel(l3, 10**6)
2024-11-18T02:13:11.7023458Z     
2024-11-18T02:13:11.7023907Z         mine_funding_to_announce(bitcoind, [l1, l2, l3])
2024-11-18T02:13:11.7025183Z         l1_initial_cupdate_timestamp = only_one(l1.rpc.listchannels(source=l1.info['id'])['channels'])['last_update']
2024-11-18T02:13:11.7026179Z     
2024-11-18T02:13:11.7027358Z         # Get timestamps of initial updates, so we can ensure they change.
2024-11-18T02:13:11.7028171Z         # Channels should be activated locally
2024-11-18T02:13:11.7029326Z >       wait_for(lambda: [c['active'] for c in l1.rpc.listchannels()['channels']] == [True] * 4)
```

We can see in logs, it actually started pruning already:

```
2024-11-18T02:13:11.9622477Z lightningd-1 2024-11-18T01:52:03.570Z DEBUG   gossipd: Pruning channel 105x1x0 from network view (ages 1731894723 and 0)
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-19 17:51:18 +10:30
Rusty Russell
79d39e0296 pytest: fix flake in test_onionmessage_ratelimit
Sometimes l1 ratelimits before l2, and l2 receives the warning message, not l1:

```
>       assert l1.daemon.is_in_log('WARNING: Ratelimited onion_message: exceeded one per 250msec')
E       AssertionError: assert None
E        +  where None = <bound method TailableProc.is_in_log of <pyln.testing.utils.LightningD object at 0x7f13435f45b0>>('WARNING: Ratelimited onion_message: exceeded one per 250msec')
E        +    where <bound method TailableProc.is_in_log of <pyln.testing.utils.LightningD object at 0x7f13435f45b0>> = <pyln.testing.utils.LightningD object at 0x7f13435f45b0>.is_in_log
E        +      where <pyln.testing.utils.LightningD object at 0x7f13435f45b0> = <fixtures.LightningNode object at 0x7f13435cbb80>.daemon
...
lightningd-1 2024-11-19T00:45:43.721Z DEBUG   022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59-connectd: peer_in WIRE_ONION_MESSAGE
lightningd-1 2024-11-19T00:45:43.721Z DEBUG   022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59-connectd: peer_out WIRE_WARNING
lightningd-2 2024-11-19T00:45:43.722Z DEBUG   0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-connectd: peer_out WIRE_ONION_MESSAGE
lightningd-2 2024-11-19T00:45:43.722Z DEBUG   connectd: REPLY WIRE_CONNECTD_INJECT_ONIONMSG_REPLY with 0 fds
lightningd-2 2024-11-19T00:45:43.722Z DEBUG   0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-connectd: peer_in WIRE_WARNING
lightningd-2 2024-11-19T00:45:43.722Z INFO    0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-connectd: Received WIRE_WARNING: WARNING: Ratelimited onion_message: exceeded one per 250msec
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-19 17:51:18 +10:30
Rusty Russell
c4f8716a55 pytest: fix race in test_autoclean.
We can see the log message about cleanup just after the test ends.

```
>       assert l3.rpc.autoclean_status()['autoclean']['expiredinvoices']['cleaned'] == 1
E       assert 0 == 1

tests/test_plugin.py:3232: AssertionError
```
...
```

lightningd-3 2024-11-18T07:52:55.402Z INFO    lightningd: setconfig: autoclean-cycle 10 (updated /tmp/ltests-ao0p8pem/test_autoclean_1/lightning-3/regtest/config:4)
...
lightningd-3 2024-11-18T07:52:59.747Z DEBUG   022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59-connectd: peer_out WIRE_QUERY_CHANNEL_RANGE
lightningd-3 2024-11-18T07:52:59.747Z DEBUG   022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59-gossipd: reply_channel_range 0+109 (of 0+109) 2 scids
lightningd-3 2024-11-18T07:52:59.747Z DEBUG   gossipd: seeker: state = NORMAL No unannounced nodes
{'github_repository': 'ElementsProject/lightning', 'github_sha': '0729de783e95c5208b1706f7d27b23904596bb71', 'github_ref': 'refs/pull/7835/merge', 'github_ref_name': 'HEAD', 'github_run_id': 11887300979, 'github_head_ref': 'guilt/fix-flakes8', 'github_run_number': 11566, 'github_base_ref': 'master', 'github_run_attempt': '1', 'testname': 'test_autoclean', 'start_time': 1731916359, 'end_time': 1731916385, 'outcome': 'fail'}
--------------------------- Captured stdout teardown ---------------------------
lightningd-3 2024-11-18T07:53:05.503Z DEBUG   plugin-autoclean: cleaned 1 from expiredinvoices
lightningd-3 2024-11-18T07:53:05.503Z DEBUG   plugin-autoclean: setting next timer
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-19 17:51:18 +10:30
Rusty Russell
1a5916cb81 pytest: try to fix flake in test_lightningd_still_loading
I can't reproduce this, but CI did (with Elements):

```
[gw3] linux -- Python 3.8.18 /home/runner/.cache/pypoetry/virtualenvs/cln-meta-project-AqJ9wMix-py3.8/bin/python

node_factory = <pyln.testing.utils.NodeFactory object at 0x7fd0e20f57f0>
bitcoind = <pyln.testing.utils.ElementsD object at 0x7fd0e307dbe0>
executor = <concurrent.futures.thread.ThreadPoolExecutor object at 0x7fd0e307da30>

    @pytest.mark.openchannel('v1')
    @pytest.mark.openchannel('v2')
    def test_lightningd_still_loading(node_factory, bitcoind, executor):
        """Test that we recognize we haven't got all blocks from bitcoind"""
    
        mock_release = Event()
    
        # This is slow enough that we're going to notice.
        def mock_getblock(r):
            conf_file = os.path.join(bitcoind.bitcoin_dir, 'bitcoin.conf')
            brpc = RawProxy(btc_conf_file=conf_file)
            if r['params'][0] == slow_blockid:
                mock_release.wait(TIMEOUT)
            return {
                "result": brpc._call(r['method'], *r['params']),
                "error": None,
                "id": r['id']
            }
    
        # Start it, establish channel, get extra funds.
        l1, l2, l3 = node_factory.get_nodes(3, opts=[{'may_reconnect': True,
                                                      'wait_for_bitcoind_sync': False},
                                                     {'may_reconnect': True,
                                                      'wait_for_bitcoind_sync': False},
                                                     {}])
        node_factory.join_nodes([l1, l2])
    
        # Balance l1<->l2 channel
        l1.pay(l2, 10**9 // 2)
    
        l1.stop()
    
        # Now make sure l2 is behind.
        bitcoind.generate_block(2)
        # Make sure l2/l3 are synced
        sync_blockheight(bitcoind, [l2, l3])
    
        # Make it slow grabbing the final block.
        slow_blockid = bitcoind.rpc.getblockhash(bitcoind.rpc.getblockcount())
        l1.daemon.rpcproxy.mock_rpc('getblock', mock_getblock)
    
        l1.start(wait_for_bitcoind_sync=False)
    
        # It will warn about being out-of-sync.
        assert 'warning_bitcoind_sync' not in l1.rpc.getinfo()
        assert 'warning_lightningd_sync' in l1.rpc.getinfo()
    
        # Make sure it's connected to l2 (otherwise we get TEMPORARY_CHANNEL_FAILURE)
        wait_for(lambda: only_one(l1.rpc.listpeers(l2.info['id'])['peers'])['connected'])
    
        # Payments will succced.
        l1.pay(l2, 1000)
>       assert l1.daemon.is_in_log(r"Sending HTLC while still syncing with bitcoin network \(104 vs 105\)")
E       AssertionError: assert None
E        +  where None = <bound method TailableProc.is_in_log of <pyln.testing.utils.LightningD object at 0x7fd0e20f9fa0>>('Sending HTLC while still syncing with bitcoin network \\(104 vs 105\\)')
E        +    where <bound method TailableProc.is_in_log of <pyln.testing.utils.LightningD object at 0x7fd0e20f9fa0>> = <pyln.testing.utils.LightningD object at 0x7fd0e20f9fa0>.is_in_log
E        +      where <pyln.testing.utils.LightningD object at 0x7fd0e20f9fa0> = <fixtures.LightningNode object at 0x7fd0e20f59d0>.daemon
```

What was in logs was:

```
lightningd-1 2024-11-18T05:33:50.634Z DEBUG   022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59-chan#1: Sending HTLC while still syncing with bitcoin network (103 vs 105)
```

Implying that l1 was an extra block behind.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-19 17:51:18 +10:30
Rusty Russell
576d003cf0 pytest: fix flake in test_wumbo_channels
We need to wait for *l2* to see the channel in CHANNELD_NORMAL,
otherwise the array here is empty:

```
	chan = only_one([c for c in l1.rpc.listpeerchannels(l2.info['id'])['channels'] if c['state'] == 'CHANNELD_NORMAL'])
        amount = chan['funding']['local_funds_msat']
        assert amount > Millisatoshi(str((1 << 24) - 1) + "sat")
    
        # We should know we can spend that much!
        spendable = chan['spendable_msat']
        assert spendable > Millisatoshi(str((1 << 24) - 1) + "sat")
    
        # So should peer.
>       chan = only_one([c for c in l2.rpc.listpeerchannels(l1.info['id'])['channels'] if c['state'] == 'CHANNELD_NORMAL'])

tests/test_connection.py:3552: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

arr = []

    def only_one(arr):
        """Many JSON RPC calls return an array; often we only expect a single entry
        """
>       assert len(arr) == 1
E       AssertionError
```
2024-11-19 17:51:18 +10:30
Michael Schmoock
e79a275209 pytest: fix a test that broke because of docstring usage 2024-11-19 11:50:42 +10:30
ShahanaFarooqui
2e52df41dd test: Fixed test plugin source paths for reckless 2024-11-19 09:06:28 +10:30
ShahanaFarooqui
e352a72e5e shellcheck: shellcheck fixes 2024-11-19 09:05:55 +10:30
Rusty Russell
0cc52bc281 pytest: don't set experimental-offers in tests: it's the default now.
And about to be deprecated.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-18 10:42:54 +01:00
Rusty Russell
ca41414da4 offers: make it the default.
Changelog-Changed: offers: bolt12 now enabled by default (finally!)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-18 10:42:54 +01:00
Rusty Russell
05fbcb4d9b xpay: make sure to call preapproveinvoice!
This is required for VLS which wants to know (and potentially decline) invoices
we're trying to pay.

As a nice side effect, our "check" command for xpay now does much more thorough
checking of arguments.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-18 11:03:26 +10:30
Rusty Russell
2c15dc0e5b xpay: make self-pay work.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-18 11:03:26 +10:30
Rusty Russell
229fc3f2b4 xpay: make the xpay layer persistent.
As the first user of a persistent layer, this tripped tests which
assumed the datastore would be empty!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-18 11:03:26 +10:30
Rusty Russell
c93153ec37 lightningd: allow builtin plugins to be stopped.
These are automatically marked "important", in the sense that we won't startup
if they are not working, but this wasn't meant to disallow stopping them.

Changelog-Changed: JSON-RPC: built-in plugins can now be stopped using "plugin stop".
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-18 11:03:26 +10:30
Rusty Russell
c715253af7 xpay: option to steal easy commands from pay.
Note: won't work with grpc (or probably other tools), since the output
is different.  But good for testing.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Config: option `xpay-handle-pay` can be used to call xpay when pay is used in many cases (but output is different from pay!)
2024-11-18 11:03:26 +10:30
Rusty Russell
dd4d3940b1 xpay: implement partial payments.
Explicitly tell xpay to only make part of a payment.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-18 11:03:26 +10:30
Rusty Russell
47318a7875 xpay: add deadline.
We promised this in the schema originally, now support it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-18 11:03:26 +10:30
Rusty Russell
06a9aa8907 pytest: test xpay with channeld_fakenet to pay top 100 nodes.
We also put broken status messages for invalid CLTVs and amounts in onions,
to catch them in CI.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-18 11:03:26 +10:30
Rusty Russell
7c2407ef48 xpay: new plugin which uses askrene, injectpaymentonion.
Changelog-Added: Plugins: cln-xpay, with associated `xpay` command for payments.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-18 11:03:26 +10:30
Rusty Russell
611e430754 lightningd: injectpaymentonion should fail on re-attempts.
This is clearer than transparently succeeding: the user might think they
paid twice.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-18 11:03:26 +10:30
ShahanaFarooqui
2dd0d4c88a test: listaddresses tests
Added a new test for listaddresses rpc.
2024-11-17 20:27:13 +10:30
Rusty Russell
c79a89d557 pytest: adapt tests to avoid deprecated APIs in close (tx and txid).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-17 16:04:06 +10:30
Dusty Daemon
4500661bbf channeld: Add aggressive restart test
Changelog-None
2024-11-17 15:32:28 +10:30
niftynei
cb2163e439 bkpr: add bookkeeping assertions to splice in + out tests
Make sure that the fees and channel balances for splice ins and outs
work as expected.

Note this is for a single-sided splice.
Changelog-None: Tests!
2024-11-17 14:25:29 +10:30
Rusty Russell
fcebb33180 lightningd: deprecate decodepay.
It only works on BOLT11, and has long been replaced by the more
generic "decode".

Removing it will stop the confusion!

(Note: documentation claims it was introduced in 23.08, but that was
 wrong, as it's been in CLN since the beginning).

[ Fixup from: niftynei <niftynei@gmail.com> ]
Fixes: https://github.com/ElementsProject/lightning/issues/6419
Changelog-Deprecated: JSON-RPC: `decodepay`: use `decode`.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-17 14:21:23 +10:30
Rusty Russell
7ed2126ee7 pytest: fix flake in test_fetchinvoice_disconnected_reply.
Fails when l3 doesn't know address for l1, to connect to it:

```
2024-11-16T04:45:42.2243366Z lightningd-3 2024-11-16T04:35:10.582Z DEBUG   022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59-connectd: peer_in WIRE_ONION_MESSAGE
2024-11-16T04:45:42.2244342Z lightningd-3 2024-11-16T04:35:10.582Z DEBUG   lightningd: Got onionmsg reply_path
2024-11-16T04:45:42.2245398Z lightningd-3 2024-11-16T04:35:10.582Z DEBUG   plugin-offers: Note: disallowing deprecated onion_message_recv.blinding
2024-11-16T04:45:42.2246408Z lightningd-3 2024-11-16T04:35:10.586Z UNUSUAL plugin-offers: No incoming channel for 5msat, so no blinded path
2024-11-16T04:45:42.2247289Z lightningd-3 2024-11-16T04:35:10.605Z DEBUG   hsmd: Client: Received message 25 from client
2024-11-16T04:45:42.2248372Z lightningd-3 2024-11-16T04:35:10.606Z DEBUG   plugin-offers: connecting directly to 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518
2024-11-16T04:45:42.2249451Z lightningd-3 2024-11-16T04:35:10.606Z DEBUG   gossipd: REPLY WIRE_GOSSIPD_GET_ADDRS_REPLY with 0 fds
2024-11-16T04:45:42.2250743Z lightningd-3 2024-11-16T04:35:10.607Z DEBUG   0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-connectd: Failed connected out: Unable to connect, no address known for peer
```

This is because the test which was supposed to wait for addresses is
wrong: it passes when l3 knows nothing!  (`all([])` == `True`)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-17 14:09:10 +10:30
Rusty Russell
9af01b062c pytest: fix hsmtool which reports leak under address sanitizer.
Couldn't figure out why hsmtool.proc.wait(WAIT_TIMEOUT) returns 1?
hsmtool doesn't ever seem to exit status 1!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-17 14:09:10 +10:30
Rusty Russell
0741d4dea1 pytest: fix flake in test_grpc_connect_notification
Looking at the logs (and comparing a successful run), it seems the connect happens before
the connect_stream is ready, so we miss it:

```
________________________ test_grpc_connect_notification ________________________
[gw7] linux -- Python 3.8.18 /home/runner/.cache/pypoetry/virtualenvs/cln-meta-project-AqJ9wMix-py3.8/bin/python

node_factory = <pyln.testing.utils.NodeFactory object at 0x7fb08bb969d0>

    def test_grpc_connect_notification(node_factory):
        l1, l2 = node_factory.get_nodes(2)
    
        # Test the connect notification
        connect_stream = l1.grpc.SubscribeConnect(clnpb.StreamConnectRequest())
        l2.connect(l1)
    
>       for connect_event in connect_stream:

tests/test_cln_rs.py:425: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../../.cache/pypoetry/virtualenvs/cln-meta-project-AqJ9wMix-py3.8/lib/python3.8/site-packages/grpc/_channel.py:543: in __next__
    return self._next()
../../../.cache/pypoetry/virtualenvs/cln-meta-project-AqJ9wMix-py3.8/lib/python3.8/site-packages/grpc/_channel.py:960: in _next
    _common.wait(self._state.condition.wait, _response_ready)
../../../.cache/pypoetry/virtualenvs/cln-meta-project-AqJ9wMix-py3.8/lib/python3.8/site-packages/grpc/_common.py:156: in wait
    _wait_once(wait_fn, MAXIMUM_WAIT_TIMEOUT, spin_cb)
../../../.cache/pypoetry/virtualenvs/cln-meta-project-AqJ9wMix-py3.8/lib/python3.8/site-packages/grpc/_common.py:116: in _wait_once
    wait_fn(timeout=timeout)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <Condition(<unlocked _thread.RLock object owner=0 count=0 at 0x7fb089730f00>, 0)>
timeout = 0.1

    def wait(self, timeout=None):
        """Wait until notified or until a timeout occurs.
    
        If the calling thread has not acquired the lock when this method is
        called, a RuntimeError is raised.
    
        This method releases the underlying lock, and then blocks until it is
        awakened by a notify() or notify_all() call for the same condition
        variable in another thread, or until the optional timeout occurs. Once
        awakened or timed out, it re-acquires the lock and returns.
    
        When the timeout argument is present and not None, it should be a
        floating point number specifying a timeout for the operation in seconds
        (or fractions thereof).
    
        When the underlying lock is an RLock, it is not released using its
        release() method, since this may not actually unlock the lock when it
        was acquired multiple times recursively. Instead, an internal interface
        of the RLock class is used, which really unlocks it even when it has
        been recursively acquired several times. Another internal interface is
        then used to restore the recursion level when the lock is reacquired.
    
        """
        if not self._is_owned():
            raise RuntimeError("cannot wait on un-acquired lock")
        waiter = _allocate_lock()
        waiter.acquire()
        self._waiters.append(waiter)
        saved_state = self._release_save()
        gotit = False
        try:    # restore state no matter what (e.g., KeyboardInterrupt)
            if timeout is None:
                waiter.acquire()
                gotit = True
            else:
                if timeout > 0:
>                   gotit = waiter.acquire(True, timeout)
E                   Failed: Timeout >1200.0s
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-17 14:09:10 +10:30
Rusty Russell
102e22347d pytest: fix flake in test_wss_proxy
```
>       wss = BindWebSecureSocket('localhost', wss_port)

tests/test_connection.py:4584: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/test_connection.py:4570: in __init__
    self.ws.connect("wss://" + hostname + ":" + str(port))
../../../.cache/pypoetry/virtualenvs/cln-meta-project-AqJ9wMix-py3.8/lib/python3.8/site-packages/websocket/_core.py:256: in connect
    self.sock, addrs = connect(
../../../.cache/pypoetry/virtualenvs/cln-meta-project-AqJ9wMix-py3.8/lib/python3.8/site-packages/websocket/_http.py:145: in connect
    sock = _open_socket(addrinfo_list, options.sockopt, options.timeout)
../../../.cache/pypoetry/virtualenvs/cln-meta-project-AqJ9wMix-py3.8/lib/python3.8/site-packages/websocket/_http.py:232: in _open_socket
    raise err
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

addrinfo_list = [(<AddressFamily.AF_INET6: 10>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('::1', 45521, 0, 0)), (<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('127.0.0.1', 45521))]
sockopt = [], timeout = None

    def _open_socket(addrinfo_list, sockopt, timeout):
        err = None
        for addrinfo in addrinfo_list:
            family, socktype, proto = addrinfo[:3]
            sock = socket.socket(family, socktype, proto)
            sock.settimeout(timeout)
            for opts in DEFAULT_SOCKET_OPTION:
                sock.setsockopt(*opts)
            for opts in sockopt:
                sock.setsockopt(*opts)
    
            address = addrinfo[4]
            err = None
            while not err:
                try:
>                   sock.connect(address)
E                   ConnectionRefusedError: [Errno 111] Connection refused
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-17 14:09:10 +10:30
Rusty Russell
40a4d83764 pytest: don't assume gossip order in test_gossip_query_channel_range
```
        # reply_channel_range == 264
>       assert msgs == ['0108'
                        # blockhash
                        + genesis_blockhash
                        # first_blocknum, number_of_blocks, complete
                        + format(0, '08x') + format(1000000, '08x') + '01'
                        # encoded_short_ids
                        + format(len(encoded) // 2, '04x')
                        + encoded]
E       AssertionError: assert ['010806226e46111a0b59caaf126043eb5bbf28c34f3a5e332a1fc7b2b73cf188910f00000000000f42400100110000006900000100000000680000010000'] == ['010806226e46111a0b59caaf126043eb5bbf28c34f3a5e332a1fc7b2b73cf188910f00000000000f42400100110000006800000100000000690000010000']
E         At index 0 diff: '010806226e46111a0b59caaf126043eb5bbf28c34f3a5e332a1fc7b2b73cf188910f00000000000f42400100110000006900000100000000680000010000' != '010806226e46111a0b59caaf126043eb5bbf28c34f3a5e332a1fc7b2b73cf188910f00000000000f42400100110000006800000100000000690000010000'
E         Full diff:
E         - ['010806226e46111a0b59caaf126043eb5bbf28c34f3a5e332a1fc7b2b73cf188910f00000000000f42400100110000006800000100000000690000010000']
E         ?                                                                                                    ^               ^
E         + ['010806226e46111a0b59caaf126043eb5bbf28c34f3a5e332a1fc7b2b73cf188910f00000000000f42400100110000006900000100000000680000010000']
E         ?                                                                                                    ^               ^
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-17 14:09:10 +10:30
Rusty Russell
a5ba3f8f24 pytest: reduce askrene tests again for CI.
```
2024-11-14T23:50:43.8609029Z test_real_data failed (1 runs remaining out of 2).
2024-11-14T23:50:43.8609157Z 	<class 'Failed'>
2024-11-14T23:50:43.8609244Z 	Timeout >1200.0s
...
2024-11-14T23:50:43.8629292Z test_real_biases failed (1 runs remaining out of 2).
2024-11-14T23:50:43.8629407Z 	<class 'Failed'>
2024-11-14T23:50:43.8629494Z 	Timeout >1200.0s
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-17 14:09:10 +10:30
Rusty Russell
dd569b0b7a pytest: disable broken test
Seems like it's using an old version of pyln-client, which had the bug
of referring to long_desc:

```
def test_local_dir_install(node_factory):
        """Test search and install from local directory source."""
        n = get_reckless_node(node_factory)
        n.start()
        r = reckless([f"--network={NETWORK}", "-v", "source", "add",
                      "tests/data/recklessrepo/lightningd/testplugpass"],
                     dir=n.lightning_dir)
        assert r.returncode == 0
        r = reckless([f"--network={NETWORK}", "-v", "install", "testplugpass"], dir=n.lightning_dir)
        assert r.returncode == 0
>       assert 'testplugpass enabled' in r.stdout
E       assert 'testplugpass enabled' in '[2024-11-14 06:47:12,999] DEBUG: Searching for testplugpass\nfound testplugpass in source: tests/data/recklessrepo/lightningd/testplugpass\n[2024-11-14 06:47:13,031] DEBUG: entry: testplugpass.py\n[2024-11-14 06:47:13,031] DEBUG: Retrieving testplugpass from tests/data/recklessrepo/lightningd/testplugpass\n[2024-11-14 06:47:13,032] DEBUG: Install requested from InstInfo(testplugpass, tests/data/recklessrepo/lightningd/testplugpass, None, testplugpass.py, requirements.txt, None).\n[2024-11-14 06:47:13,032] DEBUG: copying local directory contents from tests/data/recklessrepo/lightningd/testplugpass\n[2024-11-14 06:47:13,038] DEBUG: cloned_src: InstInfo(testplugpass, /tmp/reckless-0721411112afw6ng7/clone, None, testplugpass.py, requirements.txt, testplugpass)\n[2024-11-14 06:47:13,038] DEBUG: using installer python3venv\n[2024-11-14 06:47:13,038] DEBUG: creating /tmp/ltests-chy6ayqu/test_local_dir_install_2/lightning-1/reckless/testplugpass\n[2024-11-14 06:47:13,038] DEBUG: creating /tmp/ltests-chy6ayqu/test_local_dir_install_2/lightning-1/reckless/testplugpass/source\n[2024-11-14 06:47:13,038] DEBUG: copying /tmp/reckless-0721411112afw6ng7/clone/testplugpass tree to /tmp/ltests-chy6ayqu/test_local_dir_install_2/lightning-1/reckless/testplugpass/source/testplugpass\n[2024-11-14 06:47:13,039] DEBUG: linking source /tmp/ltests-chy6ayqu/test_local_dir_install_2/lightning-1/reckless/testplugpass/source/testplugpass/testplugpass.py to /tmp/ltests-chy6ayqu/test_local_dir_install_2/lightning-1/reckless/testplugpass/testplugpass.py\n[2024-11-14 06:47:13,039] DEBUG: InstInfo(testplugpass, /tmp/ltests-chy6ayqu/test_local_dir_install_2/lightning-1/reckless/testplugpass, None, testplugpass.py, requirements.txt, source/testplugpass)\n[2024-11-14 06:47:20,508] DEBUG: configuring a python virtual environment (pip) in /tmp/ltests-chy6ayqu/test_local_dir_install_2/lightning-1/reckless/testplugpass/.venv\n[2024-11-14 06:47:20,508] DEBUG: virtual environment created in /tmp/ltests-chy6ayqu/test_local_dir_install_2/lightning-1/reckless/testplugpass/.venv.\nCollecting pyln-client\n  Using cached pyln_client-24.8.2-py3-none-any.whl (35 kB)\nCollecting pyln-proto>=23\n  Using cached pyln_proto-24.8.2-py3-none-any.whl (31 kB)\nCollecting pyln-bolt7>=1.0\n  Using cached pyln_bolt7-1.0.246-py3-none-any.whl (18 kB)\nCollecting base58<3.0.0,>=2.1.1\n  Using cached base58-2.1.1-py3-none-any.whl (5.6 kB)\nCollecting cryptography<43,>=42\n  Using cached cryptography-42.0.8-cp37-abi3-manylinux_2_28_x86_64.whl (3.9 MB)\nCollecting bitstring<5.0.0,>=4.1.0\n  Using cached bitstring-4.2.3-py3-none-any.whl (71 kB)\nCollecting coincurve<21,>=20\n  Using cached coincurve-20.0.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.2 MB)\nCollecting PySocks<2,>=1\n  Using cached PySocks-1.7.1-py3-none-any.whl (16 kB)\nCollecting bitarray<3.0.0,>=2.9.0\n  Using cached bitarray-2.9.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (288 kB)\nCollecting cffi>=1.3.0\n  Using cached cffi-1.17.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (446 kB)\nCollecting asn1crypto\n  Using cached asn1crypto-1.5.1-py2.py3-none-any.whl (105 kB)\nCollecting pycparser\n  Using cached pycparser-2.22-py3-none-any.whl (117 kB)\nInstalling collected packages: bitarray, asn1crypto, PySocks, pyln-bolt7, pycparser, bitstring, base58, cffi, cryptography, coincurve, pyln-proto, pyln-client\nSuccessfully installed PySocks-1.7.1 asn1crypto-1.5.1 base58-2.1.1 bitarray-2.9.3 bitstring-4.2.3 cffi-1.17.1 coincurve-20.0.0 cryptography-42.0.8 pycparser-2.22 pyln-bolt7-1.0.246 pyln-client-24.8.2 pyln-proto-24.8.2\ndependencies installed successfully\n[2024-11-14 06:47:37,424] DEBUG: virtual environment for cloned plugin: .venv\n[2024-11-14 06:47:37,746] DEBUG: plugin testing error:\n[2024-11-14 06:47:37,746] DEBUG:   Traceback (most recent call last):\n[2024-11-14 06:47:37,746] DEBUG:     File "/tmp/ltests-chy6ayqu/test_local_dir_install_2/lightning-1/reckless/testplugpass/testplugpass.py", line 9, in <module>\n[2024-11-14 06:47:37,746] DEBUG:       runpy.run_module("testplugpass", {}, "__main__")\n[2024-11-14 06:47:37,746] DEBUG:     File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/runpy.py", line 210, in run_module\n[2024-11-14 06:47:37,746] DEBUG:       return _run_code(code, {}, init_globals, run_name, mod_spec)\n[2024-11-14 06:47:37,746] DEBUG:     File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/runpy.py", line 87, in _run_code\n[2024-11-14 06:47:37,746] DEBUG:       exec(code, run_globals)\n[2024-11-14 06:47:37,746] DEBUG:     File "/tmp/ltests-chy6ayqu/test_local_dir_install_2/lightning-1/reckless/testplugpass/source/testplugpass/testplugpass.py", line 25, in <module>\n[2024-11-14 06:47:37,746] DEBUG:       plugin.run()\n[2024-11-14 06:47:37,746] DEBUG:     File "/tmp/ltests-chy6ayqu/test_local_dir_install_2/lightning-1/reckless/testplugpass/.venv/lib/python3.8/site-packages/pyln/client/plugin.py", line 877, in run\n[2024-11-14 06:47:37,747] DEBUG:       return self.print_usage()\n[2024-11-14 06:47:37,747] DEBUG:     File "/tmp/ltests-chy6ayqu/test_local_dir_install_2/lightning-1/reckless/testplugpass/.venv/lib/python3.8/site-packages/pyln/client/plugin.py", line 831, in print_usage\n[2024-11-14 06:47:37,747] DEBUG:       doc = method.long_desc if method.long_desc is not None else "No documentation found"\n[2024-11-14 06:47:37,747] DEBUG:   AttributeError: \'Method\' object has no attribute \'long_desc\'\n[2024-11-14 06:47:37,747] ERROR: plugin testing failed\n[2024-11-14 06:47:37,937] WARNING: testplugpass: installation aborted\n'
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-17 14:09:10 +10:30
Rusty Russell
44f63a360b build: fix race where channeld_fakenet doesn't depend on external modules.
Because it wasn't in ALL_OBJS.  Copy the Makefile pattern!

```
Submodule 'src/secp256k1' (https://github.com/ElementsProject/secp256k1-zkp.git) registered for path 'external/libwally-core/src/secp256k1'
Cloning into '/home/runner/work/lightning/lightning/external/libwally-core/src/secp256k1'...
cc tests/plugins/channeld_fakenet.c
In file included from ./bitcoin/script.h:4,
                 from tests/plugins/channeld_fakenet.c:14:
./bitcoin/signature.h:6:10: fatal error: secp256k1.h: No such file or directory
    6 | #include <secp256k1.h>
      |          ^~~~~~~~~~~~~
compilation terminated.
make: *** [Makefile:301: tests/plugins/channeld_fakenet.o] Error 1
make: *** Waiting for unfinished jobs....
Submodule path 'external/libwally-core/src/secp256k1': checked out
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-17 14:09:10 +10:30
Rusty Russell
4f4bf7730e pytest: fix test_onchain_unwatch under valgrind.
valgrind slowed things down so the HTLC was resolved, so we didn't use
an anchor.  Make that case consistent.

```
    def test_onchain_unwatch(node_factory, bitcoind, chainparams):
        """Onchaind should not watch random spends"""
        # We track channel balances, to verify that accounting is ok.
        coin_mvt_plugin = os.path.join(os.getcwd(), 'tests/plugins/coin_movements.py')
        l1, l2 = node_factory.line_graph(2, opts={'plugin': coin_mvt_plugin})
        channel_id = first_channel_id(l1, l2)
    
        l1.pay(l2, 200000000)
    
        l1.rpc.dev_fail(l2.info['id'])
        l1.daemon.wait_for_log('Failing due to dev-fail command')
        l1.wait_for_channel_onchain(l2.info['id'])
    
        # Make sure we see anchor too
>       l1.bitcoin.generate_block(1, wait_for_mempool=2)

tests/test_closing.py:1769: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
contrib/pyln-testing/pyln/testing/utils.py:477: in generate_block
    wait_for(lambda: len(self.rpc.getrawmempool()) >= wait_for_mempool)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

success = <function BitcoinD.generate_block.<locals>.<lambda> at 0x7f23f909c310>
timeout = 180

    def wait_for(success, timeout=TIMEOUT):
        start_time = time.time()
        interval = 0.25
        while not success():
            time_left = start_time + timeout - time.time()
            if time_left <= 0:
>               raise ValueError("Timeout while waiting for {}".format(success))
E               ValueError: Timeout while waiting for <function BitcoinD.generate_block.<locals>.<lambda> at 0x7f23f909c310>
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-17 14:09:10 +10:30
Rusty Russell
4f5865167e pytest: fix spurious error in test_important_plugin
```
**BROKEN** plugin-spenderp: Lost connection to the RPC socket.
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-17 14:09:10 +10:30
Rusty Russell
ecc3a00839 pytest: don't try exposesecret test under valgrind.
There's a known issue with hsm passwords and valgrind:

```
        write_all(master_fd, (password + '\n').encode("utf-8"))
>       l1.daemon.wait_for_log("Server started with public key")

tests/test_plugin.py:4526: 
...
                        if self.is_in_log(r):
                            print("({} was previously in logs!)".format(r))
>                   raise TimeoutError('Unable to find "{}" in logs.'.format(exs))
E                   TimeoutError: Unable to find "[re.compile('Server started with public key')]" in logs.
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-17 14:09:10 +10:30
Rusty Russell
b7d77b49df hsmtool.c: extend generatehsm to allow command-line args.
Based on the patch by bstin <barry.github@capsmx.com>, which added a separate command,
this simply extends "generatehsm" to allow more options.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: hsmtool: generatehsm can run non-interactive, taking options on the cmdline.
2024-11-15 12:48:44 +10:30
Rusty Russell
0dc1c5a061 onchaind: explicit ack for onchaind_spent method.
This means it always tells us explicitly whether to keep watching or not,
and we know it's processed it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-13 14:44:03 +10:30
Rusty Russell
5c81b0c3dc lightningd: use block-at-a-time replay for onchaind on restart.
And we hook in the replay watch code.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: `onchaind` can miss conclusion of final txs in some cases, will now replay independently.
2024-11-13 14:44:03 +10:30
Rusty Russell
3ad7167cfa fuzz: fix up compilation with hsmtool change.
```
tests/fuzz/fuzz-hsm_encryption.c:28:80: error: passing 'char **' to parameter of type 'const char **' discards qualifiers in nested pointer types [-Werror,-Wincompatible-pointer-types-discards-qualifiers]
                assert(!hsm_secret_encryption_key_with_exitcode(passphrase, &encryption_key, &emsg));
                                                                                             ^~~~~
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-13 13:24:06 +10:30
Rusty Russell
d3ad68c647 pytest: reduce time for askrene test_real_data test.
Timing out under CI (1800 seconds!)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-13 13:24:06 +10:30
Rusty Russell
8293352425 config: add the ability for plugins to specify that config values should be concealed.
And use it for `exposesecret-passphrase`.  This is probably overly
cautious, but it makes me feel a little better that we won't leak it
to someone with read-only access.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-13 08:49:55 +10:30
Rusty Russell
e261301301 exposesecret: new plugin and command to get hsm_secret.
Being able to back up the hsm_secret is critical, but you cannot do
this through a UI, because of course we do not allow such access.
People have lost funds because they didn't back up.

This allows access to the hsm_secret if you use a password set in the
config file.  (If it's not set, the command does not work).  This is a
compromise, of course.

Changelog-Added: `exposesecret` command for encouraging hsm_secret backups.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-11-13 08:49:55 +10:30