Commit graph

15825 commits

Author SHA1 Message Date
ShahanaFarooqui
127ceb6ddb ci: Lock MacOS and poetry versions 2025-01-09 11:15:05 +01:00
ShahanaFarooqui
9c34bd6108 ci: Lock Ubuntu version on 22.04
These workflows are failing due to `ubuntu-latest` being updated from version `22.04` to `24.04`.

Reference: https://github.com/actions/runner-images/issues/10636

Changelog-None.
2025-01-09 11:15:05 +01:00
Rusty Russell
98679aa6cf pytest: fix flake in test_restorefrompeer.
Just because we've seen the block doesn't mean onchaind has finished
starting up.

```
 _____________________________ test_restorefrompeer _____________________________
[gw0] linux -- Python 3.10.15 /home/runner/.cache/pypoetry/virtualenvs/cln-meta-project-AqJ9wMix-py3.10/bin/python

node_factory = <pyln.testing.utils.NodeFactory object at 0x7fb8f3887f70>
bitcoind = <pyln.testing.utils.BitcoinD object at 0x7fb8f3886f50>

    @unittest.skipIf(os.getenv('TEST_DB_PROVIDER', 'sqlite3') != 'sqlite3', "deletes database, which is assumed sqlite3")
    def test_restorefrompeer(node_factory, bitcoind):
        """
        Test restorefrompeer
        """
        l1, l2 = node_factory.get_nodes(2, [{'broken_log': 'ERROR: Unknown commitment #.*, recovering our funds!',
                                             'experimental-peer-storage': None,
                                             'may_reconnect': True,
                                             'allow_bad_gossip': True},
                                            {'experimental-peer-storage': None,
                                             'may_reconnect': True}])
    
        l1.rpc.connect(l2.info['id'], 'localhost', l2.port)
    
        c12, _ = l1.fundchannel(l2, 10**5)
        assert l1.daemon.is_in_log('Peer storage sent!')
        assert l2.daemon.is_in_log('Peer storage sent!')
    
        l1.stop()
        os.unlink(os.path.join(l1.daemon.lightning_dir, TEST_NETWORK, "lightningd.sqlite3"))
    
        l1.start()
        assert l1.daemon.is_in_log('Server started with public key')
    
        # If this happens fast enough, connect fails with "disconnected
        # during connection"
        try:
            l1.rpc.connect(l2.info['id'], 'localhost', l2.port)
        except RpcError as err:
            assert "disconnected during connection" in err.error['message']
    
        l1.daemon.wait_for_log('peer_in WIRE_YOUR_PEER_STORAGE')
    
        assert l1.rpc.restorefrompeer()['stubs'][0] == _['channel_id']
    
        l1.daemon.wait_for_log('peer_out WIRE_ERROR')
        l2.daemon.wait_for_log('State changed from CHANNELD_NORMAL to AWAITING_UNILATERAL')
    
        bitcoind.generate_block(5, wait_for_mempool=1)
        sync_blockheight(bitcoind, [l1, l2])
    
        l1.daemon.wait_for_log(r'All outputs resolved.*')
        wait_for(lambda: l1.rpc.listfunds()["channels"][0]["state"] == "ONCHAIN")
    
        # Check if funds are recovered.
        assert l1.rpc.listfunds()["channels"][0]["state"] == "ONCHAIN"
>       assert l2.rpc.listfunds()["channels"][0]["state"] == "ONCHAIN"
E       AssertionError: assert 'FUNDING_SPEND_SEEN' == 'ONCHAIN'
E         - ONCHAIN
E         + FUNDING_SPEND_SEEN

tests/test_misc.py:3044: AssertionError
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-12-20 15:36:07 +10:30
Rusty Russell
9758b05bd0 pytest: fix flake in test_penalty_htlc_tx_fulfill
Make sure balancing payment is fully cleared before trying to get a routeL

```
    def test_penalty_htlc_tx_fulfill(node_factory, bitcoind, chainparams, anchors):

        # now we send one 'sticky' htlc: l4->l1
        amt = 10**8 // 2
        sticky_inv = l1.rpc.invoice(amt, '2', 'sticky')
>       route = l4.rpc.getroute(l1.info['id'], amt, 1)['route']

tests/test_closing.py:1232:

>           raise RpcError(method, payload, resp['error'])
E           pyln.client.lightning.RpcError: RPC call failed: method: getroute, payload: {'id': '0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518', 'amount_msat': 50000000, 'riskfactor': 1, 'cltv': 9}, error: {'code': 205, 'message': 'Could not find a route'}
```
2024-12-20 15:36:07 +10:30
Rusty Russell
69bfa6f5b1 channeld_fakenet: don't be as brute-force trying to derive keys.
Keep a proper cache of all possible ones.  I think this may be the
timeout problem: according to the logs, channeld_fakenet stops responding
and thus HTLCs eventually time out.

```
```
2024-12-16T23:16:16.4874420Z lightningd-1 2024-12-16T22:45:14.068Z UNUSUAL 022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59-channeld-chan#1: Adding HTLC 18446744073709551615 too slow: killing connection
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-12-20 15:36:07 +10:30
Rusty Russell
4b283eb96e pytest: fix flake in test_gossip_throttle
We can get the reply_short_channel_ids_end in the messages when
we make a query:

```
2024-11-29T07:39:28.8550652Z         time_fast = time.time() - start_fast
2024-11-29T07:39:28.8551067Z         assert time_fast < 2
2024-11-29T07:39:28.8551487Z         out3 = [m for m in out3 if not m.startswith(b'0109')]
2024-11-29T07:39:28.8552158Z >       assert set(out1) == set(out3)
...
2024-11-29T07:39:28.8675516Z E         Extra items in the right set:
2024-11-29T07:39:28.8675887Z E         b'010606226e46111a0b59caaf126043eb5bbf28c34f3a5e332a1fc7b2b73cf188910f01'
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-12-20 15:36:07 +10:30
Rusty Russell
3a0e3a1591 pytest: fix test in test_gossip_pruning
It's possible that listchannels doesn't show the channel yet:

```
    
        l1.rpc.connect(l2.info['id'], 'localhost', l2.port)
        l2.rpc.connect(l3.info['id'], 'localhost', l3.port)
    
        scid1, _ = l1.fundchannel(l2, 10**6)
        scid2, _ = l2.fundchannel(l3, 10**6)
    
        mine_funding_to_announce(bitcoind, [l1, l2, l3])
>       l1_initial_cupdate_timestamp = only_one(l1.rpc.listchannels(source=l1.info['id'])['channels'])['last_update']

tests/test_gossip.py:43: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

arr = []

    def only_one(arr):
        """Many JSON RPC calls return an array; often we only expect a single entry
        """
>       assert len(arr) == 1
E       AssertionError
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-12-20 15:36:07 +10:30
Rusty Russell
e998a72ef6 lightningd: fix overzealous memleak detection in json_add_datastore.
The lifetime is bound to the response, but memleak gets upset if it
sees it.  Use the standard take() pattern insteas.

```
lightningd-5 2024-11-28T05:02:27.503Z **BROKEN** lightningd: MEMLEAK: 0x55b58f329338
lightningd-5 2024-11-28T05:02:27.503Z **BROKEN** lightningd:   label=common/utils.c:150:char[]
lightningd-5 2024-11-28T05:02:27.504Z **BROKEN** lightningd:   alloc:
lightningd-5 2024-11-28T05:02:27.585Z **BROKEN** lightningd:     ccan/ccan/tal/tal.c:488 (tal_alloc_)
lightningd-5 2024-11-28T05:02:27.586Z **BROKEN** lightningd:     ccan/ccan/tal/tal.c:517 (tal_alloc_arr_)
lightningd-5 2024-11-28T05:02:27.586Z **BROKEN** lightningd:     ccan/ccan/tal/tal.c:861 (tal_dup_)
lightningd-5 2024-11-28T05:02:27.586Z **BROKEN** lightningd:     common/utils.c:150 (utf8_str)
lightningd-5 2024-11-28T05:02:27.589Z **BROKEN** lightningd:     lightningd/datastore.c:23 (json_add_datastore)
lightningd-5 2024-11-28T05:02:27.589Z **BROKEN** lightningd:     lightningd/datastore.c:205 (json_datastore)
lightningd-5 2024-11-28T05:02:27.590Z **BROKEN** lightningd:     lightningd/jsonrpc.c:808 (command_exec)
lightningd-5 2024-11-28T05:02:27.590Z **BROKEN** lightningd:     lightningd/jsonrpc.c:954 (rpc_command_hook_final)
lightningd-5 2024-11-28T05:02:27.590Z **BROKEN** lightningd:     lightningd/plugin_hook.c:196 (plugin_hook_call_next)
lightningd-5 2024-11-28T05:02:27.591Z **BROKEN** lightningd:     lightningd/plugin_hook.c:183 (plugin_hook_callback)
lightningd-5 2024-11-28T05:02:27.593Z **BROKEN** lightningd:     lightningd/plugin.c:663 (plugin_response_handle)
lightningd-5 2024-11-28T05:02:27.593Z **BROKEN** lightningd:     lightningd/plugin.c:775 (plugin_read_json_one)
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-12-20 15:36:07 +10:30
Rusty Russell
5c53ed73fb CI: disable --force-flaky.
This should help us fix the flakes.  Dealing with them was a major
headache at the end of the last release (and they covered up real bugs!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-12-20 15:36:07 +10:30
Rusty Russell
fa10323847 Changelog: update to match 24.11.1 branch now that's released.
Changelog-None: changelog updated.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-12-19 10:52:35 +10:30
Rusty Russell
2c8d9d0deb pytest: actually test xpay/pay return similarity.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-12-18 14:04:14 +10:30
Rusty Russell
d7adf06dfe xpay: created_at response from pay compatibility is a number, not integer.
Reported-by: @hMsats
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-12-18 14:04:14 +10:30
Rusty Russell
3e9a4fc750 xpay: simplify printing of destination.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-12-18 14:04:14 +10:30
Lagrang3
545220dfbc xpay: add destination to output
Changelog-Add: xpay: add destination to xpay success output when xpay-handle-pay is active.

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2024-12-18 14:04:14 +10:30
Rusty Russell
84f30b12f7 pytest: bonus test to make sure xpay uses zeroconf channels correctly.
It needs to use the channel alias here, and it does.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-12-17 15:59:30 +10:30
Rusty Russell
a2f58a28ba lightningd: injectpaymentonion can use scids of unannounced channels.
Cut & paste from the forwarding code, where we don't let onions use the
unannounced scids.  Obviously local commands can use them.

Reported-by: @michael1011
Changelog-Fixed: JSON-RPC: xpay now works through unannounced channels.
2024-12-17 15:59:30 +10:30
Rusty Russell
92c45712d2 pytest: test xpay using unannounced channels.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-12-17 15:59:30 +10:30
Rusty Russell
d26ea6673d xpay: more accurately reflect pay when xpay-handle-pay is set.
Note that the slight code reorder changes the JSON order, which is generally
undefined, but our doc checker is very strict!

Changelog-Changed: `xpay` now gives the same JSON success return as documented by `pay` when `xpay-handle-pay` is set.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes: https://github.com/ElementsProject/lightning/issues/7923
2024-12-17 15:49:03 +10:30
Rusty Russell
428c76068c xpay: emulate maxfeepercent and exemptfee when xpay-handle-pay used
maxfeepercent is use by Zeus, so let's make that work.

maxfee is more precise, so it's the only xpay option (maxfee was added
to pay later).

[ Fix to ppm logic by Lagrang3, thanks! --RR ]

Fixes: https://github.com/ElementsProject/lightning/issues/7926
Changelog-Changed: JSON-RPC: With `xpay-handle-pay` set, xpay will now be used even if `pay` uses maxfeeprecent or exemptfee parameters (e.g. Zeus)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-12-17 10:54:31 +10:30
Rusty Russell
cf22762c8f xpay: tell injectpaymentonion what the amount being delivered to destination is.
This means that it gets shown in listsendpays: omitting this broke spark, apparently!

Changelog-Changed: `xpay` now populates more fields, so `listsendpays` and `listpays` show `destination` and `amount_msat` fields for xpay payments.
Fixes: https://github.com/ElementsProject/lightning/issues/7881
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-12-17 08:14:45 +10:30
Rusty Russell
8202929a00 lightningd: populate listsendpays destination from injectpaymentonion if we can.
If they give us the invstring, we can at least set who signed the invoice.  Of course,
it might not be a real node_id (with blinded paths).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-12-17 08:14:45 +10:30
Rusty Russell
80c43ec97d injectpaymentionion: allow specification of actual amount which reaches destination.
This appears in listsendpays / listpays, and is useful information (if we know!).

This doesn't fix old payments, but means that xpay can use this for new payments.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-12-17 08:14:45 +10:30
Rusty Russell
aaca16d313 doc: fix injectpaymentonion schema.
I used an unknown arg and it didn't complain.  injectpaymentonion's schema predated
the sweep which fixed up use of "additionalProperties".

Indeed, we were missing some properties!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-12-17 08:14:45 +10:30
Lagrang3
7be96ae8bc lightning-cli: fix malformed response bug
There was a wrong assumption that the number of bytes read
by `cli_read` would get us for each correctly read token
two extra CR characters. As a matter of fact one could read
enough characters to parse the first token, but the two
extra CR characters are not guaranteed.

```
==143570== Memcheck, a memory error detector
==143570== Copyright (C) 2002-2022, and GNU GPL'd, by Julian Seward et al.
==143570== Using Valgrind-3.19.0 and LibVEX; rerun with -h for copyright info
==143570== Command: /home/lagrange/BACKUP/l4-appdata/github/lagrang3/lightning/cli/lightning-cli --lightning-dir=/tmp/askrene_benchmark/lightning -k getroutes source=032ed0d87ba2bd68e3a386717cf2faaae4fa6d6da247986b1997113930e4f841d5 destination=03b2f16bf472dd03c55c2ce9910aab717321db4489cd87df5225adadb08031da4b amount_msat=100000sat final_cltv=6 layers=[] maxfee_msat=500sat
==143570==
==143570== Invalid read of size 1
==143570==    at 0x484A430: memmove (vg_replace_strmem.c:1382)
==143570==    by 0x10C3D2: main (lightning-cli.c:871)
==143570==  Address 0x4a62f80 is 0 bytes after a block of size 1,040 alloc'd
==143570==    at 0x48407B4: malloc (vg_replace_malloc.c:381)
==143570==    by 0x11402E: allocate (tal.c:256)
==143570==    by 0x11471E: tal_alloc_ (tal.c:473)
==143570==    by 0x1147EA: tal_alloc_arr_ (tal.c:517)
==143570==    by 0x10C206: main (lightning-cli.c:816)
==143570==
==143570== Invalid read of size 1
==143570==    at 0x484A43D: memmove (vg_replace_strmem.c:1382)
==143570==    by 0x10C3D2: main (lightning-cli.c:871)
==143570==  Address 0x4a62f81 is 1 bytes after a block of size 1,040 alloc'd
==143570==    at 0x48407B4: malloc (vg_replace_malloc.c:381)
==143570==    by 0x11402E: allocate (tal.c:256)
==143570==    by 0x11471E: tal_alloc_ (tal.c:473)
==143570==    by 0x1147EA: tal_alloc_arr_ (tal.c:517)
==143570==    by 0x10C206: main (lightning-cli.c:816)
==143570==
==143570== Invalid write of size 1
==143570==    at 0x484A433: memmove (vg_replace_strmem.c:1382)
==143570==    by 0x10C3D2: main (lightning-cli.c:871)
==143570==  Address 0x4a62f80 is 0 bytes after a block of size 1,040 alloc'd
==143570==    at 0x48407B4: malloc (vg_replace_malloc.c:381)
==143570==    by 0x11402E: allocate (tal.c:256)
==143570==    by 0x11471E: tal_alloc_ (tal.c:473)
==143570==    by 0x1147EA: tal_alloc_arr_ (tal.c:517)
==143570==    by 0x10C206: main (lightning-cli.c:816)
```

Changelog-Fixed: lightning-cli: fix "malformed response" bug

Signed-off-by: Lagrang3 <lagrang3@protonmail.com>
2024-12-16 10:36:32 +10:30
Rusty Russell
73415d35c9 common: don't send trace messages by default, don't ratelimit at all.
We ratelimited DEBUG messages, but that can be annoying and cause us to miss things.
We demoted the worst offenders in the last release, to TRACE level.

Now, only log trace if it's wanted, and never suppress DEBUG.

Changelog-Changed: Logging: we no longer suppress DEBUG messages from subdaemons.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes: https://github.com/ElementsProject/lightning/issues/7917
2024-12-16 09:48:51 +10:30
Lakshya Singh
7703cf7fec splice: alpine build warning causing failure
header sys/errno.h gets re-directed to errno.h leading to warning and
then failure so instead directly referencing the header

Changelog-None: change header

Signed-off-by: Lakshya Singh <lakshay.singh1108@gmail.com>
2024-12-16 09:36:17 +10:30
Marcel Hernandez
d6bd61157e Changelog-Fixed: xpay no longer logs "Got command" at info level. 2024-12-15 21:53:43 +10:30
daywalker90
dcb6f2cb2e rust: bump versions for 24.11 release
Changelog-None
2024-12-10 09:37:30 +01:00
Rusty Russell
cb1bd82c86 Final changes for v24.11
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-12-09 14:54:50 +10:30
Rusty Russell
8547cc15c8 doc: fix final release make line.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-12-09 14:54:50 +10:30
Rusty Russell
cc07f9afc6 v24.11rc4: hopefully the final Release Candidate
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-12-06 15:35:11 +10:30
Christian Decker
1adbd70678 logs: Correct an sign mismatch printing the depth change
We are checking if `txw->depth` is `-1` and then print it, when we
clearly want `depth` instead.

Changelog-Fixed logs: When printing depths some unsigned numbers could overflow
2024-12-06 15:12:51 +10:30
Rusty Russell
f0c5ea2e1e doc: document and test the onionmessage_forward_fail notification.
Doing exactly what we expect to do: initiate a connection and then
forward the message.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-12-05 17:38:16 +10:30
Rusty Russell
e655f69b73 lightningd: notification for onionmessage_forward_fail.
Changelog-Added: Plugins: new notification `onionmessage_forward_fail`.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-12-05 17:38:16 +10:30
Rusty Russell
8e4b589a9e connectd: message to tell lightningd if we couldn't forward an onion message.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-12-05 17:38:16 +10:30
Rusty Russell
d196b9bb53 doc: document (and test) the injectonionmessage API.
It's actually tested by fetchinvoice, but doing an explicit test in Python
allows for schema checking!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: `injectonionmessage` API simplified and documented.
2024-12-05 17:38:16 +10:30
Rusty Russell
57b199bb16 lightningd: simplify injectonionmessage API.
It's undocumented and only used in one place, so we can change it (it
was new in 24.08).

We really want to be able to just handle a raw onionmessage: this allows
oblivious sending of messages, but also, in combination with the coming
onionmessage_forward_fail notification, allows us to connect then
reinject.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-12-05 17:38:16 +10:30
fanquake
e00e7053fd doc: minor changes to repro.md
Clarify that the ability to reproducibly build a piece of software has
nothing to do with being a maintainer.

Minor typo/grammar fixes.

Changelog-None.
2024-12-05 13:12:39 +10:30
fanquake
ff471679b9 doc: remove use of sudo from repro.md
`sudo` is not required to complete the steps outlined here.

Changelog-None.
2024-12-05 13:12:39 +10:30
Rusty Russell
b520543867 gossipd: log at trace, not debug for regular messages.
See: https://github.com/ElementsProject/lightning/issues/7899

A node with 23 connections gets far too many debug messages.

Changelog-Fixed: `gossipd` now does logging at trace, not debug level.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-12-05 11:43:50 +10:30
ShahanaFarooqui
2791c60bb2 ci: Removed branch testing due to checkout confusion between branch vs tag
- Removed branch testing as it is not required. Point releases will trigger the event due to `tag` push even when it is not from master branch.

- Added version in inputs for testing

- Added missing `inputs` tag

Changelog-None.
2024-12-03 21:32:22 +10:30
Rusty Russell
f2b4b507fb CHANGELOG: update for rc3.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-None: The changelog itself was changed.
2024-12-03 12:25:28 +10:30
Se7enZ
42d43b2708 build: Prepare release draft in Github actions. 2024-12-03 11:56:23 +10:30
Se7enZ
1f31aa73fa build: Sign aggregate release artifact from Github. 2024-12-03 11:56:23 +10:30
Se7enZ
79273220f4 build: Add architecture to Ubuntu repro artifacts ([#6228]).
Changelog-Changed: Added architecture identifier to Ubuntu release
artifacts. ([#6228])
2024-12-03 11:56:23 +10:30
Se7enZ
f2716092be build: Remove unused Dockerfiles, scripts and Github action. 2024-12-03 11:56:23 +10:30
Se7enZ
659d607fc4 doc: Update release checklist for CI build. 2024-12-03 11:56:23 +10:30
Se7enZ
f789f9cb21 build: Allow for signing releases without building a zip.
Adds a `--without-zip` parameter default `false` which optionally
skips the archive, to allow for signing releases from CI.
2024-12-03 11:56:23 +10:30
Se7enZ
b9b81c6918 build: Matrix strategy for release builds ([#7776]).
Also modifies the `build-release.sh` script to optionally
accept specific distribution targets for Ubuntu releases.
2024-12-03 11:56:23 +10:30
Se7enZ
c99b72da1c build: Release build automation ([#7776]).
Changelog-None
2024-12-03 11:56:23 +10:30