Commit graph

12363 commits

Author SHA1 Message Date
Christian Decker
69e37a8865 gci: Re-add tests of pre-compiled binaries 2023-01-16 13:53:48 +10:30
Christian Decker
10abb620a8 gci: Clone BOLTs repository in pre-build checks 2023-01-16 13:53:48 +10:30
Christian Decker
6d67eb934d py: Ignore missing whitespace after keyword for now 2023-01-16 13:53:48 +10:30
Christian Decker
4d668e76a0 gci: Split out the stages better
We want to compile with one set of dependencies, and run the tests
with another. This also helps us cut down on the times we compile CLN
itself, by sharing them among stages, and simplifies the logic of each
stage to have one specific goal.
2023-01-16 13:53:48 +10:30
Christian Decker
a00190dce5 gci: Update Github Actions steps to their latest versions
They were causing warnings.
2023-01-16 13:53:48 +10:30
Christian Decker
8c075c4cda gci: Stop uploading unit test results
We never used them, and they were creating distracting warnings.
2023-01-16 13:53:48 +10:30
Christian Decker
1fbf774e04 gci: Split out the lnprototest from the larger CI run
The lnprototests are often blocking PRs, due to them failing, but we
can't restart them when valgrind runs are still ongoing, since they'd
also restart. Splitting allows us to rerun them selectively and waste
less time.

Ideally we'd just fix them, but I am by no means knowledgeable enough
to fix them now.
2023-01-16 13:53:48 +10:30
Rusty Russell
9d5841a0e6 pytest: fix flake in tests/test_closing.py::test_closing_specified_destination
The close call can fail, since we already unilaterally closed since we mined blocks
too fast:

```
2023-01-14T01:00:10.2502199Z E           pyln.client.lightning.RpcError: RPC call failed: method: close, payload: ['107x1x1', None, 'bcrt1qeyyk6sl5pr49ycpqyckvmttus5ttj25pd0zpvg'], error: {'code': -32602, 'message': "Short channel ID not active: '107x1x1'"}
...
2023-01-14T01:00:10.5288050Z lightningd-4 2023-01-14T00:59:59.650Z UNUSUAL 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-chan#1: Peer permanent failure in CHANNELD_NORMAL: Fulfilled HTLC 0 SENT_REMOVE_COMMIT cltv 113 hit deadline
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-14 21:17:45 +10:30
Rusty Russell
932ca9e91f lightningd: deprecate p2sh-segwit addresses for newaddr addresstype
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: JSON-RPC: `newaddr`: `addresstype` `p2sh-segwit` (use default, or `bech32`).
2023-01-14 21:17:45 +10:30
Rusty Russell
2f36c03307 pytest: use bech32 addresses everywhere.
We used to create some p2sh-segwit addresses just to mix things up.  This
streamlines back to just bech32.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-14 21:17:45 +10:30
Rusty Russell
85fc46f76c pyln-testing: don't default openchannel and fundwallet to p2sh-segwit, use bech32.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-14 21:17:45 +10:30
Christian Decker
e5d384a427 gossip: Do not send warnings if we fail to parse a channel_update
We removed a warning about the channel_update being malformed since
the warning could cause lnd to disconnect (seems they treat
channel-unrelated warnings as fatal?). This was caused by lnd not
enforcing the `htlc_maximum`, thus the parsing would fail. We can
re-add the warning once our assumption that `htlc_maximum` being set
is valid.

Changelog-Fixed: gossip: We no longer send warning that lnd would not understand if we get outdated gossip
2023-01-14 12:39:10 +10:30
Rusty Russell
2b5f4d14d1 CI: fix schema diff check.
GitHub's master branch is called "main" apparently.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-14 10:13:28 +10:30
Riccardo Casatta
6b977f0292 cln-grpc: update listpeers json fixing tests 2023-01-13 21:42:27 +01:00
Rusty Russell
f1373fd98c doc: remove manual field descriptions from listpeerchannels(7).
Some are best copied into the schema, but some are already
out-of-date, so cleanest to remove them and rely on the generated (and
thus, checked!) fields.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-13 10:42:42 +10:30
Vincenzo Palazzo
6c0b9b0c78 lightningd: deprecate listpeers.channels
Changelog-Deprecated: JSON-RPC: `listpeers` `channels` array: use `listpeerchannels`

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2023-01-13 10:42:42 +10:30
Rusty Russell
9ffaab7d22 pytest: fix race in test_bookkeeping_closing_subsat_htlcs
With the next change (which, as a side-effect, speeds up listpeers),
we seem to hit a race in this test.  The bookkeeper doesn't get to
process the final payment before the node is shutdown.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-13 10:42:42 +10:30
Vincenzo Palazzo
a2347c7452 tests: use listpeerchannels. 2023-01-13 10:42:42 +10:30
Vincenzo Palazzo
1fa32333b9 tests/utils.py: use listpeerchannels. 2023-01-13 10:42:42 +10:30
Vincenzo Palazzo
f08d3516f7 contrib/pyln-testing: use listpeerchannels. 2023-01-13 10:42:42 +10:30
Vincenzo Palazzo
c488561282 plugins/topology: use listpeerchannels. 2023-01-13 10:42:42 +10:30
Rusty Russell
a56c890ae5 plugins: use listpeerchannels instead of listpeers.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-13 10:42:42 +10:30
Rusty Russell
ff2d7e6833 pay: use json_to_listpeers_channels() for local_channel_hints.
Don't parse the listpeers.channels output ourselves: with two extra fields
we can simply reuse json_to_listpeers_channels().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-13 10:42:42 +10:30
Rusty Russell
5d5b9c6812 libplugin: don't return unopened channels from json_to_listpeers_channels().
This way we always have an SCID and a direction.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-13 10:42:42 +10:30
Rusty Russell
57dcf68c0b plugins/libplugin: flatten return from json_to_listpeers_result.
Instead of returning a peers -> channels heirarchy, return (as callers
want!) a flat array of channels.

This is actually most of the transition work to make them work with
listpeerchannels.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-13 10:42:42 +10:30
Vincenzo Palazzo
cb5ee7e49c plugins: make bookkeeper use the new listpeerchannels command. 2023-01-13 10:42:42 +10:30
Vincenzo Palazzo
6fa904b4fb lightningd: add listpeerchannels command
Changelog-Added: JSON-RPC: new command `listpeerchannels` now contains information on direct channels with our peers.

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2023-01-13 10:42:42 +10:30
Rusty Russell
1d8b899551 lightningd: prepare internal json routines for listpeerchannels.
We're soon going to call json_add_unsaved_channel and
json_add_uncommitted_channel from a new place, where we want the peer
state directly included.

Based on patch by @vincenzopalazzo.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-13 10:42:42 +10:30
Greg Sanders
300f732bbe proposal_meets_depth tracked output always has a proposal 2023-01-12 14:17:38 +10:30
Rusty Russell
6044184323 lightningd: don't call memcpy with NULL.
Thanks C committee!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-12 11:44:10 +10:30
Rusty Russell
5dfcd15782 all: no longer need to call htable_clear to free htable contents.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-12 11:44:10 +10:30
Rusty Russell
f07e37018d setup: make all htables use tal.
This makes them easier to clean up.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-12 11:44:10 +10:30
Rusty Russell
0d93841cc7 plugins/command: ensure htables are always tal objects.
We want to change the htable allocator to use tal, which will need
this.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-12 11:44:10 +10:30
Rusty Russell
3c4ce9e448 plugins/pay: ensure htables are always tal objects.
We want to change the htable allocator to use tal, which will need
this.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-12 11:44:10 +10:30
Rusty Russell
94e8ce030a gossipd: use pointer to hash table for channels in node.
We actually reduce the size of struct node by 1 pointer, which
is mildly smaller.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-12 11:44:10 +10:30
Rusty Russell
4200371020 gossipd: ensure htables are always tal objects.
We want to change the htable allocator to use tal, which will need
this.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-12 11:44:10 +10:30
Rusty Russell
851cbf6c83 memleak: prepare for htable to be a tal object.
Since it gets resized during traverse, we would crash by
keeping a pointer to the old one.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-12 11:44:10 +10:30
Rusty Russell
81e57dce52 connectd: ensure htables are always tal objects.
We want to change the htable allocator to use tal, which will need
this.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-12 11:44:10 +10:30
Rusty Russell
4a570c9419 gossmap: ensure htables are always tal objects.
We want to change the htable allocator to use tal, which will need
this.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-12 11:44:10 +10:30
Rusty Russell
763d02e424 lightningd: ensure htlc htables are always tal objects.
We want to change the htable allocator to use tal, which will need
this.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-12 11:44:10 +10:30
Rusty Russell
da5eb03bae lightningd/chaintopology: ensure htables are always tal objects.
We want to change the htable allocator to use tal, which will need
this.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-12 11:44:10 +10:30
niftynei
42c6d49082 tests: add account_id's and match by acct id, not test ordering
There's no guarantee as to iteration order for accounts/channels, but
this test was relying on them.

Adding account attribution and comparing by account_ids fixes

Fixes: #5869
Reported-By: @rustyrussell
2023-01-12 11:40:17 +10:30
niftynei
66a4d50098 tests: mark test as dev (times out otherwise) 2023-01-12 11:40:17 +10:30
Rusty Russell
3a39c635b6 CI: rough check that schema changes seem to mention added, don't delete non-deprcated.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-11 12:51:18 +10:30
Rusty Russell
717cb03f51 doc: add recent additions, fix annotation on listpeers to actually deprecate.
TODO: It would be great to similarly annotate new/deprecated commands and
their parameters.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: doc: we now annotate what versions JSON field additions and deprecations happenened.
2023-01-11 12:51:18 +10:30
Rusty Russell
b2148d0eab docs: handle "added": "version" and "deprecated": "version" from schemas.
This means we will document deprecations and additions, rather than just
pretending they've always been that way!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-11 12:51:18 +10:30
Rusty Russell
3f8199bbfd doc: document that we should annotate added and deprecated schemas.
No longer just delete them!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-11 12:51:18 +10:30
Rusty Russell
404e961bad cli: add -c/--commando support.
It's easier to type:

```
lightning-cli --commando=03ce2d830369fc903ffec52ca1d7aba095c3cf5d17175b6c9a3ff058f6aece37bc:V08OylkJ2ZZPClAXbTaxrXJ9YpKnmucJxcQI-wvIGiE9MA== invoice any "Invoice Label" "Invoice Description"
lightning-cli --commando=03ce2d830369fc903ffec52ca1d7aba095c3cf5d17175b6c9a3ff058f6aece37bc:V08OylkJ2ZZPClAXbTaxrXJ9YpKnmucJxcQI-wvIGiE9MA== commando amount_msat=100000 label="invoice label" description="invoice description"
```

Than:

```
commando 03ce2d830369fc903ffec52ca1d7aba095c3cf5d17175b6c9a3ff058f6aece37bc invoice '["any", "Invoice Label", "Invoice Description"]' V08OylkJ2ZZPClAXbTaxrXJ9YpKnmucJxcQI-wvIGiE9MA==
commando 03ce2d830369fc903ffec52ca1d7aba095c3cf5d17175b6c9a3ff058f6aece37bc invoice '{"amount_msat": "100000", "label": "invoice label", "description": "invoice description"}' V08OylkJ2ZZPClAXbTaxrXJ9YpKnmucJxcQI-wvIGiE9MA==
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: cli: `--commando=peerid:rune` (or `-c peerid:rune`) as convenient shortcut for running commando commands.
2023-01-11 11:13:27 +10:30
Rusty Russell
3f0c5b985b commando: add filtering support.
1. When we receive a commando command from a remote using the `filter`
   field, use it.
2. Add a `filter` parameter to `commando` to send it: this is usually
   more efficient than using filtering locally.

Of course, older remote nodes will ignore the filter, but that's
harmless.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Plugins: `commando` now supports `filter` as a parameter (for send and receive).
2023-01-11 11:13:27 +10:30
Rusty Russell
1250806060 commando: correctly replace the id field in responses.
This was reported a while ago: now do it properly.

Fixes: #5637
Changelog-Fixed: Plugins: `commando` now responds to remote JSON calls with the correct JSON `id` field.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-11 11:13:27 +10:30