Commit Graph

2452 Commits

Author SHA1 Message Date
Rusty Russell
e8f9366a29 pytest: add test for using offers with allow-deprecated-apis=True
Demonstrates that offers doesn't handle msat fields correctly.

Reported-by: @SimonVrouwe
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-02-03 16:59:51 +01:00
Rusty Russell
ff1d537b87 pytest: neaten test_cln_sendpay_weirdness, rename.
1. Allow 'any' as an option to zeroconf-selective.py plugin so we can use
   it in line_graph where we don't know ids yet.
2. Use modern helpers like line_graph and remove debugging statement.
3. Don't use listchannels(): it's true that it shows local channels as well,
   but that's a quirk I'd like to remove.
4. Make flake8 happy.
5. Rename to be more specific now it's a more narrow test.

I manually tested that the test still failed with the fixes removed, too,
so it is still the same test!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-02-03 09:17:33 +10:30
Rusty Russell
9e9686df20 pay: specify the channel to send the first hop.
If we only specify the node_id, we get the "first" channel.

Closes: #5803
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Plugins: `pay` uses the correct local channel for payments when there are multiple available (not just always the first!)
2023-02-03 09:17:33 +10:30
Rusty Russell
0cbd9e02de pytest: limit test_cln_sendpay_weirdness to only failures.
We fixed most of them.  Now hone in to the case which fails: `pay`
when it needs to use the direct zero-conf channel.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-02-03 09:17:33 +10:30
Carl Dong
7c7e32b324 tests: add Carl Dong's example exhaustive zeroconf channel pay test.
Modifications from issue #5803 to work here:

1. import json
2. Add xfail
3. Increase channel sizes by 10x so we can open them
4. Fix plugin path

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-02-03 09:17:33 +10:30
Rusty Russell
38510202c4 pytest: fix flake in test_closing_simple when we mine too fast.
We can actually catch l2 with HTLCs still closing and mine blocks,
then it force closes due to HTLC timeout.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-02-02 14:56:04 -06:00
Rusty Russell
f87c7ed439 plugins/sql: fix foreign keys.
I noticed that our subtables were not being cleaned, despite being "ON
DELETE CASCADE".  This is because foreign keys were not enabled, but
then I got foreign key errors: rowid cannot be a foreign key anyway!

So create a real "rowid" column.  We want "ON DELETE CASCADE" for
nodes and channels (and other tables in future) where we update
partially.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-02-02 14:56:04 -06:00
Peter Neuroth
80250f9b60 datastore: Add check for empty key array
We need to check if the key parameter is an empty array in
`listdatastore` as we do assume an array of at least length 1 in
`wallet.c:5306`.

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2023-01-31 10:23:18 +10:30
Rusty Russell
0240c24936 plugins/sql: listsqlschemas command to retrieve schemas.
Good for detection of what fields are present.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell
d8320f015f plugins/sql: add bkpr-listaccountevents and bkpr-listincome support.
This *would* be a 1-line change (add it to Makefile) except that we
previously assumed a "list" prefix on commands.

These use the default refreshing, but they could be done better using
the time-range parameters.

Suggested-by: @niftynei
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell
9a591277f5 plugins/sql: allow some simple functions.
And document them!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell
20654ebd49 plugins/sql: pay attention to deprecated in schema.
For now, we ignore every deprecated field, but put in the logic so
that future deprecations will work as expected.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell
9b08c4f25a plugins/sql: refresh listnodes and listchannels by monitoring the gossip_store.
It's quite a lot of code, but these are the most expensive commands we
do so it's worth it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell
aa3a1131aa plugins/sql: include the obvious indexes.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell
68370a203e pytest: perform more thorough testing.
Painfully created by hand from the source.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell
8a0ee5f56e plugins/sql: add listpeerchannels support.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell
c230291141 plugins/sql: rework to parse schemas.
This requires us to rename "index" fields, rename fields if we have a
sub-object, and create sub-tables if we have an array, and handle the
fact that some listX commands don't contain array X (listsendpays
contains "payments").

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell
24d86a85c3 plugins/sql: initial commit of new plugin.
This is designed to allow you to perform complex server-side
queries.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell
9ab488fc41 plugins/topology: add direction field to listchannels.
It's a core concept in the spec which isn't directly exposed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `listchannels` added a `direction` field (0 or 1) as per gossip specification.
2023-01-30 15:15:41 -06:00
Alex Myers
98bfd23fff pytest: test_channel_resurrection now succeeds 2023-01-30 16:33:03 +10:30
Alex Myers
6bff10cd40 gossip_store: add a flag for zombie entries
This will allow gossipd to store and persist gossip for channels rather
than deleting them entirely when the channels are pruned from the
network.
2023-01-30 16:33:03 +10:30
Alex Myers
f2f05117aa pytest: gossipd: test resumption of pruned channels 2023-01-30 16:33:03 +10:30
Michael Schmoock
1fb1e0eec4 pytest: test ip discovery for custom port 2023-01-25 17:57:04 +01:00
Michael Schmoock
a2b94f16f8 pytest: fix and adapt test_remote_addr_disabled 2023-01-25 14:37:56 +01:00
Michael Schmoock
3babbc5201 opts: announce-addr-discovered on/off/auto switch
This adds the option to explicitly enable ip-discovery, which maybe
helpful for example when a user wants TOR announced along with
discovered IPs to improve connectivity and have TOR just as a fallback.

Changelog-Added: Adds config switch 'announce-addr-discovered': on/off/auto
2023-01-25 14:37:56 +01:00
Rusty Russell
17aa047b17 pytest: fix output order assumption in test_setchannel_all
Hash table coming up makes it random!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-21 08:05:31 -06:00
Christian Decker
1f21796454 gha: Temporarily disable test_notify
It is failing due to an error somewhere:

```
*** buffer overflow detected ***: terminated
```
2023-01-16 13:53:48 +10:30
Christian Decker
e76618e2a6 ci: Use bzip2 and release to reduce artifact size 2023-01-16 13:53:48 +10:30
Rusty Russell
9d5841a0e6 pytest: fix flake in tests/test_closing.py::test_closing_specified_destination
The close call can fail, since we already unilaterally closed since we mined blocks
too fast:

```
2023-01-14T01:00:10.2502199Z E           pyln.client.lightning.RpcError: RPC call failed: method: close, payload: ['107x1x1', None, 'bcrt1qeyyk6sl5pr49ycpqyckvmttus5ttj25pd0zpvg'], error: {'code': -32602, 'message': "Short channel ID not active: '107x1x1'"}
...
2023-01-14T01:00:10.5288050Z lightningd-4 2023-01-14T00:59:59.650Z UNUSUAL 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-chan#1: Peer permanent failure in CHANNELD_NORMAL: Fulfilled HTLC 0 SENT_REMOVE_COMMIT cltv 113 hit deadline
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-14 21:17:45 +10:30
Rusty Russell
932ca9e91f lightningd: deprecate p2sh-segwit addresses for newaddr addresstype
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: JSON-RPC: `newaddr`: `addresstype` `p2sh-segwit` (use default, or `bech32`).
2023-01-14 21:17:45 +10:30
Rusty Russell
2f36c03307 pytest: use bech32 addresses everywhere.
We used to create some p2sh-segwit addresses just to mix things up.  This
streamlines back to just bech32.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-14 21:17:45 +10:30
Rusty Russell
85fc46f76c pyln-testing: don't default openchannel and fundwallet to p2sh-segwit, use bech32.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-14 21:17:45 +10:30
Rusty Russell
9ffaab7d22 pytest: fix race in test_bookkeeping_closing_subsat_htlcs
With the next change (which, as a side-effect, speeds up listpeers),
we seem to hit a race in this test.  The bookkeeper doesn't get to
process the final payment before the node is shutdown.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-13 10:42:42 +10:30
Vincenzo Palazzo
a2347c7452 tests: use listpeerchannels. 2023-01-13 10:42:42 +10:30
Vincenzo Palazzo
1fa32333b9 tests/utils.py: use listpeerchannels. 2023-01-13 10:42:42 +10:30
niftynei
42c6d49082 tests: add account_id's and match by acct id, not test ordering
There's no guarantee as to iteration order for accounts/channels, but
this test was relying on them.

Adding account attribution and comparing by account_ids fixes

Fixes: #5869
Reported-By: @rustyrussell
2023-01-12 11:40:17 +10:30
niftynei
66a4d50098 tests: mark test as dev (times out otherwise) 2023-01-12 11:40:17 +10:30
Rusty Russell
404e961bad cli: add -c/--commando support.
It's easier to type:

```
lightning-cli --commando=03ce2d830369fc903ffec52ca1d7aba095c3cf5d17175b6c9a3ff058f6aece37bc:V08OylkJ2ZZPClAXbTaxrXJ9YpKnmucJxcQI-wvIGiE9MA== invoice any "Invoice Label" "Invoice Description"
lightning-cli --commando=03ce2d830369fc903ffec52ca1d7aba095c3cf5d17175b6c9a3ff058f6aece37bc:V08OylkJ2ZZPClAXbTaxrXJ9YpKnmucJxcQI-wvIGiE9MA== commando amount_msat=100000 label="invoice label" description="invoice description"
```

Than:

```
commando 03ce2d830369fc903ffec52ca1d7aba095c3cf5d17175b6c9a3ff058f6aece37bc invoice '["any", "Invoice Label", "Invoice Description"]' V08OylkJ2ZZPClAXbTaxrXJ9YpKnmucJxcQI-wvIGiE9MA==
commando 03ce2d830369fc903ffec52ca1d7aba095c3cf5d17175b6c9a3ff058f6aece37bc invoice '{"amount_msat": "100000", "label": "invoice label", "description": "invoice description"}' V08OylkJ2ZZPClAXbTaxrXJ9YpKnmucJxcQI-wvIGiE9MA==
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: cli: `--commando=peerid:rune` (or `-c peerid:rune`) as convenient shortcut for running commando commands.
2023-01-11 11:13:27 +10:30
Rusty Russell
3f0c5b985b commando: add filtering support.
1. When we receive a commando command from a remote using the `filter`
   field, use it.
2. Add a `filter` parameter to `commando` to send it: this is usually
   more efficient than using filtering locally.

Of course, older remote nodes will ignore the filter, but that's
harmless.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Plugins: `commando` now supports `filter` as a parameter (for send and receive).
2023-01-11 11:13:27 +10:30
Rusty Russell
1250806060 commando: correctly replace the id field in responses.
This was reported a while ago: now do it properly.

Fixes: #5637
Changelog-Fixed: Plugins: `commando` now responds to remote JSON calls with the correct JSON `id` field.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-11 11:13:27 +10:30
Rusty Russell
b75ada7017 commando: track incoming and outgoing JSON IDs. Get upset if they don't match!
They currently don't, so we get some BROKEN messages.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-11 11:13:27 +10:30
Rusty Russell
0201e6977f commando: build ID of command based on the id they give us.
We change the libplugin API so commando can provide its own ID base.

This id chaining enables much nicer diagnostics!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-11 11:13:27 +10:30
Rusty Russell
b3fa4b932e commando: send id inside JSON request.
All JSON-RPC calls should have one!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-11 11:13:27 +10:30
Rusty Russell
19db6a25e4 commando: require that we have an id field in JSON request.
We don't do this yet, so we add deprecated to those test (until next
patch!).

Changelog-Deprecated: plugins: `commando` JSON commands without an `id` (see doc/lightningd-rpc.7.md for how to construct a good id field).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-11 11:13:27 +10:30
Rusty Russell
435f8d84dc lightning-cli: fix error code on invalid options, document them.
The top of the file indicates the following errors:

    #define NO_ERROR 0
    #define ERROR_FROM_LIGHTNINGD 1
    #define ERROR_TALKING_TO_LIGHTNINGD 2
    #define ERROR_USAGE 3

But we didn't use the right one for opt_parse failure, and didn't use the
correct constants everywhere.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-11 11:13:27 +10:30
Rusty Russell
a1e894a445 lightningd: treat JSON ids as direct tokens.
This avoids any confusion between primitive and string ids, and in
particular stops an issue with commando once it starts chaining ids,
that weird ids can be double-escaped and commando will not recognize
the response, leaving the client hanging.  It's the client's fault for
using a weird id, but it's still rude (and triggered by our tests!).

It also makes substituting the id in passthrough simpler, FTW.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-11 11:13:27 +10:30
Gregg Zigler
9ed138f5e5 pytest: add tests for devtools/mkfunding
Issue #5363 documented an earlier bug in mkfunding. These tests validate that fix as well as checking that basic usage produces the proper response and exit status.

Changelog-None
2023-01-09 14:50:30 -06:00
Rusty Russell
d1b5943e90 tests: make test_libplugin use the datastore.
It's a bit convoluted, but it exercises all our datastore fetch APIs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-12-22 12:04:30 +01:00
Rusty Russell
ec025344cc lightningd: don't announce names as DNS by default.
This broke BTCPayServer, so revert.  I originally (accidentally!)
implemented this such that it broadcast both DNS and IP entries, but
Michael reported earlier that they still don't propagage well, so
simply suppress them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes: #5795
Changelog-Changeed: Config: `announce-addr-dns` needs to be set to *true* to put DNS names into node announcements, otherwise they are suppressed.
Changelog-Deprecated: Config: `announce-addr-dns` (currently defaults to `false`).  This will default to `true` once enough of the network has upgraded to understand DNS entries.
2022-12-08 11:04:53 +01:00
Rusty Russell
3ca1c70c44 lightningd: don't cap spendable_msat/receivable_msat for wumbo channels.
If we both support large channels, we can actually send giant HTLCs.

This, in turn, fixes pay (which relies on this field!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Plugins: `pay` now knows it can use locally-connected wumbo channels for large payments.
Fixes: #5250
Fixes: #5417
2022-11-30 19:51:22 +01:00