Commit Graph

2475 Commits

Author SHA1 Message Date
Rusty Russell
70aee52903 libplugin: don't spew datastore errors to LOG_DEBUG.
People get upset, especially as our "not found" error can be a bit
hard to read!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
See-also: #5990
2023-02-11 12:22:40 -06:00
Rusty Russell
a71bd3ea37 options: create enable/disable option for peer storage.
Since it's not spec-final yet (hell, it's not even properly specified
yet!) we need to put it behind an experimental flag.

Unfortunately, we don't have support for doing this in a plugin; a
plugin must present features before parsing options.  So we need to do
it in core.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-02-08 08:37:59 -06:00
adi2011
fc382dd87e tests/test_misc.py: Add test_restorefrompeer. 2023-02-08 08:37:59 -06:00
niftynei
3586559fac test (df): check 'require-confirmed-inputs' for v2 opens 2023-02-07 21:03:36 -06:00
niftynei
9f53e3c7f5 df: wire up peer's "require-confirmed-inputs"
We push this info out to the various RPCs/hooks.
2023-02-07 21:03:36 -06:00
Michael Schmoock
30454ddf19 pytest: listpeers new attribute num_channels 2023-02-07 14:46:04 -06:00
Greg Sanders
eef0c087fc More accurate elements commitment tx size estimation 2023-02-06 18:50:26 -06:00
Carl Dong
1dbc29b8c0 lightningd: Add signinvoice to sign a BOLT11 invoice.
Though there's already a `createinvoice` command, there are usecases where a
user may want to sign an invoice that they don't yet have the preimage to. For
example, they may have an htlc_accepted plugin that pays to obtain the preimage
from someone else and returns a `{ "result": "resolve", ... }`.

This RPC command addresses this usecase without overly complicating the
semantics of the existing `createinvoice` command.

Changelog-Added: JSON-RPC: `signinvoice` new command to sign BOLT11
invoices.
2023-02-06 15:54:32 -06:00
Rusty Russell
d06c1871a9 pytest: fix flake in test_closing_disconnected_notify
We might be disconnected, but the subd isn't dead yet:

```
>       assert out[0] == '# peer is offline, will negotiate once they reconnect (5 seconds before unilateral close).'
E       AssertionError: assert '# Timed out, forcing close.' == ('# peer is offline, will negotiate once they reconnect (5 seconds before '\n 'unilateral close).')
E         - # peer is offline, will negotiate once they reconnect (5 seconds before unilateral close).
E         + # Timed out, forcing close.

tests/test_closing.py:164: AssertionError
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-02-06 18:51:44 +10:30
Rusty Russell
28b31c19dd pytest: fix flake in test_bolt11_null_after_pay
It's possible that l2 hasn't completely processed the connection yet:

```
        # create l2->l1 channel.
        l2.fundwallet(amount_sat * 5)
        l1.rpc.connect(l2.info['id'], 'localhost', l2.port)
>       l2.rpc.fundchannel(l1.info['id'], amount_sat * 3)

tests/test_pay.py:3974:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
contrib/pyln-client/pyln/client/lightning.py:833: in fundchannel
    return self.call("fundchannel", payload)
contrib/pyln-testing/pyln/testing/utils.py:706: in call
    res = LightningRpc.call(self, method, payload, cmdprefix, filter)
...
E           pyln.client.lightning.RpcError: RPC call failed: method: fundchannel, payload: {'id': '0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518', 'amount': 3000000, 'announce': True}, error: {'code': 400, 'message': 'Unable to connect, no address known for peer', 'data': {'id': '0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518', 'method': 'connect'}}

contrib/pyln-client/pyln/client/lightning.py:422: RpcError
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-02-05 10:02:46 +01:00
niftynei
4342043382 tests: de-flake test that was failing on cltv expiry
make the number of blocks mined father away from the cltv timeout

from borked/flakey test run:

	lightningd-3 2023-01-26T21:45:19.261Z DEBUG   022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59-channeld-chan#2: billboard perm: Received error channel 27a4a4dd880e86
	1e390517de3e786a237c5ad1f00faab277382664e76b5c3870: Fulfilled HTLC 0 SENT_REMOVE_COMMIT cltv 116 hit deadline
2023-02-05 10:02:46 +01:00
niftynei
195a2cf44b dual-open: use tx-abort instead of warning/errors
When a channel open fails, we use tx-abort instead of warning/error.

This means that the peer won't disconnect! And instead when a new
message arrives, we'll need to rebuild the dualopend subd (if missing).

Makes opens a bit easer to retry (no reconnect needed), as well as keeps
the connection alive for other channels we may have with that peer.

Changelog-Changed: Experimental-Dual-Fund: open failures don't disconnect, but instead fail the opening process
2023-02-05 10:02:46 +01:00
niftynei
b375a35fa0 v2 open tests: don't drop connection when an openchannel fails
Dropping the connection is bad behavior on an openchannel failure,
especially given that there might be other channels currently connected.

We should maintain the connection but close out the dualopend
daemon for that attempt.

This test partially documents the behaivor, but fails

Changelog-None
2023-02-05 10:02:46 +01:00
Greg Sanders
eee7ad3e1c relax log check for test_closing_higherfee 2023-02-05 09:39:11 +01:00
niftynei
35f12b4ca1 upgradewallet: JSONRPC call to update p2sh outputs to a native segwit
v2 opens require you to use native segwit inputs

Changelog-Added: JSONRPC: `upgradewallet` command, sweeps all p2sh-wrapped outputs to a native segwit output
2023-02-04 15:31:16 +10:30
niftynei
4c46750001 dual-open-rbf: remember the requested lease amount btw restarts
Don't forget the requested lease across restarts.
2023-02-04 15:31:16 +10:30
niftynei
314c021e2c test: restart node during rbf
state isn't kept around for lease amount, so it should fail
2023-02-04 15:31:16 +10:30
niftynei
4fe8e1eccf tests: check that non-anchor opens can't use liquidity ads
liquidity ads (as proposed) rely on the CSV addition to the to_remote output
that anchors introduced.
2023-02-04 15:31:16 +10:30
niftynei
cbe38dd350 tests: anchors is only EXPERIMENTAL_FEATURES
we've removed the EXPERIMENTAL_DUAL_FUND requirement
2023-02-04 15:31:16 +10:30
niftynei
c9c367d770 dual-fund: remove anchor assumption for all dual-funded channels
Only add the anchor channel_type if it's negotiated separately!
2023-02-04 15:31:16 +10:30
niftynei
976f6ef51a df tests: use the amount from the logs to check for! 2023-02-04 15:31:16 +10:30
Michael Schmoock
ad249607d6 dual-fund: update extracted CSVs to latest bolt draft
Changelog-None
2023-02-04 15:31:16 +10:30
Rusty Russell
35acdc112f offers: fix pay where we are using deprecated apis.
In this case, "fee_base_msat" from `listincoming` has `msat` appended.

Fixes: #5850
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-02-03 16:59:51 +01:00
Rusty Russell
e8f9366a29 pytest: add test for using offers with allow-deprecated-apis=True
Demonstrates that offers doesn't handle msat fields correctly.

Reported-by: @SimonVrouwe
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-02-03 16:59:51 +01:00
Rusty Russell
ff1d537b87 pytest: neaten test_cln_sendpay_weirdness, rename.
1. Allow 'any' as an option to zeroconf-selective.py plugin so we can use
   it in line_graph where we don't know ids yet.
2. Use modern helpers like line_graph and remove debugging statement.
3. Don't use listchannels(): it's true that it shows local channels as well,
   but that's a quirk I'd like to remove.
4. Make flake8 happy.
5. Rename to be more specific now it's a more narrow test.

I manually tested that the test still failed with the fixes removed, too,
so it is still the same test!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-02-03 09:17:33 +10:30
Rusty Russell
9e9686df20 pay: specify the channel to send the first hop.
If we only specify the node_id, we get the "first" channel.

Closes: #5803
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Plugins: `pay` uses the correct local channel for payments when there are multiple available (not just always the first!)
2023-02-03 09:17:33 +10:30
Rusty Russell
0cbd9e02de pytest: limit test_cln_sendpay_weirdness to only failures.
We fixed most of them.  Now hone in to the case which fails: `pay`
when it needs to use the direct zero-conf channel.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-02-03 09:17:33 +10:30
Carl Dong
7c7e32b324 tests: add Carl Dong's example exhaustive zeroconf channel pay test.
Modifications from issue #5803 to work here:

1. import json
2. Add xfail
3. Increase channel sizes by 10x so we can open them
4. Fix plugin path

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-02-03 09:17:33 +10:30
Rusty Russell
38510202c4 pytest: fix flake in test_closing_simple when we mine too fast.
We can actually catch l2 with HTLCs still closing and mine blocks,
then it force closes due to HTLC timeout.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-02-02 14:56:04 -06:00
Rusty Russell
f87c7ed439 plugins/sql: fix foreign keys.
I noticed that our subtables were not being cleaned, despite being "ON
DELETE CASCADE".  This is because foreign keys were not enabled, but
then I got foreign key errors: rowid cannot be a foreign key anyway!

So create a real "rowid" column.  We want "ON DELETE CASCADE" for
nodes and channels (and other tables in future) where we update
partially.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-02-02 14:56:04 -06:00
Peter Neuroth
80250f9b60 datastore: Add check for empty key array
We need to check if the key parameter is an empty array in
`listdatastore` as we do assume an array of at least length 1 in
`wallet.c:5306`.

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2023-01-31 10:23:18 +10:30
Rusty Russell
0240c24936 plugins/sql: listsqlschemas command to retrieve schemas.
Good for detection of what fields are present.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell
d8320f015f plugins/sql: add bkpr-listaccountevents and bkpr-listincome support.
This *would* be a 1-line change (add it to Makefile) except that we
previously assumed a "list" prefix on commands.

These use the default refreshing, but they could be done better using
the time-range parameters.

Suggested-by: @niftynei
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell
9a591277f5 plugins/sql: allow some simple functions.
And document them!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell
20654ebd49 plugins/sql: pay attention to deprecated in schema.
For now, we ignore every deprecated field, but put in the logic so
that future deprecations will work as expected.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell
9b08c4f25a plugins/sql: refresh listnodes and listchannels by monitoring the gossip_store.
It's quite a lot of code, but these are the most expensive commands we
do so it's worth it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell
aa3a1131aa plugins/sql: include the obvious indexes.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell
68370a203e pytest: perform more thorough testing.
Painfully created by hand from the source.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell
8a0ee5f56e plugins/sql: add listpeerchannels support.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell
c230291141 plugins/sql: rework to parse schemas.
This requires us to rename "index" fields, rename fields if we have a
sub-object, and create sub-tables if we have an array, and handle the
fact that some listX commands don't contain array X (listsendpays
contains "payments").

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell
24d86a85c3 plugins/sql: initial commit of new plugin.
This is designed to allow you to perform complex server-side
queries.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Rusty Russell
9ab488fc41 plugins/topology: add direction field to listchannels.
It's a core concept in the spec which isn't directly exposed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `listchannels` added a `direction` field (0 or 1) as per gossip specification.
2023-01-30 15:15:41 -06:00
Alex Myers
98bfd23fff pytest: test_channel_resurrection now succeeds 2023-01-30 16:33:03 +10:30
Alex Myers
6bff10cd40 gossip_store: add a flag for zombie entries
This will allow gossipd to store and persist gossip for channels rather
than deleting them entirely when the channels are pruned from the
network.
2023-01-30 16:33:03 +10:30
Alex Myers
f2f05117aa pytest: gossipd: test resumption of pruned channels 2023-01-30 16:33:03 +10:30
Michael Schmoock
1fb1e0eec4 pytest: test ip discovery for custom port 2023-01-25 17:57:04 +01:00
Michael Schmoock
a2b94f16f8 pytest: fix and adapt test_remote_addr_disabled 2023-01-25 14:37:56 +01:00
Michael Schmoock
3babbc5201 opts: announce-addr-discovered on/off/auto switch
This adds the option to explicitly enable ip-discovery, which maybe
helpful for example when a user wants TOR announced along with
discovered IPs to improve connectivity and have TOR just as a fallback.

Changelog-Added: Adds config switch 'announce-addr-discovered': on/off/auto
2023-01-25 14:37:56 +01:00
Rusty Russell
17aa047b17 pytest: fix output order assumption in test_setchannel_all
Hash table coming up makes it random!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-21 08:05:31 -06:00
Christian Decker
1f21796454 gha: Temporarily disable test_notify
It is failing due to an error somewhere:

```
*** buffer overflow detected ***: terminated
```
2023-01-16 13:53:48 +10:30