Commit Graph

1770 Commits

Author SHA1 Message Date
Rusty Russell
33736b860a lightningd: attach HTLC timeout to htlc itself, fix gratuitous disconnect bug.
We set the timeout on first HTLC, but didn't clear it if that HTLC failed.

It's saner to have a per-HTLC timeout (since that's what it is!) and
also our timer infra is specially coded to scale approximately infinitely so
trying to optimize this is vastly premature.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Protocol: We would sometimes gratuitously disconnect 30 seconds after an HTLC failed.
2021-05-21 14:45:05 +09:30
Rusty Russell
11180e7aa1 pytest: add test for HTLC timeout on failed htlcs.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-21 14:45:05 +09:30
Rusty Russell
e141b71675 pytest: fix flake in test_reconnect_no_update
If l2 didn't get FUNDING_LOCKED from l1 before it disconnected, it
won't be in state CHANNELD_NORMAL: it will be in DUALOPEND_AWAITING_LOCKIN.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-17 18:58:10 -05:00
niftynei
ce1e5bd388 df-test: make flakey test not as flakey
This would flake fairly regularly, what we really care about is
asserting that the l2 node is in CHANNELD_NORMAL state, while the l1
node hasn't progressed that far yet.
2021-05-12 11:25:41 +09:30
niftynei
e375932108 df-tests: test_coin_movement_notices, make work?
No idea how this slipped past the first time
2021-05-12 11:25:41 +09:30
niftynei
71a4a2e31c df: rework closing logic
Trying to put all the disconnect logic into the same path was a dumb
idea. If you asked to reconnect but passed in an 'unsaved' channel, we
would not call the 'reconnect' code.

Instead, we make a differentiation between "unsaved" channels
(ones that we haven't received commitment tx for) and handle the
disconnect for these separate from where we want to do a reconnect.
2021-05-12 11:25:41 +09:30
niftynei
efdc36c8e9 tests: mark test as requiring developer
hangs with EXP_DF when developer=0
2021-05-12 11:25:41 +09:30
niftynei
5cac36724e df-test: v2 has reconnects 2021-05-12 11:25:41 +09:30
niftynei
6d3fb11bc6 df-tests: patch for state == AWAITING_UNILATERAL problem
Found on CI where DEVELOPER=0 EXPERIMENTAL_DUAL_FUND=1,
as we turn off automatic reconnects when DEVELOPER=1

This test has been modified to make the error happen every time, and
then fixed.

lightningd-2: 2021-05-07T20:12:03.790Z DEBUG   0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-chan#1: Peer has reconnected, state AWAITING_UNILATERAL
lightningd-2: 2021-05-07T20:12:03.812Z **BROKEN** lightningd: FATAL SIGNAL 6 (version e8b3f78)
lightningd-2: 2021-05-07T20:12:03.812Z **BROKEN** lightningd: backtrace: common/daemon.c:44 (send_backtrace) 0x56384ee072e9
lightningd-2: 2021-05-07T20:12:03.813Z **BROKEN** lightningd: backtrace: common/daemon.c:52 (crashdump) 0x56384ee0733b

----------------------------- Captured stderr call -----------------------------
lightningd: lightningd/peer_control.c:1100: peer_connected_hook_final: Assertion `channel->state == DUALOPEND_OPEN_INIT || channel->state == DUALOPEND_AWAITING_LOCKIN' failed.
lightningd: FATAL SIGNAL 6 (version e8b3f78)
0x56384ee072a1 send_backtrace
	common/daemon.c:39
0x56384ee0733b crashdump
	common/daemon.c:52
0x7f88486a020f ???
	???:0
0x7f88486a018b ???
	???:0
0x7f884867f858 ???
	???:0
0x7f884867f728 ???
	???:0
0x7f8848690f35 ???
	???:0
0x56384eddc94e peer_connected_hook_final
	lightningd/peer_control.c:1100
0x56384edea2ed plugin_hook_call_
	lightningd/plugin_hook.c:275
0x56384eddfeb8 plugin_hook_call_peer_connected
	lightningd/peer_control.c:1156
0x56384eddfeb8 peer_connected
	lightningd/peer_control.c:1209
0x56384edc30cd connectd_msg
	lightningd/connect_control.c:332
0x56384edebe6f sd_msg_read
	lightningd/subd.c:509
0x56384edebfb1 read_fds
	lightningd/subd.c:310
0x56384ee483b0 next_plan
	ccan/ccan/io/io.c:59
0x56384ee4885b do_plan
	ccan/ccan/io/io.c:407
0x56384ee488f8 io_ready
	ccan/ccan/io/io.c:417
0x56384ee4a23c io_loop
	ccan/ccan/io/poll.c:445
0x56384edcabda io_loop_with_timers
	lightningd/io_loop_with_timers.c:24
0x56384edce826 main
	lightningd/lightningd.c:1111
0x7f88486810b2 ???
	???:0
0x56384edb52ad ???
	???:0
0xffffffffffffffff ???
	???:0
2021-05-12 11:25:41 +09:30
niftynei
558abe288a tests: explicitly mark tests to run both as v1+v2
By default, tests only run as v1 unless marked as v2.

These tests we want to run as both v1+v2

Includes fixes to have tests pass
2021-05-12 11:25:41 +09:30
niftynei
dc758f616b tests: mark as v1 or v2 openchannel required 2021-05-12 11:25:41 +09:30
niftynei
3a2d602922 tests: add marker for v1/v2 channel opens
Tests that will only run when !EXPERIMENTAL_DUAL_FUND:

	@pytest.marker.openchannel('v1')
	def test_...()

Tests that will only run when EXPERIMENTAL_DUAL_FUND:

	@pytest.marker.openchannel('v2')
	def test_...()
2021-05-12 11:25:41 +09:30
niftynei
d0bbf07655 tests: not DEVELOPER -> mark.developer
Nicer syntaxtic sugar for marking pytests as 'developer required'
2021-05-12 11:25:41 +09:30
niftynei
b4e24ac8ba df: anchor outputs are on if EXP_DF
So we should treat it the same as EXPERIMENTAL_FEATURES
2021-05-12 11:25:41 +09:30
niftynei
484d6bde83 tests: move EXP_DF into the testing utils 2021-05-12 11:25:41 +09:30
niftynei
7a488cc11e df-tests: remove df_accepter plugin
The `funder` plugin, does the same stuff as this temporary test plugin,
so we move over to using that instead.
2021-05-12 11:25:41 +09:30
Rusty Russell
d8e68893f5 bcli: become less aggressive with onchain fee levels.
Users are more upset recently with the cost of unilateral closes
than they are the risk of being cheated.  While we complete our
anchor implementation so we can use low fees there, let's
get less aggressive (we already have 34 or 18 blocks to close
in the worst case).

The changes are:

- Commit transactions were "2 CONSERVATIVE" now "6 ECONOMICAL".
- HTLC resolution txs were "3 CONSERVATIVE" now "6 ECONOMICAL".
- Penalty txs were "3 CONSERVATIVE" now "12 ECONOMICAL".
- Normal txs were "4 ECONOMICAL" now "12 ECONOMICAL".

There can be no perfect levels, but we have had understandable
complaints recently about how high our default fee levels are.

Changelog-Changed: Protocol: channel feerates reduced to bitcoind's "6 block ECONOMICAL" rate.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-11 11:25:16 +09:30
Rusty Russell
9825f32874 lightningd: implement --log-timestamps=false.
Fixes: #4494
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: config: New option `log-timestamps` allow disabling of timestamp prefix in logs.
2021-05-05 17:19:19 -05:00
Christian Decker
c6fd849aa3 pay: Add notification for pay_success 2021-05-03 11:20:15 +09:30
Christian Decker
98aa3c3da7 plugin: Make unannounced notification topics no longer fatal
Since plugins will start sending them soon, and they are likely to get
it wrong sometimes, be a bit more lenient, warn them in the logs
instead and then make sure it doesn't accidentally work anyway.
2021-05-03 11:20:15 +09:30
Christian Decker
f963a6a551 libplugin: Add notification topics to plugin_main 2021-05-03 11:20:15 +09:30
Christian Decker
62e3358a5b plugin: Wrap custom notifications in a dict with additional origin
This should allow us to differentiate the origin of the notification,
and further prevent plugins from spoofing native notifications.
2021-05-03 11:20:15 +09:30
Christian Decker
083b41f090 plugin: Add a list of notification topics registered by plugin
We will eventually start emitting and dispatching custom notifications
from plugins just like we dispatch internal notifications. In order to
get reasonable error messages we need to make sure that the topics
plugins are asking for were correctly registered. When doing this we
don't really care about whether the plugin that registered the
notification is still alive or not (it might have died, but
subscribers should stay up and running), so we keep a list of all
topics attached to the `struct plugins` which gathers global plugin
information.
2021-05-03 11:20:15 +09:30
niftynei
29155c2fe8 tests: add test for funder options 2021-05-03 11:06:10 +09:30
niftynei
cd5970243a funder: add a plugin, funder. policies for dual-funding
Behold! An immaculately concepted plugin for configuring your node to do
amazing things*

*fund channel open requests

Changelog-Added:  Plugins: Add `funder` plugin, which allows you to setup a policy for funding v2 channel open requests. Requres --experimental-dual-fund option
2021-05-03 11:06:10 +09:30
niftynei
7c76363e20 openchannel2: add channel_max_msat to openchannel2 hook payload
Changelog-Added: Plugins: add a `channel_max_msat` value to the `openchannel2` hook. Tells you the total max funding this channel is allowed to have.
2021-05-03 11:06:10 +09:30
niftynei
69cc9201de plugin-tests: cleanup outdated openchannel2 hook fields
We removed/changed the fields on the openchannel2 hook but never
updated this test (which doesn't run with the CI for #reasons)
2021-05-03 11:06:10 +09:30
niftynei
0e753224ae tests: remove EXPERIMENTAL_FEATURE flag from openchannel2 hooks
These are now included in all builds
2021-05-03 11:06:10 +09:30
Christian Decker
9470ea3032 pytest: Fix racy test_unreachable_routehint
It was failing likely because we were racing on the log
processing. Using `wait_for_log` instead should address that.
2021-04-24 13:58:57 +09:30
Rusty Russell
e3d1115598 pytest: fix race in test_penalty_htlc_tx_timeout
We don't always get two transactions on line 1019; the comment is
confused (only one penalty tx successfully comes out of l3).  So make
sure we get the other transactions when we expect them, and then
make this test more specific.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-04-24 13:57:33 +09:30
Rusty Russell
b352df4179 pytest: make it work with latest bitcoind master branch.
They seem to have changed the JSON output.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-04-24 13:57:16 +09:30
Rusty Russell
32d650f9df lightningd: don't abort on incorrect versions, but try to re-exec.
You still shouldn't do this (you could get some transient failures),
but at least you have a decent chance if you reinstall over a running
daemon, instead of getting confusing internal errors if message
formats have changed.

Changelog-Added: lightningd: we now try to restart if subdaemons are upgraded underneath us.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes: #4346
2021-04-24 13:56:58 +09:30
Rusty Russell
4e2410742a tests/fuzz: update for fmt_amount changes, create pseudorand for fuzzing.
Reported-by: Christian Reitter
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-04-21 06:51:03 +09:30
Rusty Russell
8cef2c17ad pytest: fix flake in test_v2_open_sigs_restart_while_dead
We didn't wait for the sendtx to complete before shutting down,
and hence timed out:

```
        l2.daemon.wait_for_log('Broadcasting funding tx')

        l1.stop()
        l2.stop()
        bitcoind.generate_block(6)
        l1.restart()
        l2.restart()

        # Make sure we're ok.
>       l2.daemon.wait_for_log(r'to CHANNELD_NORMAL')
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-04-12 23:03:47 +02:00
Rusty Russell
c0a40b37a7 pytest: fix flake in test_pay_disconnect
Channel can be inactive before it disconnects, apparently.  Check
explicitly for the disconnected state so we get the expected error.

Here's what happened:
```
        # Can't pay while its offline.
        with pytest.raises(RpcError, match=r'failed: WIRE_TEMPORARY_CHANNEL_FAILURE \(First peer not ready\)'):
>           l1.rpc.sendpay(route, rhash)
E           Failed: DID NOT RAISE <class 'pyln.client.lightning.RpcError'>
```

And the logs show that the outgoing HTLC was sent to channeld before it
realized the connection was closed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-04-12 23:03:47 +02:00
Rusty Russell
e684f87ef7 pytest: fix spurious failure in test_local_basepoints_cache.
Change the address in the canned db: it seems we won the lottery and
l1 connected, and got an error!

```
 E           ValueError:
 E           Node errors:
 E            - lightningd-1: had warning messages
 E           Global errors:
...
 lightningd-1: 2021-04-07T02:44:53.579Z DEBUG   022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59-channeld-chan#1: peer_out WIRE_CHANNEL_REESTABLISH
 lightningd-1: 2021-04-07T02:44:53.579Z DEBUG   022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59-channeld-chan#1: peer_in WIRE_GOSSIP_TIMESTAMP_FILTER
 lightningd-1: 2021-04-07T02:44:53.580Z DEBUG   022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59-channeld-chan#1: peer_in WIRE_ERROR
 lightningd-1: 2021-04-07T02:44:53.580Z INFO    022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59-chan#1: Peer transient failure in CHANNELD_NORMAL: channeld WARNING: error channel fdeb1ea12e02aa043f66ba581e969a1882d21142b19429995c6733bb71070bb6: Multiple channels unsupported
 lightningd-1: 2021-04-07T02:44:53.580Z DEBUG   022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59-chan#1: Will try reconnect in 60 seconds
```

So I changed the port in the db to "1" which will never succeed:

```
sqlite> .dump peers
PRAGMA foreign_keys=OFF;
BEGIN TRANSACTION;
CREATE TABLE peers (  id INTEGER, node_id BLOB UNIQUE, address TEXT, PRIMARY KEY (id));
INSERT INTO peers VALUES(1,X'022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59','127.0.0.1:38723');
COMMIT;
sqlite> UPDATE peers SET address="127.0.0.1:1"
   ...> ;
sqlite> .dump peers
PRAGMA foreign_keys=OFF;
BEGIN TRANSACTION;
CREATE TABLE peers (  id INTEGER, node_id BLOB UNIQUE, address TEXT, PRIMARY KEY (id));
INSERT INTO peers VALUES(1,X'022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59','127.0.0.1:1');
COMMIT;
sqlite>
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-04-12 23:03:47 +02:00
Rusty Russell
9c3cf5aff9 newaddr: don't include "address" field.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: JSON-RPC: `newaddr` no longer includes `address` field (deprecated in 0.7.1)
2021-04-07 14:34:39 +09:30
niftynei
1275af6fbb df-tests: update to take 'no channel_id' into account; use exp-df flag
The logs we're looking for don't exist anymore, we got rid of them.
2021-03-25 20:05:11 +10:30
Rusty Russell
a52517bd81 pytest: fix up rbf tests which use dev-disconnect.
They must be marked DEVELOPER-only.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-03-25 20:05:11 +10:30
niftynei
e549746578 features: match up feature names to feature bit
Aligns the feature bit number to the name-label. Is this correct?
2021-03-25 20:05:11 +10:30
niftynei
3e8f575f9e dual-funding: convert to runtime flag, --experimental-dual-fund
You can now activate dual-funded channels using the
`--experimental-dual-fund` flag

Changelog-Changed: Config: `--experimental-dual-fund` runtime flag will enable dual-funded protocol on this node
2021-03-25 20:05:11 +10:30
niftynei
3c87e1f338 test_v2_open_sigs_restart: dont mine blocks until funding tx in mempool
This was flaky because sometimes we'd generate blocks before the funding
transaction reached the mempool.

If we wait until it's in, it works as expected.

[ Neatened to use wait_for_mempool arg --RR ]
2021-03-24 06:56:26 +10:30
niftynei
66a2826976 df-tests: make bad tests a bit better, maybe
We updated the "UNKNOWN TYPE channel_id" -> the actual channel id;
the reason for why the error fails shouldn't be restrictive (we just
know that it fails)
2021-03-24 06:56:26 +10:30
Christian Decker
6cfb72ea1b keysend: Add error when trying to send to self
There is little point in faking a self-payment, but we should also not
crash :-)

Fixes #4438

Changelog-Fixed: keysend: Keysend returns an error when a self-payment is requested
2021-03-19 10:18:42 +10:30
niftynei
2f2981c4e0 df: more reconnection tests 2021-03-17 10:25:18 +10:30
Rusty Russell
6c9d9ee9a2 connect: return address we actually connected to.
Otherwise, we might find an address other than the one given and
the user might think that address worked.

Fixes: #4185
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `connect` returns `address` it actually connected to
2021-03-17 08:38:08 +10:30
Rusty Russell
72ad260e9d pytest: fix now-obsolete use of fundchannel_complete via txid.
This was merged almost at the same time as the API change.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-03-16 14:21:42 +10:30
Rusty Russell
b563cafd83 lightningd: don't complain about bad funding PSBT for elements.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-03-16 13:10:07 +10:30
Rusty Russell
22e1107581 lightningd/opening_control: deprecate old fundchannel_complete args.
And update all the in-tree callers.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: JSON-RPC: `fundchannel_complete` `txid` and `txout` parameters (use `psbt`)
2021-03-16 13:10:07 +10:30
Rusty Russell
a1b43a3653 onchaind: see closes when wrong_funding shutdowns are used.
Fairly easy to do, though we also have to add the watch when we load
from the database.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-03-16 13:08:40 +10:30