Commit Graph

10000 Commits

Author SHA1 Message Date
Rusty Russell
0682ad9c48 fetchinvoice: don't produce invalid JSON on sent_inv timeout.
An unrelated bug in the reverse path triggered this code, and boom.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Rusty Russell
2e3e4a50ec update-mocks: prefer to find function defs from local includes.
This matters for:
	- common/wallet.h vs wallet/wallet.h
	- common/gossip_store.h vs gossipd/gossip_store.h
	- common/json.h vs lightningd/json.h
	- common/ping.h vs lightningd/ping.h

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Rusty Russell
d868d4470a Makefile: don't let update-mocks run unless DEVELOPER and EXPERIMENTAL_FEATURES.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Rusty Russell
25b5e1e099 update-mocks: make sure we cover all test programs.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Rusty Russell
cc6f1fd120 update-mocks: don't try to regenerate files.
If you do update-mocks in a dirty tree, the recursive make that it
uses will try to rebuild things!  Suppress that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Rusty Russell
33736b860a lightningd: attach HTLC timeout to htlc itself, fix gratuitous disconnect bug.
We set the timeout on first HTLC, but didn't clear it if that HTLC failed.

It's saner to have a per-HTLC timeout (since that's what it is!) and
also our timer infra is specially coded to scale approximately infinitely so
trying to optimize this is vastly premature.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Protocol: We would sometimes gratuitously disconnect 30 seconds after an HTLC failed.
2021-05-21 14:45:05 +09:30
Rusty Russell
11180e7aa1 pytest: add test for HTLC timeout on failed htlcs.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-21 14:45:05 +09:30
fiatjaf
0d68febd8d reorganize .gitignore entries across subdirs. 2021-05-18 09:43:50 +09:30
fiatjaf
e17526f64c gitignore more things. 2021-05-18 09:43:50 +09:30
Rusty Russell
e141b71675 pytest: fix flake in test_reconnect_no_update
If l2 didn't get FUNDING_LOCKED from l1 before it disconnected, it
won't be in state CHANNELD_NORMAL: it will be in DUALOPEND_AWAITING_LOCKIN.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-17 18:58:10 -05:00
Christian Decker
0f2009c7a3 make: Do not scan contrib for discouraged functions 2021-05-15 09:31:12 +09:30
Christian Decker
6c67afeb03 make: Add CPPCHECK_OPTS to allow excluding files from the check
With swig we now have C files that are generated with tools that are
not under our control, so provide an escape hatch for them.
2021-05-15 09:31:12 +09:30
Christian Decker
a4c19ba6ba libhsmd: Clean up the build descriptors for libhsmd_python 2021-05-15 09:31:12 +09:30
Christian Decker
323b1a558b libhsmd: Add missing implementation of status functions 2021-05-15 09:31:12 +09:30
Christian Decker
7f15f86718 libhsmd: First working build script for libhsmd-python
Changelog-Added: libhsmd: Added python bindings for `libhsmd`
2021-05-15 09:31:12 +09:30
Rusty Russell
214fdcc9d7 plugin notifications: minor cleanups.
1. We don't need to check for NULL before tal_count(NULL).
2. Use of json_for_each_arr iterator is probably better.
3. Weird indent fixed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-14 10:24:05 +09:30
niftynei
ce1e5bd388 df-test: make flakey test not as flakey
This would flake fairly regularly, what we really care about is
asserting that the l2 node is in CHANNELD_NORMAL state, while the l1
node hasn't progressed that far yet.
2021-05-12 11:25:41 +09:30
niftynei
c75ca8c112 df-bugs: if we've already got the funding_locked, dont renotify
Peer sends funding locked, we tell lightningd who saves it to disk.
Then we restart/reconnect and they retransmit funding_locked. We were
re-notifying lightningd about their lock-in, which was crashing/breaking
things. Instead, we ignore duplicate lock-in messages from the peer.

lightningd-1: 2021-05-11T18:00:12.844Z **BROKEN** 022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59-chan#1: Internal error DUALOPEND_AWAITING_LOCKIN: channel_got_funding_locked twice
2021-05-12 11:25:41 +09:30
niftynei
e375932108 df-tests: test_coin_movement_notices, make work?
No idea how this slipped past the first time
2021-05-12 11:25:41 +09:30
niftynei
71a4a2e31c df: rework closing logic
Trying to put all the disconnect logic into the same path was a dumb
idea. If you asked to reconnect but passed in an 'unsaved' channel, we
would not call the 'reconnect' code.

Instead, we make a differentiation between "unsaved" channels
(ones that we haven't received commitment tx for) and handle the
disconnect for these separate from where we want to do a reconnect.
2021-05-12 11:25:41 +09:30
niftynei
efdc36c8e9 tests: mark test as requiring developer
hangs with EXP_DF when developer=0
2021-05-12 11:25:41 +09:30
niftynei
5cac36724e df-test: v2 has reconnects 2021-05-12 11:25:41 +09:30
niftynei
ef333d5cee df-callbacks: dont log as broken, just reconnect 2021-05-12 11:25:41 +09:30
niftynei
4432672300 df-bug: avoid referencing null channel->owner
If dualopend dies, we shouldn't reference it
2021-05-12 11:25:41 +09:30
niftynei
5ee4c9e46c df: patch for valgrind error
We were freeing the payload, which is then subsequently freed by the
plugin_hook caller. Whoops.

Now we pass through to the callback function and just clean up neatly.

------------------------------- Valgrind errors --------------------------------
Valgrind error file: valgrind-errors.406602
==406602== Invalid read of size 8
==406602==    at 0x12AC93: openchannel2_hook_cb (dual_open_control.c:669)
==406602==    by 0x12AF0A: openchannel2_hook_deserialize (dual_open_control.c:721)
==406602==    by 0x16EF0E: plugin_hook_callback (plugin_hook.c:186)
==406602==    by 0x169746: plugin_response_handle (plugin.c:514)
==406602==    by 0x169959: plugin_read_json_one (plugin.c:620)
==406602==    by 0x169B23: plugin_read_json (plugin.c:665)
==406602==    by 0x1F4076: next_plan (io.c:59)
==406602==    by 0x1F4C5B: do_plan (io.c:407)
==406602==    by 0x1F4C9D: io_ready (io.c:417)
==406602==    by 0x1F6F35: io_loop (poll.c:445)
==406602==    by 0x13D48D: io_loop_with_timers (io_loop_with_timers.c:24)
==406602==    by 0x143388: main (lightningd.c:1111)
==406602==  Address 0x75e7418 is 56 bytes inside a block of size 3,520 free'd
==406602==    at 0x483CA3F: free (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==406602==    by 0x204FB0: del_tree (tal.c:421)
==406602==    by 0x20527E: tal_free (tal.c:486)
==406602==    by 0x122D68: delete_channel (channel.c:124)
==406602==    by 0x129291: channel_disconnect (dual_open_control.c:63)
==406602==    by 0x129364: channel_close_conn (dual_open_control.c:82)
==406602==    by 0x131CF6: peer_please_disconnect (connect_control.c:304)
==406602==    by 0x131DEB: connectd_msg (connect_control.c:326)
==406602==    by 0x172023: sd_msg_read (subd.c:509)
==406602==    by 0x1F4076: next_plan (io.c:59)
==406602==    by 0x1F4C5B: do_plan (io.c:407)
==406602==    by 0x1F4C9D: io_ready (io.c:417)
==406602==  Block was alloc'd at
==406602==    at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==406602==    by 0x204A39: allocate (tal.c:250)
==406602==    by 0x204FFA: tal_alloc_ (tal.c:428)
==406602==    by 0x123165: new_unsaved_channel (channel.c:209)
==406602==    by 0x130D34: peer_start_dualopend (dual_open_control.c:2985)
==406602==    by 0x15BD2A: peer_connected_hook_final (peer_control.c:1105)
==406602==    by 0x16F2E5: plugin_hook_call_ (plugin_hook.c:275)
==406602==    by 0x15BF5C: plugin_hook_call_peer_connected (peer_control.c:1155)
==406602==    by 0x15C16C: peer_connected (peer_control.c:1208)
==406602==    by 0x131E3B: connectd_msg (connect_control.c:332)
==406602==    by 0x172023: sd_msg_read (subd.c:509)
==406602==    by 0x171842: read_fds (subd.c:310)
2021-05-12 11:25:41 +09:30
niftynei
6d3fb11bc6 df-tests: patch for state == AWAITING_UNILATERAL problem
Found on CI where DEVELOPER=0 EXPERIMENTAL_DUAL_FUND=1,
as we turn off automatic reconnects when DEVELOPER=1

This test has been modified to make the error happen every time, and
then fixed.

lightningd-2: 2021-05-07T20:12:03.790Z DEBUG   0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-chan#1: Peer has reconnected, state AWAITING_UNILATERAL
lightningd-2: 2021-05-07T20:12:03.812Z **BROKEN** lightningd: FATAL SIGNAL 6 (version e8b3f78)
lightningd-2: 2021-05-07T20:12:03.812Z **BROKEN** lightningd: backtrace: common/daemon.c:44 (send_backtrace) 0x56384ee072e9
lightningd-2: 2021-05-07T20:12:03.813Z **BROKEN** lightningd: backtrace: common/daemon.c:52 (crashdump) 0x56384ee0733b

----------------------------- Captured stderr call -----------------------------
lightningd: lightningd/peer_control.c:1100: peer_connected_hook_final: Assertion `channel->state == DUALOPEND_OPEN_INIT || channel->state == DUALOPEND_AWAITING_LOCKIN' failed.
lightningd: FATAL SIGNAL 6 (version e8b3f78)
0x56384ee072a1 send_backtrace
	common/daemon.c:39
0x56384ee0733b crashdump
	common/daemon.c:52
0x7f88486a020f ???
	???:0
0x7f88486a018b ???
	???:0
0x7f884867f858 ???
	???:0
0x7f884867f728 ???
	???:0
0x7f8848690f35 ???
	???:0
0x56384eddc94e peer_connected_hook_final
	lightningd/peer_control.c:1100
0x56384edea2ed plugin_hook_call_
	lightningd/plugin_hook.c:275
0x56384eddfeb8 plugin_hook_call_peer_connected
	lightningd/peer_control.c:1156
0x56384eddfeb8 peer_connected
	lightningd/peer_control.c:1209
0x56384edc30cd connectd_msg
	lightningd/connect_control.c:332
0x56384edebe6f sd_msg_read
	lightningd/subd.c:509
0x56384edebfb1 read_fds
	lightningd/subd.c:310
0x56384ee483b0 next_plan
	ccan/ccan/io/io.c:59
0x56384ee4885b do_plan
	ccan/ccan/io/io.c:407
0x56384ee488f8 io_ready
	ccan/ccan/io/io.c:417
0x56384ee4a23c io_loop
	ccan/ccan/io/poll.c:445
0x56384edcabda io_loop_with_timers
	lightningd/io_loop_with_timers.c:24
0x56384edce826 main
	lightningd/lightningd.c:1111
0x7f88486810b2 ???
	???:0
0x56384edb52ad ???
	???:0
0xffffffffffffffff ???
	???:0
2021-05-12 11:25:41 +09:30
niftynei
342f783f53 df-tests: Turn on v2 tests on CI
Let's run more tests with the v2 open protocol
2021-05-12 11:25:41 +09:30
niftynei
558abe288a tests: explicitly mark tests to run both as v1+v2
By default, tests only run as v1 unless marked as v2.

These tests we want to run as both v1+v2

Includes fixes to have tests pass
2021-05-12 11:25:41 +09:30
niftynei
dc758f616b tests: mark as v1 or v2 openchannel required 2021-05-12 11:25:41 +09:30
niftynei
3a2d602922 tests: add marker for v1/v2 channel opens
Tests that will only run when !EXPERIMENTAL_DUAL_FUND:

	@pytest.marker.openchannel('v1')
	def test_...()

Tests that will only run when EXPERIMENTAL_DUAL_FUND:

	@pytest.marker.openchannel('v2')
	def test_...()
2021-05-12 11:25:41 +09:30
niftynei
d0bbf07655 tests: not DEVELOPER -> mark.developer
Nicer syntaxtic sugar for marking pytests as 'developer required'
2021-05-12 11:25:41 +09:30
niftynei
b4e24ac8ba df: anchor outputs are on if EXP_DF
So we should treat it the same as EXPERIMENTAL_FEATURES
2021-05-12 11:25:41 +09:30
niftynei
484d6bde83 tests: move EXP_DF into the testing utils 2021-05-12 11:25:41 +09:30
niftynei
7a488cc11e df-tests: remove df_accepter plugin
The `funder` plugin, does the same stuff as this temporary test plugin,
so we move over to using that instead.
2021-05-12 11:25:41 +09:30
niftynei
de854fa9d4 psbt: really hacky patch over libwally's 0-input txs parse fail
Issue being tracked at https://github.com/ElementsProject/libwally-core/pull/273
2021-05-12 11:25:41 +09:30
niftynei
ef9d8bcd5a dual-fund: reconnections were borked, this fixes them 2021-05-12 11:25:41 +09:30
niftynei
09b5c906fd funder, startupregtest: fixup default config to be 100
Fails on start otherwise
2021-05-12 11:25:41 +09:30
niftynei
6dc954bb91 df-bugs: rm duplicate call to channeld
This gets called from channel_set_owner, which both `delete_channel` and
the `channel_fail_reconnect` pathways call.

Fixes crash
------------------------------------------------------ Captured stderr teardown -------------------------------------------------------
lightning_connectd: peer_disconnected unknown peer: 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518 (version v0.10.0-144-gfaf98c9)
0x560e90d59d08 send_backtrace
        common/daemon.c:39
0x560e90d648a5 status_failed
        common/status.c:214
0x560e90d50e8f peer_disconnected
        connectd/connectd.c:1606
0x560e90d510d5 recv_req
        connectd/connectd.c:1662
0x560e90d5a266 handle_read
        common/daemon_conn.c:31
0x560e90d98ccb next_plan
        ccan/ccan/io/io.c:59
0x560e90d998b0 do_plan
        ccan/ccan/io/io.c:407
0x560e90d998f2 io_ready
        ccan/ccan/io/io.c:417
0x560e90d9bb8a io_loop
        ccan/ccan/io/poll.c:445
0x560e90d512c8 main
        connectd/connectd.c:1735
0x7fbdb828b0b2 ???
        ???:0
0x560e90d4a6dd ???
        ???:0
0xffffffffffffffff ???
        ???:0
2021-05-11 15:37:24 +09:30
Rusty Russell
e960b1015d !fixup
Log this so I can try to find dual fund connnectd race.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-11 11:25:16 +09:30
Rusty Russell
d8e68893f5 bcli: become less aggressive with onchain fee levels.
Users are more upset recently with the cost of unilateral closes
than they are the risk of being cheated.  While we complete our
anchor implementation so we can use low fees there, let's
get less aggressive (we already have 34 or 18 blocks to close
in the worst case).

The changes are:

- Commit transactions were "2 CONSERVATIVE" now "6 ECONOMICAL".
- HTLC resolution txs were "3 CONSERVATIVE" now "6 ECONOMICAL".
- Penalty txs were "3 CONSERVATIVE" now "12 ECONOMICAL".
- Normal txs were "4 ECONOMICAL" now "12 ECONOMICAL".

There can be no perfect levels, but we have had understandable
complaints recently about how high our default fee levels are.

Changelog-Changed: Protocol: channel feerates reduced to bitcoind's "6 block ECONOMICAL" rate.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-11 11:25:16 +09:30
Rusty Russell
9a22e7b3a1 plugins/bcli: make feerate calls more changeable.
This make it clearer what we're doing, IMHO, so we can easily alter
the levels if we want.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-11 11:25:16 +09:30
Rusty Russell
9825f32874 lightningd: implement --log-timestamps=false.
Fixes: #4494
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: config: New option `log-timestamps` allow disabling of timestamp prefix in logs.
2021-05-05 17:19:19 -05:00
Christian Decker
96acafcef3 libhsmd: Prefix check_client_capabilities
I wante to hide it inside the library, but it is good to have a single
place to verify that the client was permitted to send a message we are
handling, so make it officially part of the interface by prefixing it.
2021-05-04 11:18:52 +09:30
Christian Decker
fb2b107bef libhsmd: Cleanup the bip32_key_version 2021-05-04 11:18:52 +09:30
Christian Decker
4d43e84afd libhsmd: Finalize encapsulation of hsmd data in libhsmd
Changelog-Added: libhsmd: Extracted the `hsmd` logic into its own library for other projects to use
2021-05-04 11:18:52 +09:30
Christian Decker
c9e9581b35 libhsmd: Migrate bolt12 initialization into hsmd_init 2021-05-04 11:18:52 +09:30
Christian Decker
d30392c432 libhsmd: Start migrating initialization to hsmd_init 2021-05-04 11:18:52 +09:30
Christian Decker
befa7fb9f0 libhsmd: Clean up includes in hsmd.c 2021-05-04 11:18:52 +09:30
Christian Decker
1efa792edc libhsmd: Migrate handle_sign_delayer_payment_to_us 2021-05-04 11:18:52 +09:30
Christian Decker
fe82181fe3 libhsmd: Migrate handle_sign_remote_htlc_to_us 2021-05-04 11:18:52 +09:30