Commit Graph

9814 Commits

Author SHA1 Message Date
Rusty Russell
e531a38963 gossipd / plugin: clean up names in struct route_hop.
We're going to unify them, but the names are not the normal ones.

Fix that first.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Rusty Russell
9dadcc858b common/gossip_store: avoid fd pass for new store, use end marker.
This is also simpler.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Rusty Russell
c2a88912e1 common/gossmap: reload once we hit end marker.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Rusty Russell
0d4f014021 gossip_store: create end marker for EOF.
This is better than using the previous "keep statting the file" approach,
since we can also tell you how long the replacement is, to avoid a
gratitous load.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Rusty Russell
167681c709 gossmap: fix gossmap_chan_get_announce() and gossmap_node_get_announce().
These were never used, nor tested.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Rusty Russell
d54abd0e36 gossmap: function to get channel features bitmap.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Rusty Russell
770a328d56 gossmap: save the offset where the channel updates are, function for details.
This takes an extra 8 bytes per channel, but means we can go back and
get more information about them; this is implemented in
gossmap_chan_get_update_details() which is what listchannels will need.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Rusty Russell
d1f8a5f0be gossmap: add gossmap_chan_get_capacity() helper.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Rusty Russell
eb370ab8ff gossmap: remembers when a channel is private.
Steals one bit from the scid_off, and turns that into a relative
offset.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Rusty Russell
cc198748d4 common/json_tok: hoist param_short_channel_id from inside lightningd/
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Rusty Russell
7ef6dd04d9 plugins/test/run-route-overlong: modern overlong routing test.
gossmap equivalent of gossipd/test/run-overlong.c.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Rusty Russell
52e5ca3c11 plugins/libplugin: make headers update-mocks friendly.
In particular, it expects return type at start of line.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Rusty Russell
6f71aae336 plugins/libplugin-pay: extreact route function so we can write unit tests.
New "route()" doesn't need a struct payment, so is easier to test.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Rusty Russell
088cfced18 common/test/run-route and run-route-specific: modern gossmap tests
Equivalent to gossipd/test/run-find_route.c and gossipd/test/run-find_route-specific.c
except they use gossmap.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Rusty Russell
0682ad9c48 fetchinvoice: don't produce invalid JSON on sent_inv timeout.
An unrelated bug in the reverse path triggered this code, and boom.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Rusty Russell
2e3e4a50ec update-mocks: prefer to find function defs from local includes.
This matters for:
	- common/wallet.h vs wallet/wallet.h
	- common/gossip_store.h vs gossipd/gossip_store.h
	- common/json.h vs lightningd/json.h
	- common/ping.h vs lightningd/ping.h

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Rusty Russell
d868d4470a Makefile: don't let update-mocks run unless DEVELOPER and EXPERIMENTAL_FEATURES.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Rusty Russell
25b5e1e099 update-mocks: make sure we cover all test programs.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Rusty Russell
cc6f1fd120 update-mocks: don't try to regenerate files.
If you do update-mocks in a dirty tree, the recursive make that it
uses will try to rebuild things!  Suppress that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Rusty Russell
33736b860a lightningd: attach HTLC timeout to htlc itself, fix gratuitous disconnect bug.
We set the timeout on first HTLC, but didn't clear it if that HTLC failed.

It's saner to have a per-HTLC timeout (since that's what it is!) and
also our timer infra is specially coded to scale approximately infinitely so
trying to optimize this is vastly premature.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Protocol: We would sometimes gratuitously disconnect 30 seconds after an HTLC failed.
2021-05-21 14:45:05 +09:30
Rusty Russell
11180e7aa1 pytest: add test for HTLC timeout on failed htlcs.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-21 14:45:05 +09:30
fiatjaf
0d68febd8d reorganize .gitignore entries across subdirs. 2021-05-18 09:43:50 +09:30
fiatjaf
e17526f64c gitignore more things. 2021-05-18 09:43:50 +09:30
Rusty Russell
e141b71675 pytest: fix flake in test_reconnect_no_update
If l2 didn't get FUNDING_LOCKED from l1 before it disconnected, it
won't be in state CHANNELD_NORMAL: it will be in DUALOPEND_AWAITING_LOCKIN.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-17 18:58:10 -05:00
Christian Decker
0f2009c7a3 make: Do not scan contrib for discouraged functions 2021-05-15 09:31:12 +09:30
Christian Decker
6c67afeb03 make: Add CPPCHECK_OPTS to allow excluding files from the check
With swig we now have C files that are generated with tools that are
not under our control, so provide an escape hatch for them.
2021-05-15 09:31:12 +09:30
Christian Decker
a4c19ba6ba libhsmd: Clean up the build descriptors for libhsmd_python 2021-05-15 09:31:12 +09:30
Christian Decker
323b1a558b libhsmd: Add missing implementation of status functions 2021-05-15 09:31:12 +09:30
Christian Decker
7f15f86718 libhsmd: First working build script for libhsmd-python
Changelog-Added: libhsmd: Added python bindings for `libhsmd`
2021-05-15 09:31:12 +09:30
Rusty Russell
214fdcc9d7 plugin notifications: minor cleanups.
1. We don't need to check for NULL before tal_count(NULL).
2. Use of json_for_each_arr iterator is probably better.
3. Weird indent fixed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-14 10:24:05 +09:30
niftynei
ce1e5bd388 df-test: make flakey test not as flakey
This would flake fairly regularly, what we really care about is
asserting that the l2 node is in CHANNELD_NORMAL state, while the l1
node hasn't progressed that far yet.
2021-05-12 11:25:41 +09:30
niftynei
c75ca8c112 df-bugs: if we've already got the funding_locked, dont renotify
Peer sends funding locked, we tell lightningd who saves it to disk.
Then we restart/reconnect and they retransmit funding_locked. We were
re-notifying lightningd about their lock-in, which was crashing/breaking
things. Instead, we ignore duplicate lock-in messages from the peer.

lightningd-1: 2021-05-11T18:00:12.844Z **BROKEN** 022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59-chan#1: Internal error DUALOPEND_AWAITING_LOCKIN: channel_got_funding_locked twice
2021-05-12 11:25:41 +09:30
niftynei
e375932108 df-tests: test_coin_movement_notices, make work?
No idea how this slipped past the first time
2021-05-12 11:25:41 +09:30
niftynei
71a4a2e31c df: rework closing logic
Trying to put all the disconnect logic into the same path was a dumb
idea. If you asked to reconnect but passed in an 'unsaved' channel, we
would not call the 'reconnect' code.

Instead, we make a differentiation between "unsaved" channels
(ones that we haven't received commitment tx for) and handle the
disconnect for these separate from where we want to do a reconnect.
2021-05-12 11:25:41 +09:30
niftynei
efdc36c8e9 tests: mark test as requiring developer
hangs with EXP_DF when developer=0
2021-05-12 11:25:41 +09:30
niftynei
5cac36724e df-test: v2 has reconnects 2021-05-12 11:25:41 +09:30
niftynei
ef333d5cee df-callbacks: dont log as broken, just reconnect 2021-05-12 11:25:41 +09:30
niftynei
4432672300 df-bug: avoid referencing null channel->owner
If dualopend dies, we shouldn't reference it
2021-05-12 11:25:41 +09:30
niftynei
5ee4c9e46c df: patch for valgrind error
We were freeing the payload, which is then subsequently freed by the
plugin_hook caller. Whoops.

Now we pass through to the callback function and just clean up neatly.

------------------------------- Valgrind errors --------------------------------
Valgrind error file: valgrind-errors.406602
==406602== Invalid read of size 8
==406602==    at 0x12AC93: openchannel2_hook_cb (dual_open_control.c:669)
==406602==    by 0x12AF0A: openchannel2_hook_deserialize (dual_open_control.c:721)
==406602==    by 0x16EF0E: plugin_hook_callback (plugin_hook.c:186)
==406602==    by 0x169746: plugin_response_handle (plugin.c:514)
==406602==    by 0x169959: plugin_read_json_one (plugin.c:620)
==406602==    by 0x169B23: plugin_read_json (plugin.c:665)
==406602==    by 0x1F4076: next_plan (io.c:59)
==406602==    by 0x1F4C5B: do_plan (io.c:407)
==406602==    by 0x1F4C9D: io_ready (io.c:417)
==406602==    by 0x1F6F35: io_loop (poll.c:445)
==406602==    by 0x13D48D: io_loop_with_timers (io_loop_with_timers.c:24)
==406602==    by 0x143388: main (lightningd.c:1111)
==406602==  Address 0x75e7418 is 56 bytes inside a block of size 3,520 free'd
==406602==    at 0x483CA3F: free (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==406602==    by 0x204FB0: del_tree (tal.c:421)
==406602==    by 0x20527E: tal_free (tal.c:486)
==406602==    by 0x122D68: delete_channel (channel.c:124)
==406602==    by 0x129291: channel_disconnect (dual_open_control.c:63)
==406602==    by 0x129364: channel_close_conn (dual_open_control.c:82)
==406602==    by 0x131CF6: peer_please_disconnect (connect_control.c:304)
==406602==    by 0x131DEB: connectd_msg (connect_control.c:326)
==406602==    by 0x172023: sd_msg_read (subd.c:509)
==406602==    by 0x1F4076: next_plan (io.c:59)
==406602==    by 0x1F4C5B: do_plan (io.c:407)
==406602==    by 0x1F4C9D: io_ready (io.c:417)
==406602==  Block was alloc'd at
==406602==    at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==406602==    by 0x204A39: allocate (tal.c:250)
==406602==    by 0x204FFA: tal_alloc_ (tal.c:428)
==406602==    by 0x123165: new_unsaved_channel (channel.c:209)
==406602==    by 0x130D34: peer_start_dualopend (dual_open_control.c:2985)
==406602==    by 0x15BD2A: peer_connected_hook_final (peer_control.c:1105)
==406602==    by 0x16F2E5: plugin_hook_call_ (plugin_hook.c:275)
==406602==    by 0x15BF5C: plugin_hook_call_peer_connected (peer_control.c:1155)
==406602==    by 0x15C16C: peer_connected (peer_control.c:1208)
==406602==    by 0x131E3B: connectd_msg (connect_control.c:332)
==406602==    by 0x172023: sd_msg_read (subd.c:509)
==406602==    by 0x171842: read_fds (subd.c:310)
2021-05-12 11:25:41 +09:30
niftynei
6d3fb11bc6 df-tests: patch for state == AWAITING_UNILATERAL problem
Found on CI where DEVELOPER=0 EXPERIMENTAL_DUAL_FUND=1,
as we turn off automatic reconnects when DEVELOPER=1

This test has been modified to make the error happen every time, and
then fixed.

lightningd-2: 2021-05-07T20:12:03.790Z DEBUG   0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-chan#1: Peer has reconnected, state AWAITING_UNILATERAL
lightningd-2: 2021-05-07T20:12:03.812Z **BROKEN** lightningd: FATAL SIGNAL 6 (version e8b3f78)
lightningd-2: 2021-05-07T20:12:03.812Z **BROKEN** lightningd: backtrace: common/daemon.c:44 (send_backtrace) 0x56384ee072e9
lightningd-2: 2021-05-07T20:12:03.813Z **BROKEN** lightningd: backtrace: common/daemon.c:52 (crashdump) 0x56384ee0733b

----------------------------- Captured stderr call -----------------------------
lightningd: lightningd/peer_control.c:1100: peer_connected_hook_final: Assertion `channel->state == DUALOPEND_OPEN_INIT || channel->state == DUALOPEND_AWAITING_LOCKIN' failed.
lightningd: FATAL SIGNAL 6 (version e8b3f78)
0x56384ee072a1 send_backtrace
	common/daemon.c:39
0x56384ee0733b crashdump
	common/daemon.c:52
0x7f88486a020f ???
	???:0
0x7f88486a018b ???
	???:0
0x7f884867f858 ???
	???:0
0x7f884867f728 ???
	???:0
0x7f8848690f35 ???
	???:0
0x56384eddc94e peer_connected_hook_final
	lightningd/peer_control.c:1100
0x56384edea2ed plugin_hook_call_
	lightningd/plugin_hook.c:275
0x56384eddfeb8 plugin_hook_call_peer_connected
	lightningd/peer_control.c:1156
0x56384eddfeb8 peer_connected
	lightningd/peer_control.c:1209
0x56384edc30cd connectd_msg
	lightningd/connect_control.c:332
0x56384edebe6f sd_msg_read
	lightningd/subd.c:509
0x56384edebfb1 read_fds
	lightningd/subd.c:310
0x56384ee483b0 next_plan
	ccan/ccan/io/io.c:59
0x56384ee4885b do_plan
	ccan/ccan/io/io.c:407
0x56384ee488f8 io_ready
	ccan/ccan/io/io.c:417
0x56384ee4a23c io_loop
	ccan/ccan/io/poll.c:445
0x56384edcabda io_loop_with_timers
	lightningd/io_loop_with_timers.c:24
0x56384edce826 main
	lightningd/lightningd.c:1111
0x7f88486810b2 ???
	???:0
0x56384edb52ad ???
	???:0
0xffffffffffffffff ???
	???:0
2021-05-12 11:25:41 +09:30
niftynei
342f783f53 df-tests: Turn on v2 tests on CI
Let's run more tests with the v2 open protocol
2021-05-12 11:25:41 +09:30
niftynei
558abe288a tests: explicitly mark tests to run both as v1+v2
By default, tests only run as v1 unless marked as v2.

These tests we want to run as both v1+v2

Includes fixes to have tests pass
2021-05-12 11:25:41 +09:30
niftynei
dc758f616b tests: mark as v1 or v2 openchannel required 2021-05-12 11:25:41 +09:30
niftynei
3a2d602922 tests: add marker for v1/v2 channel opens
Tests that will only run when !EXPERIMENTAL_DUAL_FUND:

	@pytest.marker.openchannel('v1')
	def test_...()

Tests that will only run when EXPERIMENTAL_DUAL_FUND:

	@pytest.marker.openchannel('v2')
	def test_...()
2021-05-12 11:25:41 +09:30
niftynei
d0bbf07655 tests: not DEVELOPER -> mark.developer
Nicer syntaxtic sugar for marking pytests as 'developer required'
2021-05-12 11:25:41 +09:30
niftynei
b4e24ac8ba df: anchor outputs are on if EXP_DF
So we should treat it the same as EXPERIMENTAL_FEATURES
2021-05-12 11:25:41 +09:30
niftynei
484d6bde83 tests: move EXP_DF into the testing utils 2021-05-12 11:25:41 +09:30
niftynei
7a488cc11e df-tests: remove df_accepter plugin
The `funder` plugin, does the same stuff as this temporary test plugin,
so we move over to using that instead.
2021-05-12 11:25:41 +09:30
niftynei
de854fa9d4 psbt: really hacky patch over libwally's 0-input txs parse fail
Issue being tracked at https://github.com/ElementsProject/libwally-core/pull/273
2021-05-12 11:25:41 +09:30
niftynei
ef9d8bcd5a dual-fund: reconnections were borked, this fixes them 2021-05-12 11:25:41 +09:30