Commit Graph

6843 Commits

Author SHA1 Message Date
Rusty Russell
4b33b50625 gossipd: ask a peer for *every* channel it knows on startup.
Asking for the last few blocks was logical, but my node is missing
most gossip in practice.

For the moment, simply ask a peer for every channel it knows, once
we're started up.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-18 16:12:42 +02:00
Rusty Russell
79df507442 gossipd: exclude early blocks from random probes.
When probing, no point probing for before lightning became cool.  Current
logic means we often probe below block 500,000, and think things are OK
because there are no short_channel_ids.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-18 16:12:42 +02:00
Christian Decker
be49a599bc elements: Do not get upset if we see a confidential asset or value
I made some assumptions that turn out not to be true, mea culpa.
2019-10-18 16:10:17 +02:00
Christian Decker
d35ec902f4 elements: Work around libwally getting upset with helpful flags
libwally really is pedantic about the kind of hints it will accept.

Signed-off-by: Christian Decker <@cdecker>
2019-10-18 16:10:17 +02:00
Christian Decker
3c3d7e2df4 connectd: Do not clobber the for-variable when resolving over DNS
We were using `i` as index variable in two nested loops. This works as long as
the DNS seed resolves to a single address, but will crash if the node has both
an A as well as an AAAA entry, at which point we'll try to index the hostname
without a matching entry.

Signed-off-by: Christian Decker <@cdecker>
2019-10-18 14:34:49 +02:00
lisa neigut
6b1b99d7de release 0.7.3 2019-10-18 08:32:15 +02:00
Christian Decker
0a641c9443 pytest: Don't assume UTXO ordering when selecting coins
This was failing because the internal coin-selection doesn't go by insertion
order when using postgres.
2019-10-18 08:29:46 +02:00
Christian Decker
ec8d774b29 db: Make column access way more pedantic
We should never access a nulled field, so add an assert to that effect.
2019-10-18 08:29:46 +02:00
Christian Decker
12f40f2227 db: Add _or_default variants for column accesses
We were implicitly relying on sqlite3 behavior that returns the zero-value for
nulled fields when accessing them. This adds the same behavior explicitly to
the DB abstraction in order to reduce `db_column_is_null` checks in the logic,
but still make it evident what is happening here.

Fixes https://github.com/fiatjaf/mcldsp/issues/1

Signed-off-by: Christian Decker <@cdecker>
2019-10-18 08:29:46 +02:00
Christian Decker
894627a287 db: Fix report and commit order for DB transactions
Reported-by: Simon Vrouwe <@SimonVrouwe>
Signed-off-by: Christian Decker <@cdecker>
2019-10-18 08:29:46 +02:00
Christian Decker
2ab56214ca json-rpc: Fix spacing issue in error report
There was a non-breakable space and a missing space in the error message for
missing utxo params.
2019-10-18 08:29:46 +02:00
darosior
4083e077a4 configure: test XChaCha20Poly1305 stream state for HAVE_GOOD_LIBSODIUM
author: @rustyrussell
2019-10-17 23:42:15 +02:00
Rusty Russell
79e2c3f89a gossipd: don't crash if we're forced to discard corrupt gossip store.
When we're in remove_all_gossip, we *don't* call free_chan, but free it
manually.  This trips over the developer-mode check that we called free_chan!
Make it also insert the magic so that destroy_chan_check passes:

lightning_gossipd: gossipd/routing.c:496: destroy_chan_check: Assertion `chan->sat.satoshis == (u64)chan' failed.
lightning_gossipd: FATAL SIGNAL 6 (version v0.7.3rc2-2-gf89d7c1)
0x5632436a4544 send_backtrace
	common/daemon.c:41
0x5632436a45ea crashdump
	common/daemon.c:54
0x7f053c3c7f5f ???
	???:0
0x7f053c3c7ed7 ???
	???:0
0x7f053c3a9534 ???
	???:0
0x7f053c3a940e ???
	???:0
0x7f053c3b9011 ???
	???:0
0x563243698b9d destroy_chan_check
	gossipd/routing.c:496
0x5632436dca46 notify
	ccan/ccan/tal/tal.c:235
0x5632436dcf35 del_tree
	ccan/ccan/tal/tal.c:397
0x5632436dd2c1 tal_free
	ccan/ccan/tal/tal.c:481
0x56324369f004 remove_all_gossip
	gossipd/routing.c:2981
0x563243692f5d gossip_store_load
	gossipd/gossip_store.c:772
0x56324368eff4 gossip_init
	gossipd/gossipd.c:872
0x563243690cbb recv_req
	gossipd/gossipd.c:1580
0x5632436a4a69 handle_read
	common/daemon_conn.c:31
0x5632436cc7ae next_plan
	ccan/ccan/io/io.c:59
0x5632436cd32b do_plan
	ccan/ccan/io/io.c:407
0x5632436cd369 io_ready
	ccan/ccan/io/io.c:417
0x5632436cf52f io_loop
	ccan/ccan/io/poll.c:445
0x56324369102f main
	gossipd/gossipd.c:1700

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-17 23:40:05 +02:00
darosior
f89d7c1d74 hsm encryption: correct salt length 2019-10-17 15:51:55 +02:00
darosior
3c038e4171 hsm encryption: don't include '\n' when deriving the encryption key 2019-10-17 15:51:55 +02:00
lisa neigut
f278416708 changelog: update to rc2 2019-10-15 22:38:34 +02:00
lisa neigut
49d5a36751 funding: integration tests for close_to
Check behavior for user supplied upfront_shutdown_script via close_to


Header from folded patch 'fix__return__not__iff_well_close_to_the_provided_addr.patch':

fix: return  not  iff we'll close to the provided addr
2019-10-15 19:10:05 +02:00
lisa neigut
422b4502d3 funding: add RPC arg to specify a 'close_to' address
Takes advantage of upfront-shutdown-script to permit users to
specify the close-to address for a channel at open, by adding
a `close_to` field to `fundchannel_start`.

Note that this only is in effect if `fundchannel_start` returns
with `close_to` set -- otherwise, peer doesn't
support `option_upfront_shutdown_script`.
2019-10-15 19:10:05 +02:00
Rusty Russell
624b76e32e gossipd: fix stale scid query.
We always ended up sending an empty query when we had stale scids!
And it turns out we consider such a query invalid:

    Bad query_short_channel_ids query_flags 010506226e46111a0b59caaf126043eb5bbf28c34f3a5e332a1fc7b2b73cf188910f000100010100

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-15 18:05:54 +02:00
Rusty Russell
e462bd4de0 gossipd: increase number of gossiping peers.
We only chose 3 peers to gossip with us (down from 8 last release).
There's no justification for this number, or reason to believe that
it is sufficient to keep us in sync.

Be more conservative for now; we can always decrease it later once
we have more data.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-15 18:05:54 +02:00
Rusty Russell
41d8308b68 seeker: be more random with node_announcement probes.
polling the last 32 is fairly useless in practice, since they tend to
be recent nodes; it won't detect long-forgotten ones.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-15 18:05:54 +02:00
Rusty Russell
1e59d2a738 gossipd: count channel_updates on new channels correctly.
If we get a channel_update while we're still verifying the channel_announcement
we didn't set the peer pointer, so it didn't get credit.  As a result, the
seeker tended to think we were done gossiping sooner than we were.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-15 18:05:54 +02:00
Rusty Russell
3702f0f17b pylightning: bump version number.
We should add this to the release checklist!  It was last changed
for version 0.7.1.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-15 17:15:23 +02:00
Rusty Russell
ab48ecf07c pylightning: wrap signmessage/checkmessage.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-15 17:15:23 +02:00
Rusty Russell
ad48814746 doc: man pages for checkmessage and signmessage.
They're a little subtle, so let's spell out exactly what checkmessage
means.  In particular, we avoid discussing key derivation from a
signature, as it tends to get people (like me!) into trouble: we
describe it instead as iterating through every known node.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-14 18:58:44 -05:00
Rusty Russell
e5d9c7effc lightningd: checkmessage can intuit pubkey in some cases.
*If* we know the key has signed something else (as is the case for
channel_announcement) then we can effectively trust the key derivation.

This matches how LND's VerifyMessage works, though in the next patch
we will document it exactly.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-14 18:58:44 -05:00
Rusty Russell
034ed1711c gossipd: fix memleak when we getnodes has no nodes.
In this case, node_arr is NULL.  Triggered by the next test.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-14 18:58:44 -05:00
Rusty Russell
150598e061 tests: add test for signmessage and lightning-checkmessage.
Thanks Twitter helpers @duck1123 and @jochemin for tests!
And @bitconner for the initial test vector.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-14 18:58:44 -05:00
Rusty Russell
1f0b86e575 lightningd: add checkmessage JSON command.
I wanted to call it verifymessage, but then I read the LND API for that
and wanted nothing to do with it!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-14 18:58:44 -05:00
Rusty Russell
b81ed5be86 devtools: lightning-lightning-checkmessage to validate zbase32 node sigs
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-14 18:58:44 -05:00
Rusty Russell
3955ddfce6 lightningd: add zbase32 encoding to signmessage
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-14 18:58:44 -05:00
Rusty Russell
fc9a2a5dba lightningd: add signmessage JSON command.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-14 18:58:44 -05:00
Rusty Russell
889216e0a9 hsmd: add support for signing a lightning msg from nodeid.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-14 18:58:44 -05:00
Rusty Russell
ca53c1b699 gossipd: push our own gossip messages harder.
I had a report of a 0.7.2 user whose node hadn't appeared on 1ml.  Their
node_announcement wasn't visible to my node, either.

I suspect this is a consequence of recent version reducing the amount of
gossip they send, as well as large nodes increasingly turning off gossip
altogether from some peers (as we do).  We should ignore timestamp filters
for our own channels: the easiest way to do this is to push them out
directly from gossipd (other messages are sent via the store).

We change channeld to wrap the local channel_announcements: previously
we just handed it to gossipd as for any other gossip message we received
from our peer.  Now gossipd knows to push it out, as it's local.

This interferes with the logic in tests/test_misc.py::test_htlc_send_timeout
which expects the node_announcement message last, so we generalize
that too.

[ Thanks to @trueptolmy for bugfix! ]
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-14 15:00:37 -05:00
Saibato
a96b4d8363 Tor CHANGELOG updates for 0.7.3
Signed-off-by: Saibato <saibato.naga@pm.me>
2019-10-14 21:08:03 +02:00
Rusty Russell
c617f19a29 Makefile: fix static linking.
And update --enable-static, since it doesn't actually create a static
binary since 65fa1c4f39.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-14 14:04:43 -05:00
lisa neigut
a91b454749 changelog updates for 0.7.3-rc1 2019-10-11 13:05:01 -05:00
darosior
541df1591e Add a changelog line about behavior change on gossip_queries response
Forgot to add it in f88fc3714c and d78d888c2d284660f05134c572f6251785e866f2..
2019-10-11 10:06:10 -05:00
darosior
c0ae3cb8c4 pytest: make test_plugin_slowinit a test again 2019-10-11 10:05:08 -05:00
darosior
744b727206 plugin_control: halve the timeout delay
20 seconds was way too long for UX, and 10 seconds is more than enough for the plugin to error
2019-10-11 10:05:08 -05:00
darosior
362775bc02 plugin: unregister hooks a plugin registered while freeing it 2019-10-11 10:05:08 -05:00
darosior
1e7b332716 plugin_control: don't assume plugin exists on error
Actually it often does not since there was an error..
2019-10-11 10:05:08 -05:00
Rusty Russell
bd55f6d940
common/features: only support a single feature bitset.
This is mainly an internal-only change, especially since we don't
offer any globalfeatures.

However, LND (as of next release) will offer global features, and also
expect option_static_remotekey to be a *global* feature.  So we send
our (merged) feature bitset as both global and local in init, and fold
those bitsets together when we get an init msg.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-11 02:52:04 +00:00
Rusty Russell
9485919a81 queries: make sure scids are in order.
I thought LND had a bug, but turns out it doesn't like out-of-order
short_channel_ids: in fact, the spec says they have to be in order!

This means we use uintmap instead of a htable for unknown_scids and
stale_scids so they're nicely ordered.

But our nodes-missing-announcements probe is harder since they can
also contain duplicates: we switch that to iterate through channels
rather than nodes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-10 21:48:52 -05:00
Rusty Russell
1f9de04ae4 ccan: import updated version to get uintmap_before().
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-10 21:48:52 -05:00
Rusty Russell
48f0362eae seeker: handle non-synced state internally.
We weren't supposed to do any gossiping until we were synced (and thus
knew blockheight), but our seeker_check() didn't wait for it!  Move the
waiting all into seeker.c, so it can handle it all consistently.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-10 21:48:52 -05:00
Rusty Russell
89d97b330e seeker: don't try to fill peers when they connect.
On testing, I found a node which would hang up every time we asked it
for query_short_channel_ids (despite it offering features 0x81, meaning
it should handle this message).

Then it would reconnect, and we'd choose it again as our
PROBING_NANNOUNCES peer!

Instead, leave finding another peer to the once-a-minute
seeker_check() function.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-10 21:48:52 -05:00
Rusty Russell
596ed6a83b gossipd: better description of what's happening with the seeker.
By combining set_state() with selected_peer() we can give a single log
line describing what we're asking for, from whom.

We also add more verbosity to a few key areas, such as gossip rotation
and when gossipd tells a peer to send an error.  And move a comment which
was above the wrong function (due to rebase?).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-10 21:48:52 -05:00
Rusty Russell
a88553ea6d gossipd: fix typo in seeker random probe logic.
No point probing past current block.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-10 21:48:52 -05:00
Rusty Russell
0c7c765a28 seeker: do set_state() in callee, not caller.
This means we sometimes do it redundantly, but this means it's done in
fewer places.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-10 21:48:52 -05:00