Commit Graph

394 Commits

Author SHA1 Message Date
Rusty Russell
9d721ecb99 gossipd: add assertions to try to catch mysterious crash.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-15 11:53:47 +02:00
Rusty Russell
5c19c55841 gossipd: fix take leak when peer is dying.
In this case, local and remote are *both* NULL; so if someone tries to
send a packet with take(), we need to free it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-15 11:53:47 +02:00
Rusty Russell
a7e6cdb418 gossipd: peer->local->peer_out queue should have lifetime of peer->local.
The current code attaches it to peer, which is a slight leak.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-15 11:53:47 +02:00
Rusty Russell
e098578731 gossipd: fix leak when we fail to dup fds.
In this case, peer would stay around, but conn would be freed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-15 11:53:47 +02:00
Rusty Russell
f6ff89e596 gossipd: fix use-after-free when we fail to make connection.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-15 11:53:47 +02:00
Christian Decker
4279e5cdbd gossip: Fix "already reaching" issue
I think this is what is causing #1536: getting disconnected causes gossipd to
attempt to reach the peer again, unconditionally setting the flag to tell the
master. At the same time the master also issues a reaching command (which is
allowed since it is its first), but then it clashes on the already set
flag. Setting this flag only when the master actually needs to be told should
fix this.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-15 01:06:42 +00:00
Christian Decker
985af483cf gossip: Wrap insert_broadcast and gossip_store_add in persistent_broadcast
They should sync up nicely otherwise we may be overestimating the stale rate.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-09 13:38:46 +02:00
Christian Decker
6632f44133 gossip: Disable gossip_store temporarily while replaying messages
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-09 13:38:46 +02:00
Christian Decker
2b5e1ee65f gossip: Enable the consistency check only when really pedantic
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-09 13:38:46 +02:00
Christian Decker
8a5bebed59 gossip: Disable future compactions if we fail a compaction
A failed compaction shouldn't be deadly, but we should also not attempt to do
one on every gossip message after the first one fails.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-09 13:38:46 +02:00
Christian Decker
74a1cbd877 gossip: Implement gossip_store compaction
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-09 13:38:46 +02:00
Christian Decker
b9a2400a5f gossip: Simplify message handling in gossip_store
`gossip_store_add` is the entry point for messages from the network, so it
should do the bookkeeping and disable on failures. `gossip_store_append` is the
shared function that wraps messages and writes it to the given file. This is
shared between the from network path and the compaction path, so we don't
directly use the `gossip_store` instance, but `fd`s.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-09 13:38:46 +02:00
Christian Decker
60efa314fe gossip: Separate writing to gossip_store fd from append
We write both when coming from outside, as well as when compacting, so we
extract the write functionality to use it in both cases.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-09 13:38:46 +02:00
Christian Decker
e6ab594904 gossip: Have gossip_store annotate gossip messages
This makes the exposed interface much smaller, cleaner and will allow us to just
replay gossip messages from the broadcast.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-09 13:38:46 +02:00
Christian Decker
0546ca446d gossip: Pass routing_state to the gossip_store
We'll need it later to annotate the raw gossip messages, e.g., the capacity of a
channel.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-09 13:38:46 +02:00
Christian Decker
eaba5a249a gossip: Introduce bookkeeping into gossip_store for rewrite
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-09 13:38:46 +02:00
Christian Decker
552ddb8dfd gossip: Pass broadcast_state to gossip_store
We'll be sourcing messages from this `broadcast_state` when rewriting the
`gossip_store`.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-09 13:38:46 +02:00
Christian Decker
37dc458b4d gossip: Have the broadcast_state track its message count
This is far more precise than bolting on the stale tracking in the
`gossip_store`.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-09 13:38:46 +02:00
Christian Decker
4e7fc99ae1 gossip: Duplicate removes can result in null pointers in broadcast
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-08 20:00:27 +02:00
Rusty Russell
5d6a9f3fb0 gossipd: check consistency.
This is a hack to check that our gossip state is consistent on every
insert and delete.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-08 17:53:34 +02:00
Rusty Russell
da55d3c0ff gossipd: handle node_announcement when channel_announcement removed.
Two cases:
1. Node no longer has any public channels: remove node_announcement.
2. Node's node_announcement now preceeds all the channel_announcements:
   move node_announcement to the end.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-08 17:53:34 +02:00
Rusty Russell
def18a7bc1 gossipd: implement broadcast_del to delete a specific index.
Required if we want to reorder node_announcement broadcasts.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-08 17:53:34 +02:00
Rusty Russell
a38c619486 gossipd: keep index of node and channel announcements.
This lets detect if a node announce preceeds a channel announce once we
delete the node announcement.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-08 17:53:34 +02:00
Rusty Russell
1bb7713274 gossipd: minor cleanups.
Suggested-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell
035d6067e4 Rename consider_own_node_announce to maybe_send_own_node_announce.
Suggested-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell
5ec454c7b2 gossipd: don't queue node_announce unless we've queued channel_announce.
We *accept* a node_announce if we have a channel_announce, but we
can't queue it until we queue the channel_announce, which we only do
once we have recieved a channel_update.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell
f52245d442 gossipd: support and use zlib encoding in short_channel_id encoding.
We still use uncompressed if zlib turns out to be larger.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell
9e51e196c1 gossipd: dev-set-max-scids-encode-size to artificially force "full" replies.
We cap each reply at a single one, which forces the code into our
recursion logic.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell
118f099dd8 gossip: dev-query-channel-range to test query_channel_range.
We keep a crappy bitmap, and finish when their replies cover
everything we asked.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell
0dda5d4e1c gossipd: handle query_channel_range
We send them all the short_channel_ids we have in a given range.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell
c34b49c356 gossipd: add dev-send-timestamp-filter command for testing timestamp filtering.
Since we currently only (ab)use it to send everything, we need a way to
generate boutique queries for testing.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell
db6a6442cb gossipd: single-thread the gossip timer.
We have a function called 'wake_pkt_out' which is really 'start
gossiping', so rename it to 'wake_gossip_out'.

In addition, it's fired both on a timer, and in response to our first
gossip_timestamp_filter, which leads to very confusing (though,
technically, not incorrect) behavior.

Keep a single timer at all times, which now doubles as the flag to
indicating we're syncing right now.  Set it once we're done syncing
gossip.

Technically this means we got from once-every-60-seconds to
quiet-for-60-seconds-between-gossip, but that's OK.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell
531c82b6ad gossipd: handle gossip_timestamp_filter message.
And initialize filter (to "never") when we negotiated LOCAL_GOSSIP_QUERIES,
and send initial filter message.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell
97bb6c5a28 gossipd: ensure incoming timestamps are reasonable.
This is kind of orthogonal to the other changes, but makes sense: if we
would instantly or never prune the message, don't accept it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell
7a32637b5f gossipd: add timestamp to each broadcast message.
This lets us filter by timestamp.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell
4d8b29089b gossipd: wire up infrastructure to generate query_short_channel_ids msg.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell
7ee5da858c gossipd: handle query_short_channel_ids message.
This doesn't handle zlib yet.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell
32c39c2979 gossipd: send node announcements after short_channel_id replies.
We use the same system as for gossip: we trickle out replies when we're
otherwise idle.

As we trickle out replies to query_short_channel_ids, we remember the
pubkeys of nodes we mention.  At the end, we sort and uniquify, and
then send any node_announcements we have for those.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell
5864415d31 gossipd: infrastructure to handle short_channel_id replies.
We use the same system as for gossip: we trickle out replies when we're
otherwise idle.

This is minimal infrastructure: we don't actually process the
query_short_channel_ids message yet, nor do we append node
announcements.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell
6c6da45f53 wire: Update to lastest BOLT draft.
This includes the gossip query messages.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell
803e4f8895 gossipd: announce nodes after channel announcement.
In general, we need to only publish node announcements after
publishing channel announcements, though we can accept node
announcements as soon as we see channel announcements.  So we keep a
flag for those node_announcement which haven't been broadcast yet.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell
c2cc3823db gossipd: announce own node only after channel announcement actually broadcast.
handle_pending_cannouncement might not actually add the announcment,
as it could be waiting for a channel_update.  We need to wait for
the actual announcement before considering announcing our node.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell
c2189229ca gossipd: only broadcast channel_announcement once we have a channel_update.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell
2431742285 gossipd: don't publish private updates after channel_announce.
We generate new ones anyway; removing this code changes fixes coming
up which now only need to change one place.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Christian Decker
c550fd1752 gossip: Clean up the code to disable a local channel
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-31 02:30:27 +00:00
Christian Decker
c17848a3f3 gossip: Disable local channels after loading the gossip_store
We don't have any connection yet, so how could they be active? Disable both
sides to avoid trying to route through them or telling others to use them as
`contact_points` in invoices.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-31 02:30:27 +00:00
Christian Decker
f2dc406172 moveonly: Hoist gossip_disable_channel higher up
We'll need it in the next commit

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-31 02:30:27 +00:00
Christian Decker
ba31dd2d9d gossip: Avoid sending duplicate disable messages
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-31 02:30:27 +00:00
Christian Decker
8e278044e3 gossip: Disable channels when we lose the connection to the peer
We're telling gossipd about disconnections anyway, so let's just use that signal
to disable both sides of the channel.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-31 02:30:27 +00:00
Christian Decker
3e5b798c60 gossip: Fix disable flags in handle_disable_channel
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-31 02:30:27 +00:00