Commit Graph

247 Commits

Author SHA1 Message Date
Rusty Russell
439dc0f991 gossipd/gossip_store.h: fix check-source for new source file header guards
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-26 00:20:01 +00:00
Christian Decker
82e1f5ade1 gossip: Make gossip_store_append private
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-25 23:56:59 +00:00
Christian Decker
44e23b3773 gossip: Replay the entire store on init instead of when idle
This now works because we no longer call out to masterd or bitcoind to verify
the channels. It's also rather quick and silent so we can just process all
stored messages until we're done.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-25 23:56:59 +00:00
Christian Decker
c4ea79cc5c Revert gossip: Track whether we read a message from store or peer
Messages from peers and messages from the gossip_store now have completely
different entrypoints, so we don't need to trace their origin around the message
handling code any longer.
2018-03-25 23:56:59 +00:00
Christian Decker
3826518bd6 gossip: Wrap node_announcements in gossip_store messages
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-25 23:56:59 +00:00
Christian Decker
db13c7e851 gossip: Wrap channel_updates in the gossip_store as well
Bypasses verification when loading from the gossip_store.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-25 23:56:59 +00:00
Christian Decker
6e01f38d7d gossip: Use the custom gossip wire msg to wrap channel_announcements
This stores and reads the channel_announcements in the wrapping message which
allows us to store associated data with the raw channel_announcements.

The gossip_store applies channel_announcements directly but it also returns it,
and it gets discarded as a duplicate. In the next commit we'll have gossip_store
apply all changes, bypassing verification, so the duplication is only temporary.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-25 23:56:59 +00:00
Christian Decker
6894f20927 gossip: Change gossip_store message length to u32
Since we now store additional data along with the original messages they exceed
the length of the peer wire protocol messages.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-25 23:56:59 +00:00
Christian Decker
b5602a0973 gossip: Add version byte to the gossip_store
If we have a non-empty file and the version doesn't match, then we truncate and
write our own version. If the file is empty we write our version and the
truncate becomes a no-op

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-25 23:56:59 +00:00
Christian Decker
0a5ea76d77 gossip: Add message types to store gossip msgs and associate data
Since we may want to extend the on-disk format by adding custom information we
may as well just go the extra mile and reuse the serialization primitives we
already have.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-25 23:56:59 +00:00
Christian Decker
1b6db5fd52 gossip: Remove redundant replaysize
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-25 23:56:59 +00:00
Christian Decker
96ad0e7044 gossip: Extract network changes into their own functions
Moves any modifications based on an incoming gossip message into its own
function separate from the message verification. This allows us to skip
verification when reading messages from a trusted source, e.g., the
gossip_store, speeding up the gossip replay.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-25 23:56:59 +00:00
Christian Decker
a571bf9d3a gossip: Track whether we read a message from store or peer
When we read from the gossip_store we set store=false so that we don't duplicate
messages in the store.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-25 23:56:59 +00:00
Christian Decker
2879d78f22 gossip: Switch gossip_store to use a single fd
As proposed by @rustyrussell this makes it a bit easier to truncate and sync on
read errors.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-25 23:56:59 +00:00
Christian Decker
5944524d01 gossip: Avoid reading gossip messages that we just wrote
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-25 23:56:59 +00:00
Christian Decker
1135760a2f gossip: Add desctructor to gossip_store
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-25 23:56:59 +00:00
Christian Decker
8de6d9a7e2 gossip: Avoid storing messages from gossip_store twice
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-25 23:56:59 +00:00
Christian Decker
1a5a4f5853 gossip: Replay gossip messages from the gossip_store
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-25 23:56:59 +00:00
Christian Decker
03fd5afbc5 gossip: Store incoming gossip messages to gossip_store
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-25 23:56:59 +00:00
Christian Decker
5c14f24bb3 gossip: Add gossip_store to the routing_state
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-25 23:56:59 +00:00
Christian Decker
e750d3cda1 gossip: Move error return into peer handler
Ee will be replaying gossip messages from the gossip_store soon. This means that
not all messages originate from a peer, so we move the queuing of error messages
up into the peer message handler.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-25 23:56:59 +00:00
Christian Decker
49b0c375ce gossip: Added gossip store primitives
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-25 23:56:59 +00:00
practicalswift
a4059ef83e Use expected LIGHTNING_DIR_FILE_H define 2018-03-25 23:54:21 +00:00
practicalswift
98f49c0837 Remove include in file foo.c that is already included in foo.h 2018-03-25 23:54:21 +00:00
practicalswift
0bf1b01425 Fix typos 2018-03-25 15:53:01 +02:00
practicalswift
d4aadd5e26 tests: Use correct privkey when calculating pubkey used in new_routing_state(...) call 2018-03-25 15:44:05 +02:00
practicalswift
03e2e404f2 Mark unused parameters as such (using "UNUSED") 2018-03-16 01:17:40 +00:00
practicalswift
20ae038080 Remove redundant declaration 2018-03-16 01:17:40 +00:00
Rusty Russell
e63b7bb539 take: allocate temporary variables off NULL.
If we're going to simply take() a pointer, don't allocate it off a random
object.  Using NULL makes our intent clear, particularly with allocating
packets we're going to take() onto a queue.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-16 00:16:10 +00:00
Rusty Russell
0a6e3d1e13 utils: remove tal_tmpctx altogether, use global.
In particular, we now only free tmpctx at the end of main().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-16 00:16:10 +00:00
Rusty Russell
ccc9414356 status: remove trc context now we have tmpctx.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-16 00:16:10 +00:00
Rusty Russell
2d919d56cb gossipd: make struct queued_message private.
Callers don't need it, and when we add timestamps it just makes
for more places to change.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-14 02:19:37 +00:00
Rusty Russell
5e333b75b9 daemon_conn: simplify msg_queue_cleared_cb.
Now it just returns true if it queued something.  This allows it
to queue multiple packets, and lets it share code paths with other code
in future patches.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-14 02:19:37 +00:00
Rusty Russell
87effd90c2 gossipd: Revert 6afc7dcc09.
This bandaid was solved properly by 94711969f9
where other daemons say where they were up to.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-14 02:19:37 +00:00
Rusty Russell
afe61cb841 gossipd: honor LOCAL_INITIAL_ROUTING_SYNC.
We currently spam the peer with all gossip whether they want it or not.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-14 02:19:37 +00:00
Rusty Russell
46cc7c281e features: more general accessor functions.
As we add more features, the current code is insufficient.

1. Keep an array of single feature bits, for easy switching on and off.
2. Create feature_offered() which checks for both compulsory and optional
   variants.
3. Invert requires_unsupported_features() and unsupported_features()
   which tend to be double-negative, all_supported_features() and
   features_supported().
4. Move single feature definition from wire/peer_wire.h to common/features.h.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-14 02:19:37 +00:00
Rusty Russell
1f443df428 gossipd: use the broadcast structure to hold gossip messages.
We currently keep two copies; one in the broadcast structure to send
in order, and one in the routing information.  Since we already keep
the broadcast index in the routing information, use that.
Conveniently, a zero index is the same as the old NULL test.

Rename struct node's announcement_idx to node_announce_msgidx to
make it match the other users.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-14 02:19:37 +00:00
Rusty Russell
b8285db263 gossipd: annotate replace_broadcast that we take() the payload, make it const.
We tal_dup_arr() it, which does take.  Make it const in the structure;
the tal_dup_arr() removes the const, so it compiles without it, but it's
misleading.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-14 02:19:37 +00:00
Rusty Russell
1290f305a2 gossipd: remove tag.
We only access via index.  We do, however, want to clean up when we
delete nodes and channels, so we tie lifetimes to that.  This leads
us to put the index into 'struct queued_message'.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-14 02:19:37 +00:00
Rusty Russell
882f9f258f gossipd: remove type from broadcast.
It's not necessary now we only access by index.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-14 02:19:37 +00:00
Rusty Russell
060182fb65 gossipd: remove unused queue_broadcast.
We only ever access broadcast via the index (which is way faster!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-14 02:19:37 +00:00
Rusty Russell
ba34e6cdc9 gossipd: use u64 for broadcast index.
uintmap uses uint64_t, we should here too.

Couldn't resist drive-by style fix for increment :)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-14 02:19:37 +00:00
Rusty Russell
640ff4b4b9 gossipd: cleanups due to feedback from cdecker.
1. make queue_peer_msg() use both if branches, as both equally likely.
2. Remove redundant *scid = NULL in handle_channel_announcement.
3. Log failing pending channel_updates.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-13 16:34:55 +01:00
Rusty Russell
1dccbb30f9 gossip: send error messages on grossly malformed channel_update.
As per BOLT #7.

We don't do this for channel_update which are queued because the
channel_announcement is pending though.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-13 16:34:55 +01:00
Rusty Russell
5d77183c94 gossip: send error messages on grossly malformed channel_announcement.
As per BOLT #7.  We also give more exact diagnosis.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-13 16:34:55 +01:00
Rusty Russell
8091e0679d gossipd: do all malformdness checks on node_announcement before queueing.
If the channel is pending, we queue the node_announcment and if the channel
is OK we re-call process_node_announcement.  Make sure that second call
won't fail if the first succeeded.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-13 16:34:55 +01:00
Rusty Russell
6d72550707 gossip: send error messages on grossly malformed node_announcement.
As per BOLT #7.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-13 16:34:55 +01:00
Rusty Russell
39c781aebb gossipd: allow calling of peer_error on peers which aren't local.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-13 16:34:55 +01:00
Rusty Russell
81e55065d9 gossipd: don't die if we get a bad msg from another daemon.
We don't trust them, by design (except master and hsmd).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-13 16:34:55 +01:00
Christian Decker
74a444eb7a jsonrpc: Return the channel capacity for listchannels
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-12 22:34:51 +00:00