Commit Graph

46 Commits

Author SHA1 Message Date
Rusty Russell
d40d22b68e gossipd: don't try to connect to non-routable addresses.
Someone could try to announce an internal address, and we might probe
it.

This breaks tests, so we add '--dev-allow-localhost' for our tests, so
we don't eliminate that one.  Of course, now we need to skip some more
tests in non-developer mode.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
practicalswift
abf510740d Force the use of the POSIX C locale for all commands and their subprocesses 2018-04-27 14:02:59 +02:00
Christian Decker
64fbea1528 gossip_store: Save local_add_channel messages and replay them
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-22 12:50:34 +02:00
Christian Decker
7497f972f1 moveonly: Move handle_local_add_channel to routing.h
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-22 12:50:34 +02:00
Rusty Russell
b0c2e3cd5c gossipd: use a separate CSV file for the gossip_store types.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-11 15:58:18 +02:00
Rusty Russell
2b8293c9f6 gossipd: don't use pwrite, better error messaging on init.
Since we open with O_APPEND, any write() will append as we want it to.

But we want to distinguish a new store creation from a truncation due
to bad version.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-11 15:58:18 +02:00
Christian Decker
63f22d70b5 gossip: Store channel deletions so we don't re-add them on restart
If we only remember the actions that added channels then we'd restore them when
re-reading the gossip_store, so put a tombstone in there to remember to delete
it. These will be cleared upon re-writing the store since the announcements wont
be written anymore.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-30 16:35:00 +02:00
Christian Decker
3826518bd6 gossip: Wrap node_announcements in gossip_store messages
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-25 23:56:59 +00:00
Christian Decker
db13c7e851 gossip: Wrap channel_updates in the gossip_store as well
Bypasses verification when loading from the gossip_store.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-25 23:56:59 +00:00
Christian Decker
6e01f38d7d gossip: Use the custom gossip wire msg to wrap channel_announcements
This stores and reads the channel_announcements in the wrapping message which
allows us to store associated data with the raw channel_announcements.

The gossip_store applies channel_announcements directly but it also returns it,
and it gets discarded as a duplicate. In the next commit we'll have gossip_store
apply all changes, bypassing verification, so the duplication is only temporary.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-25 23:56:59 +00:00
Christian Decker
5c14f24bb3 gossip: Add gossip_store to the routing_state
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-25 23:56:59 +00:00
practicalswift
20ae038080 Remove redundant declaration 2018-03-16 01:17:40 +00:00
Rusty Russell
0a6e3d1e13 utils: remove tal_tmpctx altogether, use global.
In particular, we now only free tmpctx at the end of main().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-16 00:16:10 +00:00
Rusty Russell
46cc7c281e features: more general accessor functions.
As we add more features, the current code is insufficient.

1. Keep an array of single feature bits, for easy switching on and off.
2. Create feature_offered() which checks for both compulsory and optional
   variants.
3. Invert requires_unsupported_features() and unsupported_features()
   which tend to be double-negative, all_supported_features() and
   features_supported().
4. Move single feature definition from wire/peer_wire.h to common/features.h.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-14 02:19:37 +00:00
Rusty Russell
1290f305a2 gossipd: remove tag.
We only access via index.  We do, however, want to clean up when we
delete nodes and channels, so we tie lifetimes to that.  This leads
us to put the index into 'struct queued_message'.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-14 02:19:37 +00:00
Rusty Russell
882f9f258f gossipd: remove type from broadcast.
It's not necessary now we only access by index.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-14 02:19:37 +00:00
Rusty Russell
8091e0679d gossipd: do all malformdness checks on node_announcement before queueing.
If the channel is pending, we queue the node_announcment and if the channel
is OK we re-call process_node_announcement.  Make sure that second call
won't fail if the first succeeded.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-13 16:34:55 +01:00
Rusty Russell
6d72550707 gossip: send error messages on grossly malformed node_announcement.
As per BOLT #7.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-13 16:34:55 +01:00
Rusty Russell
dace9bfdcf gossipd: the great renaming.
We already have 'struct node', so rename 'struct routing_channel' to
'struct chan', and 'struct node_connection' to 'struct half_chan'.

Other minor changes:
1. rstate->channels -> rstate->chanmap.
2. 'connections' -> 'half'.
3. connection_to -> half_chan_to
4. connection_from -> half_chan_from

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-04 23:25:53 +01:00
Rusty Russell
21b2f37dfb routing: return entire routing_channel array from find_route.
Returning the separate first routing_channel was a weird API: just
return the entire array.  Sure, we have to treat the first node a bit
differently (because we don't charge ourselves fees), but it's still
simpler.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-04 23:25:53 +01:00
Rusty Russell
56349ab008 routing: work with struct routing_channel not struct node_connection.
To remove the redundant fields in `struct node_connection` (ie. 'src'
and 'dst' pointers) we need to deal with `struct routing_channel`.
This means we get a series of channels, from which the direction is
implied, so it's a bit more complex to decode.  We add a helper
`other_node` to help with this, and since we're the only user of
`connection_to` we change that function to return the index.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-04 23:25:53 +01:00
Rusty Russell
fd9c0c8543 routing: move struct node_connection into struct routing_channel.
No need to have pointers since they're always there.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-03 19:29:35 +01:00
Rusty Russell
33726b0a08 gossip: instead of refresh interval, have routing know prune_timeout.
This is twice the 'update_channel_interval' we get handed.

We delete the non-existent channel_add_connection and delete_connection
declarations from the header too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-03 19:29:35 +01:00
Rusty Russell
942d04ba87 gossipd: simplify channel_announce handling.
We make new_routing_channel() populate both connections
(active=false), so local_add_channel becomes simpler.  We also
suppress listchannels output of active=false unannounced channels, to
avoid breaking tests (also, these are unusable, so it makes sense to
omit them)

It also seems the logic in add_channel_direction is legacy: a
channel_announce cannot replace the scid (that would be a different
channel), we don't allow duplicate announcements, and the announcement
is never NULL.

And since we disallow repeated channel_announce already, I believe
'forward' is always true, greatly simplifying the logic in
handle_pending_cannouncement.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-03 19:29:35 +01:00
Rusty Russell
9b900138d0 gossip: put 'routing_channel' in charge of 'node_connection'.
This makes 'routing_channel' the primary object in the system; it can have
one or two 'node_connection's attached, and points to two nodes.

The nodes are freed when no more routing_channel refer to them.  The
routing_channel are freed when they contain no more 'node_connection'.
This fixes #1072 which I surmise was caused by a dangling
routing_channel after pruning.

Each node contains a single array of 'routing_channel's, not one for
each direction.  The 'routing_channel' itself orders nodes in key
order (conveniently the index is equal to the direction flag we use),
and 'node_connection' with source in the same order.

There are helpers to assist with common questions like "which
'node_connection' leads out of this node?".

There are now two ways to find a channel:
1. Direct scid lookup via rstate->channels map.
2. Node key lookup, followed by channel traversal.

Several FIXMEs are inserted for where we can now do things more optimally.

Fixes: #1072
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-03 19:29:35 +01:00
ZmnSCPxj
aff52ce5a8 gossipd: Implement improved randomization of routes.
Fixes: #928
2018-02-26 02:36:27 +00:00
Rusty Russell
e92b710406 tools/generate-wire.py: remove length argument from fromwire_ routines.
We always hand in "NULL" (which means use tal_len on the msg), except
for two places which do that manually for no good reason.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-20 22:36:21 +01:00
Rusty Russell
26617d1d64 gossipd/test: update mocks.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-14 11:31:58 +01:00
Rusty Russell
fd498be7ca status: generate messages rather than marshal/unmarshal manually.
Now we have wirestring, this is much more natural.  And with the
24M length limit, we needn't be so concerned about dumping 64k peer
messages in hex.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-08 19:07:12 +01:00
Rusty Russell
84bf60f934 status: add multiple levels of logging.
status_trace maps to status_debug.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-07 00:46:49 +00:00
Christian Decker
26b5588656 gossip: Use replace_broadcast when enqueuing a channel_announcement
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-02-05 18:28:39 +00:00
ZmnSCPxj
67e2275383 gossipd: Add handling of channel_update to routing failures. 2018-02-01 00:46:06 +00:00
ZmnSCPxj
b255d82ae2 gossipd: Implement gossip_routing_failure for master to report routing failures. 2018-02-01 00:46:06 +00:00
Rusty Russell
047a2ea043 gossip: don't use assert around code with side effects.
The use of status_failed() requires a stubs update, which fails
with unnamed parameters, so tweak the status.h header as well.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-22 15:47:17 +01:00
Christian Decker
83caf1fdab routing: Add tracking of eviction for broadcasts
We should never be evicting channel_announcements because a) they were
deeply buried and should not change the short_channel_id/tag, b) they
are static.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-22 15:47:17 +01:00
Christian Decker
1b28220b64 Revert "broadcast: don't reorder channel_announce when we get the real one."
This reverts commit f293ff0a6a.
2017-12-22 15:47:17 +01:00
Rusty Russell
810abb6b21 bitcoin: create new wrapper type bitcoin_blkid, log backward endianness.
It's just a sha256_double, but importantly when we convert it to a
string (in type_to_string, which is used in logging) we use
bitcoin_blkid_to_hex() so it's reversed as people expect.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-21 11:05:38 +00:00
Rusty Russell
f293ff0a6a broadcast: don't reorder channel_announce when we get the real one.
If channel_announce is rebroadcast, it should replace the existing one
in-place.  We currently only do this if we start from the unsigned one
and replace it with the signed one when we hit 6 confirms.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-21 09:13:28 +01:00
Rusty Russell
a7eee0b669 routing: precalc per-block risk factor.
Saves a little by doing up-front calculation.

# 1M nodes:
$ /gossipd/test/run-bench-find_route 1000000 1 > /tmp/out
=> 42863 msec

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-18 22:03:31 +01:00
Rusty Russell
589cb673ce routing: remove negative fee support.
We can't get them; channel_update doesn't support it.

# 1M nodes:
$ /gossipd/test/run-bench-find_route 1000000 1 > /tmp/out
=> 47677 msec

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-18 22:03:31 +01:00
Christian Decker
19c030ea1f routing: Make routing_state aware of its own ID
This is used to identify our own announcements.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-17 02:44:20 +00:00
Rusty Russell
78cd25d620 ipaddr: rename to wireaddr.
In future it will have TOR support, so the name will be awkward.

We collect the to/fromwire functions in common/wireaddr.c, and the
parsing functions in lightningd/netaddress.c.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-26 21:01:09 +00:00
Rusty Russell
7e022b522c gossipd: don't try to handle padding inside fromwire_ipaddr.
It makes it impossible to embed an ipaddr in another structure, since we
always try to skip over any zeroes, which may swallow a following field.

Do the skip specially for the case where we're parsing routing messages:
we never use padding for our own internal messages anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-24 16:12:22 +02:00
Rusty Russell
ee9e300da0 gossip: fix address descriptor handling.
1. The code to skip over padding didn't take into account max.
2. It also didn't use symbolic names.
3. We are not supposed to fail on unknown addresses, just stop parsing.
4. We don't use the read_ip/write_ip code, so get rid of it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-03 02:01:54 +02:00
Rusty Russell
9c35603275 gossipd/routing: free everything at end of tests.
valgrind complains, but using a destructor on the node map is a good
idea anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-03 02:01:54 +02:00
Rusty Russell
c8aa50a382 gossipd: fix routing issue.
I had a routing problem, and wrote a simple unit test which passed.  So
I wrote one which copied the failure case (and importantly, had a non-1
fee factor), which triggerd it.

In that real example, we underflowed which resulted in us not finding
a route.  Simply don't consider routes which are infinite.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-03 02:01:54 +02:00