Commit Graph

433 Commits

Author SHA1 Message Date
Christian Decker
5944524d01 gossip: Avoid reading gossip messages that we just wrote
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-25 23:56:59 +00:00
Christian Decker
1135760a2f gossip: Add desctructor to gossip_store
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-25 23:56:59 +00:00
Christian Decker
8de6d9a7e2 gossip: Avoid storing messages from gossip_store twice
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-25 23:56:59 +00:00
Christian Decker
1a5a4f5853 gossip: Replay gossip messages from the gossip_store
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-25 23:56:59 +00:00
Christian Decker
03fd5afbc5 gossip: Store incoming gossip messages to gossip_store
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-25 23:56:59 +00:00
Christian Decker
5c14f24bb3 gossip: Add gossip_store to the routing_state
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-25 23:56:59 +00:00
Christian Decker
e750d3cda1 gossip: Move error return into peer handler
Ee will be replaying gossip messages from the gossip_store soon. This means that
not all messages originate from a peer, so we move the queuing of error messages
up into the peer message handler.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-25 23:56:59 +00:00
Christian Decker
49b0c375ce gossip: Added gossip store primitives
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-25 23:56:59 +00:00
practicalswift
a4059ef83e Use expected LIGHTNING_DIR_FILE_H define 2018-03-25 23:54:21 +00:00
practicalswift
98f49c0837 Remove include in file foo.c that is already included in foo.h 2018-03-25 23:54:21 +00:00
practicalswift
0bf1b01425 Fix typos 2018-03-25 15:53:01 +02:00
practicalswift
d4aadd5e26 tests: Use correct privkey when calculating pubkey used in new_routing_state(...) call 2018-03-25 15:44:05 +02:00
practicalswift
03e2e404f2 Mark unused parameters as such (using "UNUSED") 2018-03-16 01:17:40 +00:00
practicalswift
20ae038080 Remove redundant declaration 2018-03-16 01:17:40 +00:00
Rusty Russell
e63b7bb539 take: allocate temporary variables off NULL.
If we're going to simply take() a pointer, don't allocate it off a random
object.  Using NULL makes our intent clear, particularly with allocating
packets we're going to take() onto a queue.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-16 00:16:10 +00:00
Rusty Russell
0a6e3d1e13 utils: remove tal_tmpctx altogether, use global.
In particular, we now only free tmpctx at the end of main().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-16 00:16:10 +00:00
Rusty Russell
ccc9414356 status: remove trc context now we have tmpctx.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-16 00:16:10 +00:00
Rusty Russell
2d919d56cb gossipd: make struct queued_message private.
Callers don't need it, and when we add timestamps it just makes
for more places to change.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-14 02:19:37 +00:00
Rusty Russell
5e333b75b9 daemon_conn: simplify msg_queue_cleared_cb.
Now it just returns true if it queued something.  This allows it
to queue multiple packets, and lets it share code paths with other code
in future patches.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-14 02:19:37 +00:00
Rusty Russell
87effd90c2 gossipd: Revert 6afc7dcc09.
This bandaid was solved properly by 94711969f9
where other daemons say where they were up to.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-14 02:19:37 +00:00
Rusty Russell
afe61cb841 gossipd: honor LOCAL_INITIAL_ROUTING_SYNC.
We currently spam the peer with all gossip whether they want it or not.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-14 02:19:37 +00:00
Rusty Russell
46cc7c281e features: more general accessor functions.
As we add more features, the current code is insufficient.

1. Keep an array of single feature bits, for easy switching on and off.
2. Create feature_offered() which checks for both compulsory and optional
   variants.
3. Invert requires_unsupported_features() and unsupported_features()
   which tend to be double-negative, all_supported_features() and
   features_supported().
4. Move single feature definition from wire/peer_wire.h to common/features.h.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-14 02:19:37 +00:00
Rusty Russell
1f443df428 gossipd: use the broadcast structure to hold gossip messages.
We currently keep two copies; one in the broadcast structure to send
in order, and one in the routing information.  Since we already keep
the broadcast index in the routing information, use that.
Conveniently, a zero index is the same as the old NULL test.

Rename struct node's announcement_idx to node_announce_msgidx to
make it match the other users.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-14 02:19:37 +00:00
Rusty Russell
b8285db263 gossipd: annotate replace_broadcast that we take() the payload, make it const.
We tal_dup_arr() it, which does take.  Make it const in the structure;
the tal_dup_arr() removes the const, so it compiles without it, but it's
misleading.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-14 02:19:37 +00:00
Rusty Russell
1290f305a2 gossipd: remove tag.
We only access via index.  We do, however, want to clean up when we
delete nodes and channels, so we tie lifetimes to that.  This leads
us to put the index into 'struct queued_message'.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-14 02:19:37 +00:00
Rusty Russell
882f9f258f gossipd: remove type from broadcast.
It's not necessary now we only access by index.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-14 02:19:37 +00:00
Rusty Russell
060182fb65 gossipd: remove unused queue_broadcast.
We only ever access broadcast via the index (which is way faster!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-14 02:19:37 +00:00
Rusty Russell
ba34e6cdc9 gossipd: use u64 for broadcast index.
uintmap uses uint64_t, we should here too.

Couldn't resist drive-by style fix for increment :)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-14 02:19:37 +00:00
Rusty Russell
640ff4b4b9 gossipd: cleanups due to feedback from cdecker.
1. make queue_peer_msg() use both if branches, as both equally likely.
2. Remove redundant *scid = NULL in handle_channel_announcement.
3. Log failing pending channel_updates.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-13 16:34:55 +01:00
Rusty Russell
1dccbb30f9 gossip: send error messages on grossly malformed channel_update.
As per BOLT #7.

We don't do this for channel_update which are queued because the
channel_announcement is pending though.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-13 16:34:55 +01:00
Rusty Russell
5d77183c94 gossip: send error messages on grossly malformed channel_announcement.
As per BOLT #7.  We also give more exact diagnosis.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-13 16:34:55 +01:00
Rusty Russell
8091e0679d gossipd: do all malformdness checks on node_announcement before queueing.
If the channel is pending, we queue the node_announcment and if the channel
is OK we re-call process_node_announcement.  Make sure that second call
won't fail if the first succeeded.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-13 16:34:55 +01:00
Rusty Russell
6d72550707 gossip: send error messages on grossly malformed node_announcement.
As per BOLT #7.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-13 16:34:55 +01:00
Rusty Russell
39c781aebb gossipd: allow calling of peer_error on peers which aren't local.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-13 16:34:55 +01:00
Rusty Russell
81e55065d9 gossipd: don't die if we get a bad msg from another daemon.
We don't trust them, by design (except master and hsmd).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-13 16:34:55 +01:00
Christian Decker
74a444eb7a jsonrpc: Return the channel capacity for listchannels
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-12 22:34:51 +00:00
Christian Decker
2abf72e7df gossip: Store channel capacity in the routing table
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-12 22:34:51 +00:00
Christian Decker
a8e553098a gossip: Pass output value to gossipd
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-12 22:34:51 +00:00
John Barboza
30b290cb8f Explicit error message when disconnect fails
Whether the peer is not connected or not gossiping.
2018-03-07 16:14:01 +01:00
John Barboza
0a1351fd51 gossipd: command to disconnect from a gossiping peer
lightning-cli disconnect <peer id>
2018-03-07 16:14:01 +01:00
Rusty Russell
dace9bfdcf gossipd: the great renaming.
We already have 'struct node', so rename 'struct routing_channel' to
'struct chan', and 'struct node_connection' to 'struct half_chan'.

Other minor changes:
1. rstate->channels -> rstate->chanmap.
2. 'connections' -> 'half'.
3. connection_to -> half_chan_to
4. connection_from -> half_chan_from

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-04 23:25:53 +01:00
Rusty Russell
61bcb054e0 routing: remove redundant fields from struct node_connection.
The containing `struct routing_channel` contains src and dst, so
remove them.  However, the channel_update msgidx does belong int
`struct node_connection` along with the channel_update.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-04 23:25:53 +01:00
Rusty Russell
172af04247 gossip: remove short_channel_id from struct node_connection.
It's in the containing routing_channel.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-04 23:25:53 +01:00
Rusty Russell
21b2f37dfb routing: return entire routing_channel array from find_route.
Returning the separate first routing_channel was a weird API: just
return the entire array.  Sure, we have to treat the first node a bit
differently (because we don't charge ourselves fees), but it's still
simpler.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-04 23:25:53 +01:00
Rusty Russell
56349ab008 routing: work with struct routing_channel not struct node_connection.
To remove the redundant fields in `struct node_connection` (ie. 'src'
and 'dst' pointers) we need to deal with `struct routing_channel`.
This means we get a series of channels, from which the direction is
implied, so it's a bit more complex to decode.  We add a helper
`other_node` to help with this, and since we're the only user of
`connection_to` we change that function to return the index.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-04 23:25:53 +01:00
Rusty Russell
fd9c0c8543 routing: move struct node_connection into struct routing_channel.
No need to have pointers since they're always there.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-03 19:29:35 +01:00
Rusty Russell
be14b52423 routing: connections are now never null; simplify.
Failure and pruning were the two places where a node_connection could
be freed; now they both deal with entire channels, we can remove the
NULL checks, and the destructor.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-03 19:29:35 +01:00
Rusty Russell
d7b5882f61 routing: don't free a single nc on prune, only entire channel.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-03 19:29:35 +01:00
Rusty Russell
d0f68ffb15 routing: free entire channel, not just node_connection on PERM error.
And do it by reparenting onto the tmpctx, restoring some simpliciy.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-03 19:29:35 +01:00
Rusty Russell
00194b6130 handle_disable_channel: don't use get_connection_by_scid.
This removes the final user, so we remove it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-03 19:29:35 +01:00
Rusty Russell
74ee448bda routing: expose setter for struct node_connection fields.
And use it in gossip's handle_local_add_channel.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-03 19:29:35 +01:00
Rusty Russell
1e4959af1f gossipd: fix case where we get an update on a pruned channel.
We discarded it; we should populate it.  The comment is wrong, since
local_add_channel() doesn't add public channels, and we test that above.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-03 19:29:35 +01:00
Rusty Russell
961321a6d8 gossipd: move route pruning to routing.c.
And reimplement it to walk the channels map rather than the nodes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-03 19:29:35 +01:00
Rusty Russell
33726b0a08 gossip: instead of refresh interval, have routing know prune_timeout.
This is twice the 'update_channel_interval' we get handed.

We delete the non-existent channel_add_connection and delete_connection
declarations from the header too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-03 19:29:35 +01:00
Rusty Russell
b7bf414ac4 gossipd: prune announced-but-not-updated channels eventually.
We currently give them a free pass.  The simplest fix is to give them
an old timestamp on initialization.

We still skip unannounced channels, on the assumption that they're
ours.  And we set the last_update_timestamp to -1 when we convert to
gossip_getchannels_entry to indicate no update.

This breaks the DEVELOPER=1 pruning test, since we hardcode the 1
week timeout.  That's fixed in the next patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-03 19:29:35 +01:00
Rusty Russell
b7ec2c8c9c node_connection: move channel_announcement field into struct routing_channel.
We don't actually *use it*, mind you: the copy in the broadcast message
is the one we use.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-03 19:29:35 +01:00
Rusty Russell
942d04ba87 gossipd: simplify channel_announce handling.
We make new_routing_channel() populate both connections
(active=false), so local_add_channel becomes simpler.  We also
suppress listchannels output of active=false unannounced channels, to
avoid breaking tests (also, these are unusable, so it makes sense to
omit them)

It also seems the logic in add_channel_direction is legacy: a
channel_announce cannot replace the scid (that would be a different
channel), we don't allow duplicate announcements, and the announcement
is never NULL.

And since we disallow repeated channel_announce already, I believe
'forward' is always true, greatly simplifying the logic in
handle_pending_cannouncement.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-03 19:29:35 +01:00
Rusty Russell
6dec2ed1c1 gossipd: hoist new_node_connection above new_routing_channel.
This is moveonly, wanted for next patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-03 19:29:35 +01:00
Rusty Russell
fc410eff60 resolve_channel_req: use get_channel.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-03 19:29:35 +01:00
Rusty Russell
a79879637a gossipd: local_add_channel can just use get_channel()
Also upgrade the status messages in the weird cases.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-03 19:29:35 +01:00
Rusty Russell
6bc634badf gossip: handle_get_update can just use get_channel.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-03 19:29:35 +01:00
Rusty Russell
a9b1d73148 getchannels_req: iterate over channel map rather than nodes.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-03 19:29:35 +01:00
Rusty Russell
5f5d0b3e25 gossip_local_add_channel: remove unused flags field.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-03 19:29:35 +01:00
Rusty Russell
9b900138d0 gossip: put 'routing_channel' in charge of 'node_connection'.
This makes 'routing_channel' the primary object in the system; it can have
one or two 'node_connection's attached, and points to two nodes.

The nodes are freed when no more routing_channel refer to them.  The
routing_channel are freed when they contain no more 'node_connection'.
This fixes #1072 which I surmise was caused by a dangling
routing_channel after pruning.

Each node contains a single array of 'routing_channel's, not one for
each direction.  The 'routing_channel' itself orders nodes in key
order (conveniently the index is equal to the direction flag we use),
and 'node_connection' with source in the same order.

There are helpers to assist with common questions like "which
'node_connection' leads out of this node?".

There are now two ways to find a channel:
1. Direct scid lookup via rstate->channels map.
2. Node key lookup, followed by channel traversal.

Several FIXMEs are inserted for where we can now do things more optimally.

Fixes: #1072
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-03 19:29:35 +01:00
Rusty Russell
f8426600a6 gossipd: don't create a routing_channel while we're waiting.
We're going to make it a first-class citizen, and pending routing_channel
are not real ones (in particular, we don't want to create pending nodes).

We had a linked list called rstate->pending_cannouncement which we didn't
actually use, so put that back for now and add a FIXME to use a faster
data structure.

We need to check that list now in handle_channel_update, but we never
have a real routing_channel and a pending, unless the routing_channel
isn't public.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-03 19:29:35 +01:00
Rusty Russell
2ec38ec687 gossip: factor out appending code in getchannels_req().
This moveonly makes a coming cleanup easier.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-03 19:29:35 +01:00
John Barboza
b50912d8b1 Print node alias when listing peers 2018-03-02 02:25:30 +00:00
Rusty Russell
ca4603455b short_channel_id: remove short_channel_id_to_uint accessor.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-01 23:33:56 +01:00
Rusty Russell
c5d41a23d7 short_channel_id: just use structeq.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-01 23:33:56 +01:00
Rusty Russell
affc1be4d6 gossipd: get_channel() wrapper for channel lookup.
We rename the current routing.c get_channel to find_channel.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-01 23:33:56 +01:00
Rusty Russell
be5d945e97 gossip: only send out gossip to non-local peers when we're supposed to.
This hook is called when the queue is empty; we should only send gossip
according to the gossip timer.  We're currently dribbling it out after
every message, in violation of the spec.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-28 12:48:45 +01:00
Rusty Russell
c8390ca403 gossipd: fix obsolete comments.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-28 12:48:45 +01:00
Rusty Russell
a073c201e0 gossip: expose and use get_node() helper.
It's a trivial helper function from routing.c, but let's expose it
and use it in gossip.c too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-28 12:48:45 +01:00
Rusty Russell
5970890fae gossipd: use status levels for unusual reporting.
Now we have them, let's use them.  I missed one case deliberately, since
that causes merge conflicts when I replace it in a following patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-28 12:48:45 +01:00
Rusty Russell
04eb101aa6 routing: handle pending replies which are missing.
I'm not completely conviced that we can't end up removing pending things,
so change asserts to simple returns.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-28 12:48:45 +01:00
Rusty Russell
60e20b502e gossipd: simplify pruning code.
If we make destroy_node() remove itself from the map, then we simply
need to free it.

We can batch the frees (as we need) simply by reparenting all the pruned
nodes onto a single temporary parent, then freeing it, relying on tal's
internal datastructures.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-28 12:48:45 +01:00
Rusty Russell
792feb8532 gossip: Only do a single lookup in channel_update.
get_connection_by_scid() and update_to_pending() both do the same
lookup which we did in handle_channel_update().

Do the lookup once, and simplify the others.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-28 12:48:45 +01:00
Rusty Russell
76ff52b601 routing: remove unused txout_state field in struct routing_channel.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-28 12:48:45 +01:00
Rusty Russell
cf3f19524e gossip: formalize passing of siphash_seed.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-26 06:35:02 +00:00
ZmnSCPxj
6767434ea9 routing: Use siphash24 for route randomization
Primary idea by @rustyrussell
2018-02-26 02:36:27 +00:00
ZmnSCPxj
aff52ce5a8 gossipd: Implement improved randomization of routes.
Fixes: #928
2018-02-26 02:36:27 +00:00
ZmnSCPxj
216c52940d gossipd: Add fuzz and seed to getroute request. 2018-02-26 02:36:27 +00:00
Christian Decker
d499c34db2 fixup! options: Add --offline to disable listening and auto-reconnection 2018-02-23 06:30:15 +00:00
Christian Decker
6f6176a91e options: Add --offline to disable listening and auto-reconnection 2018-02-23 06:30:15 +00:00
Rusty Russell
b8c636514b pong: embed version string into ping replies if DEVELOPER=1.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-22 12:24:26 +01:00
practicalswift
99ce580e20 Remove unused parameter rstate in routing_failure_on_nc(struct routing_state *rstate, ...) 2018-02-22 10:46:30 +01:00
practicalswift
0013caedfe Remove unused parameter rstate in get_out_node_connection_of(struct routing_state *rstate, ...) 2018-02-22 10:46:30 +01:00
practicalswift
a3cd9495ef Remove unused parameter rstate in delete_connection(struct routing_state *rstate, ...) 2018-02-22 10:46:30 +01:00
practicalswift
91a9c2923f Mark intentionally unused parameters as such (with "UNUSED") 2018-02-22 01:09:12 +00:00
practicalswift
3dbace3421 Remove redundant casts to same type 2018-02-21 13:07:40 +01:00
Rusty Russell
e92b710406 tools/generate-wire.py: remove length argument from fromwire_ routines.
We always hand in "NULL" (which means use tal_len on the msg), except
for two places which do that manually for no good reason.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-20 22:36:21 +01:00
Rusty Russell
55d962046b Rename (almost) all destructors to destroy_<type>.
We usually did this, but sometimes they were named after what they did,
rather than what they cleaned up.

There are still a few exceptions:
1. I didn't bother creating destroy_xxx wrappers for htable routines
   which already existed.
2. Sometimes destructors really are used for side-effects (eg. to simply
   mark that something was freed): these are clearer with boutique names.
3. Generally destructors are static, but they don't need to be: in some
   cases we attach a destructor then remove it later, or only attach
   to *some* cases.  These are best with qualifiers in the destroy_<type>
   name.

Suggested-by: @ZmnSCPxj
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-14 11:31:58 +01:00
Rusty Russell
26617d1d64 gossipd/test: update mocks.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-14 11:31:58 +01:00
Rusty Russell
e76a0b4ddc gossipd: fix race where we can handoff peer with bad cryptostate.
DEBUG:root:lightningd(16333): 2018-02-08T02:12:21.158Z lightningd(8262): lightning_openingd(0382ce59ebf18be7d84677c2e35f23294b9992ceca95491fcf8a56c6cb2d9de199): Failed hdr decrypt with rn=2

We only hand off the peer if we've not started writing, but that was
insufficient: we increment the sn twice on encrypting packet, so there's
a window before we've actually started writing where this is now
wrong.

The simplest fix is only to hand off from master when we've just written,
and have the read-packet path simply wake the write-packet path.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-13 12:10:35 +01:00
practicalswift
fe670b9aaa Improve log message when receiving unknown message type 2018-02-12 09:31:00 +01:00
ZmnSCPxj
fecfd55e06 routing: Suppress UNUSUAL log message for UPDATE failcodes from local. 2018-02-09 15:13:49 +01:00
Rusty Russell
d5effcb961 test_lightning: fix race on testing, esp. test_closing_different_fees.
We get intermittant failure: WIRE_UNKNOWN_NEXT_PEER (First peer not ready)
because CHANNELD_NORMAL and actually telling gossipd that the channel
is available are distinct things: we need both.

(For test_closing_different_fees, we were testing CHANNELD_NORMAL on
the peer, not on l1, too).

But we may also directly send the announcement sigs if the height is
sufficient, so the simplest is to unify the messages.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-09 13:01:19 +01:00
Rusty Russell
fd498be7ca status: generate messages rather than marshal/unmarshal manually.
Now we have wirestring, this is much more natural.  And with the
24M length limit, we needn't be so concerned about dumping 64k peer
messages in hex.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-08 19:07:12 +01:00
Rusty Russell
526d3a232e tools/generate_wire.py: generate varlen arrays properly.
These are now logically arrays of pointers.  This is much more natural,
and gets rid of the horrible utxo array converters.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-08 19:07:12 +01:00
ZmnSCPxj
0d0ef2c733 gossipd: Implement gossip_mark_channel_unroutable.
Temporarily marks a channel as unroutable.
2018-02-08 00:51:30 +00:00
Rusty Russell
84bf60f934 status: add multiple levels of logging.
status_trace maps to status_debug.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-07 00:46:49 +00:00
Wladimir J. van der Laan
b2abe0d250 gossipd: Move include of time.h to routing.h
Commit a57a2dcb86 introduced a time_t
in routing.h. So also move the time.h include to the header. This
fixes the build on FreeBSD.

Signed-off-by: Wladimir J. van der Laan <laanwj@gmail.com>
2018-02-06 01:36:16 +00:00
Christian Decker
7dd19d5b93 gossipd: Cleanup and less verbose on deferred node_announcements
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-02-05 18:28:39 +00:00
Christian Decker
f6c9c4fc3a gossipd: Use channels map to lookup pending cannouncements
Iterating over lists is so 1980s

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-02-05 18:28:39 +00:00
Christian Decker
b09666c51c gossip: Remember the channel_update broadcast index
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-02-05 18:28:39 +00:00
Christian Decker
0790d9b52f gossip: Add a map for pending node_announcements
We were dropping these on the floor while checking for txout. So now
we add a map that holds announcements while we are checking.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-02-05 18:28:39 +00:00
Christian Decker
35b44d5294 gossip: Remember which broadcast index the node_announcement had
Avoids searching for it in the uintmap.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-02-05 18:28:39 +00:00
Christian Decker
24d52ec23a gossip: Only replace with newer updates while txout check is pending
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-02-05 18:28:39 +00:00
Christian Decker
739e78a8c7 gossip: Reduce verbosity
Now we only report serious failures, or messages that actually changed
the local view.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-02-05 18:28:39 +00:00
Christian Decker
fa069d7d97 gossip: Use the channels map to look up by scid
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-02-05 18:28:39 +00:00
Christian Decker
a88076b1b0 gossip: Adding locally added channels to channels map
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-02-05 18:28:39 +00:00
Christian Decker
26b5588656 gossip: Use replace_broadcast when enqueuing a channel_announcement
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-02-05 18:28:39 +00:00
Christian Decker
01b7e2a7c0 broadcast: Added option to replace a specific index
We are wasting way too much time looking for announcements and updates
in the broadcast. We can just hint where to find the message to be
evicted and safe the traversal.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-02-05 18:28:39 +00:00
Christian Decker
60ab947a2f gossip: Add new channels to channels map
Adding channels that we are currently verifying to the map, and
skipping if we already have a channel at that position.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-02-05 18:28:39 +00:00
Christian Decker
0bc2f8714d gossip: Added uintmap based channels map, indexed by scids
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-02-05 18:28:39 +00:00
ZmnSCPxj
a57a2dcb86 routing: Only mark channel unroutable for a short time.
Fixes: #867
2018-02-03 21:01:58 +01:00
ZmnSCPxj
89663454c8 routing: Factor out function to determine if we can route through a channel. 2018-02-03 21:01:58 +01:00
ZmnSCPxj, ZmnSCPxj jxPCSmnZ
ddf052982a routing: Correct the sense of "PERM bit is NOT set" 2018-02-02 20:36:28 +01:00
Rusty Russell
8fcb45c1b2 routing: use marshalled short_channel_id, not string as tag for channel_announce
We use this technique for the other tags, so use it here too.

This was drawn to my attention when I made more than 10 channels in a
block, and the string changed length:

 Valgrind error file: valgrind-errors.31415
==31415== Conditional jump or move depends on uninitialised value(s)
==31415==    at 0x4C35E20: bcmp (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31415==    by 0x11A624: queue_broadcast (broadcast.c:40)
==31415==    by 0x118D93: handle_pending_cannouncement (routing.c:704)
==31415==    by 0x1109E3: handle_txout_reply (gossip.c:1796)
==31415==    by 0x111177: recv_req (gossip.c:1955)
==31415==    by 0x136723: next_plan (io.c:59)
==31415==    by 0x137220: do_plan (io.c:387)
==31415==    by 0x13725E: io_ready (io.c:397)
==31415==    by 0x138B97: io_loop (poll.c:305)
==31415==    by 0x111352: main (gossip.c:2022)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-01 12:20:15 +01:00
ZmnSCPxj
67e2275383 gossipd: Add handling of channel_update to routing failures. 2018-02-01 00:46:06 +00:00
ZmnSCPxj
b255d82ae2 gossipd: Implement gossip_routing_failure for master to report routing failures. 2018-02-01 00:46:06 +00:00
Christian Decker
7a651c62fa gossip: Pull up the check for new channels before checking the txout
We drop all but the first announcement, so any work that is done for a
channel that we already know is wasted. Pulling this up duplicates
some of the work but allows us to skip the costly txout check.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-02-01 00:12:43 +00:00
Christian Decker
fa1806da61 gossip: Deduplicate short_channel_id checks
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-02-01 00:12:43 +00:00
Christian Decker
e2f5e4bb3c moveonly: Move find_pending_cannouncement up 2018-02-01 00:12:43 +00:00
Rusty Russell
6daa770b0e Minor whitespace/comment spelling fixes.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-29 06:01:56 +00:00
Christian Decker
acc6afc8de gossip: Directly return an error if we don't know a node's address
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-29 06:01:56 +00:00
Christian Decker
932dbd8c2c gossip: Give up connecting after 10 attempts
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-29 06:01:56 +00:00
Christian Decker
f5147f27c9 gossip: Added message for unsuccessful connection attempts
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-29 06:01:56 +00:00
Christian Decker
1da27c51c2 gossip: Remove annoying nonlocal_gossip_broadcast_done status
Reported-by: @RvMP
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-27 02:27:35 +00:00
Christian Decker
3d2ef44064 gossipd: Do not report bind failures for IPv4 2018-01-27 02:27:35 +00:00
Christian Decker
007c349dec gossip: Implement handler for enabling and disabling channels
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-26 01:09:44 +00:00
Christian Decker
c6ae4b521f gossip: Add message to enable and disable a channel
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-26 01:09:44 +00:00
Rusty Russell
cf54f23947 JSONRPC: add id argument to listpeers.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-19 22:23:45 +00:00
Rusty Russell
06c4f6ddca JSONRPC: add optional short_channel_id argument to listchannels
In order to just list one (though it may return two entries, one for each
channel direction!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-19 22:23:45 +00:00
Rusty Russell
e1e7f289fb JSONRPC listnodes: return timestamp, alias and color.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-19 22:23:45 +00:00
Rusty Russell
784ec96841 gossip: save alias from node_announcement
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-19 22:23:45 +00:00
Rusty Russell
ced572b973 JSONRPC: getnodes: rename to listnodes.
Like listinvoice, and add optional 'id' parameter to ask about a
specific node.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-19 22:23:45 +00:00
Christian Decker
f27013c338 gossip: Only skip updates if we don't have channels yet 2018-01-14 23:40:04 +00:00
Christian Decker
8f16f6ca09 gossip: Forget nodes that have no associated channels anymore
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-14 23:40:04 +00:00
Christian Decker
3a42e52bcd gossip: Fix a memcmp with unset memory in broadcast queue
`tal_fmt` overallocates the returned string under some circumstances,
meaning that the trailer of the formatted string is unset, but still
considered in `tal_len`. The solution then is to truncate the
formatted string to the real string length. Only necessary here, since
we mix strings and `tal_len`.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-14 23:40:04 +00:00
Christian Decker
ca6c6feaad gossip: Remove channels older than 2*channel-update-interval
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-14 23:40:04 +00:00
Christian Decker
574dc8cd1f gossip: Send keepalive channel_updates
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-14 23:40:04 +00:00
Christian Decker
4470612016 gossip: Network pruning loop every channel-update-interval/2 seconds
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-14 23:40:04 +00:00
Christian Decker
7fc8e3a4e2 gossip: Pass --channel-update-interval to gossipd
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-14 23:40:04 +00:00
ZmnSCPxj
a2877232af gossipd: Check features in node_announcement and channel_announcement.
Fixes: #548
2018-01-13 11:29:42 +01:00
Christian Decker
160285f3ea gossip: Do not clobber updates attached to a channel announcement
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-12 21:55:50 +01:00
Rusty Russell
6b7b7f1007 test_lightningd.py: fix timing error in test_forward_different_fees_and_cltv
We need to make sure all the updates are known to gossip.  Since
one is the local update, we change that message to look the same.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-12 09:46:56 +01:00
Rusty Russell
d4c8210a9e gossipd: don't hang if we try to connect to already-connected peer.
Closes: #287
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-11 23:47:47 +01:00
Rusty Russell
10b40c524d gossipd: attach updates to pending transactions in preference.
Otherwise, we otherwise end up with out-of-order updates
(ie. preceeding announcements).

I assume that is because of the locally-inserted connections.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-11 23:13:23 +01:00
Rusty Russell
96ed75d8b2 gossipd: handle channel_update while we're still checking channel_announce.
We only need to keep one.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-11 23:13:23 +01:00
Rusty Russell
db30411ba5 gossipd: look up short_channel_id when we see a channel_announcement.
This is done it two parts, since we have to ask the main daemon to do
the lookup for us.

If this becomes a bottleneck, we can have a separate daemon, or even
an RPC pipe to bitcoind ourselves.

Fixes: #403
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-11 23:13:23 +01:00
Rusty Russell
e22dd5ad71 lightningd: implement gossip_get_txout/gossip_get_txout_reply.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-11 23:13:23 +01:00
Rusty Russell
61be62ddb7 gossipd: ensure node_announcement timestamps always increment.
Covers a possible source of subtle bugs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-11 23:13:23 +01:00
Rusty Russell
0f97b8cf36 subdaemon.c: subdaemon_setup() routine for all daemons.
Our handling of SIGPIPE was incoherent and inconsistent, and we had much
cut & paste between the daemons.  They should *ALL* ignore SIGPIPE, and
much of the rest of the boilerplate can be shared, so should be.

Reported-by: @ZmnSCPxj
Fixes: #528
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-08 18:16:39 +01:00
Rusty Russell
a8de8a3140 json_getchannels: add public flag.
Fixes: #509
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-08 18:09:31 +01:00
Christian Decker
202868b677 gossipd: Fix copy-paste error when adding a channel locally
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-02 18:37:39 +01:00
practicalswift
d340ee8abb Remove redundant call to get_connection_by_scid(...) 2018-01-02 18:37:39 +01:00
practicalswift
3d39312212 Fix typos 2018-01-02 15:09:36 +01:00
Christian Decker
fff7efaf97 gossipd: Remove annoying trace 2018-01-02 01:35:59 +00:00
Christian Decker
3923109f51 gossipd: Replace lookup by short_channel_id by endpoint lookup
Sometimes we could get into a situation in which we knew the channel
but couldn't find it via the short_channel_id. That'd result in a
replacement which triggered an assert.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-02 01:35:59 +00:00
Christian Decker
3af2dcba6a gossip: Fix a typo in the add_channel_direction logic
Signed-off-by: Christian Decker <decker.christian@gmail.com>
Reported-by: William Casarin @jb55
2018-01-02 01:20:34 +00:00
practicalswift
d50fb131b6 Avoid passing uninitialized value dummy with uninitialized field addrlen to {initiator,responder}_handshake_ 2017-12-30 12:28:24 +01:00
Rusty Russell
047a2ea043 gossip: don't use assert around code with side effects.
The use of status_failed() requires a stubs update, which fails
with unnamed parameters, so tweak the status.h header as well.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-22 15:47:17 +01:00
Christian Decker
0db821e2cf routing: Fix the announcement detection for channel_announcements
If we side-load a channel, using local-add or the removed JSON-RPC
call, then we could end up in a situation in which a channel is
present, but has no associated channel_announcement. The presence of
the channel_announcement was used to identify new channels, so this
could lead to channels always being considered new. This then caused
the announcements being added to the queue always, resulting in
channel_updates preceeding the announcement.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-22 15:47:17 +01:00
Christian Decker
83caf1fdab routing: Add tracking of eviction for broadcasts
We should never be evicting channel_announcements because a) they were
deeply buried and should not change the short_channel_id/tag, b) they
are static.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-22 15:47:17 +01:00
Christian Decker
1b28220b64 Revert "broadcast: don't reorder channel_announce when we get the real one."
This reverts commit f293ff0a6a.
2017-12-22 15:47:17 +01:00
Rusty Russell
887e9dcc44 travis: reenable check-source (without BOLT text).
We've been slipping, so fix up minor issues too so it compiles.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-21 14:33:27 +01:00
Rusty Russell
810abb6b21 bitcoin: create new wrapper type bitcoin_blkid, log backward endianness.
It's just a sha256_double, but importantly when we convert it to a
string (in type_to_string, which is used in logging) we use
bitcoin_blkid_to_hex() so it's reversed as people expect.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-21 11:05:38 +00:00
Rusty Russell
f293ff0a6a broadcast: don't reorder channel_announce when we get the real one.
If channel_announce is rebroadcast, it should replace the existing one
in-place.  We currently only do this if we start from the unsigned one
and replace it with the signed one when we hit 6 confirms.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-21 09:13:28 +01:00
Rusty Russell
bb601a1eeb gossipd/test/run-bench-find_route: don't abort if we try to route to ourselves
This would fail, and we'd free an uninitialized pointer.

Also, add us to .gitignore and clear up a comment.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 13:57:28 +01:00
Rusty Russell
32cee8620b routing: re-expose half_add_connection.
I made it static in one PR (745c7b223a),
Christian used it in another.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 08:19:28 +00:00
Christian Decker
da0792e69c gossip: Add local channels when channeld tells us to
This adds the channel from us to the remote node and activates it with
our local parameters.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-20 06:14:53 +00:00
Christian Decker
2988e290cf gossip: Added internal gossip message to add a local channel
Couldn't find a good place to put these messages, we probably want to
do the same capability based request routing that we did for the HSM,
but for now this just defines the message in the master messages file.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-20 06:14:53 +00:00
Rusty Russell
bd27eba6f8 bench: reduce defaults for travis run.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-18 22:03:31 +01:00
Rusty Russell
a7eee0b669 routing: precalc per-block risk factor.
Saves a little by doing up-front calculation.

# 1M nodes:
$ /gossipd/test/run-bench-find_route 1000000 1 > /tmp/out
=> 42863 msec

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-18 22:03:31 +01:00
Rusty Russell
2824ff2b30 routing: remove possibility of overflow.
This check is expensive, so just restrict msatoshi going in, as well
as turn off channels charging more than 24x fee.

# 1M nodes:
$ /gossipd/test/run-bench-find_route 1000000 1 > /tmp/out
=> 44164 msec

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-18 22:03:31 +01:00
Rusty Russell
589cb673ce routing: remove negative fee support.
We can't get them; channel_update doesn't support it.

# 1M nodes:
$ /gossipd/test/run-bench-find_route 1000000 1 > /tmp/out
=> 47677 msec

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-18 22:03:31 +01:00
Rusty Russell
21cc904b03 gossipd/test/run-bench-find_route.c: add perfme support.
Compile this, and link from perfme-start and perfme-stop in your path:

/* Simple wrapper to allow a program to perf itself. 
 * Copyright Rusty Russell, Blockstream 2015.
 *
 * This program is free software: you can redistribute it and/or modify
 * it under the terms of the GNU General Public License as published by
 * the Free Software Foundation, either version 3 of the License, or
 * (at your option) any later version.
 *
 * This program is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 * GNU General Public License for more details.
 *
 * See <http://www.gnu.org/licenses/>.
 */
#include <ccan/err/err.h>
#include <ccan/str/str.h>
#include <sys/types.h>
#include <unistd.h>
#include <signal.h>
#include <errno.h>
#include <stdio.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <stdlib.h>

#define PERFME_PREFIX "/tmp/perfme."
#define MAX_ENV_ARGS 20

static void write_noerr(int fd)
{
	int e = errno;
	if (write(fd, "", 1) != 1)
		/* Complain about warn_unused_result fascist bullshit */ ;
	errno = e;
}

/* Child.  Setup pid, run perf. */
static void exec_perf(int pfd[2], const char *perfpid, const char *perfout,
		      pid_t parent)
{
	char pid[STR_MAX_CHARS(pid_t)];
	int i, fd;
	char *cmd, *args[MAX_ENV_ARGS + 5];
	
	fd = open(perfpid, O_CREAT|O_EXCL|O_WRONLY, 0400);
	if (fd < 0) {
		write_noerr(pfd[1]);
		err(1, "opening %s", perfpid);
	}

	sprintf(pid, "%u", getpid());
	if (write(fd, pid, strlen(pid)) != strlen(pid)) {
		write_noerr(pfd[1]);
		err(1, "writing to %s", perfpid);
	}
	close(fd);

	sprintf(pid, "%u", parent);
	cmd = getenv("PERFME");
	if (!cmd)
		cmd = "perf record --call-graph dwarf -q";
	cmd = strdup(cmd);
	for (i = 0; i < MAX_ENV_ARGS; i++) {
		args[i] = strtok(i == 0 ? cmd : NULL, " ");
		if (!args[i])
			break;
	}
	if (i == 0 || i == MAX_ENV_ARGS)
		errx(1, "Too %s args in $PERFME: '%s'",
		     i ? "many" : "few", getenv("PERFME"));

	args[i++] = "-p";
	args[i++] = pid;
	args[i++] = "-o";
	args[i++] = (char *)perfout;
	args[i++] = NULL;

	execvp(args[0], args);
	write_noerr(pfd[1]);
	err(1, "Execing %s", args[0]);
}

int main(int argc, char *argv[])
{
	pid_t parent = argv[1] ? atoi(argv[1]) : getppid();
	char perfout[sizeof(PERFME_PREFIX) + STR_MAX_CHARS(parent)];	
	char perfpid[sizeof(perfout) + sizeof(".pid")];

	err_set_progname(argv[0]);

	sprintf(perfpid, PERFME_PREFIX "%u.pid", parent);
	if (strends(argv[0], "perfme-stop")) {
		char pid[STR_MAX_CHARS(pid_t)];
		int r, fd = open(perfpid, O_RDONLY);
		if (fd < 0)
			err(1, "Opening %s", perfpid);
		r = read(fd, pid, sizeof(pid) - 1);
		if (r < 0)
			err(1, "Reading %s", perfpid);
		pid[r] = 0;
		if (unlink(perfpid) != 0)
			warn("Unlinking %s", perfpid);
		if (atoi(pid) <= 0)
			errx(1, "Invalid pid '%s' from %s", pid, perfpid);
		if (kill(atoi(pid), SIGTERM) != 0)
			err(1, "Stopping %s", pid);
		exit(0);
	} else if (strends(argv[0], "perfme-start")) {
		int pfd[2];

		sprintf(perfout, PERFME_PREFIX "%u", parent);

		/* Use pipe to detect successful exec. */
		if (pipe(pfd) != 0)
			err(1, "Creating pipe");
		
		switch (fork()) {
		case 0:
			close(pfd[0]);
			fcntl(pfd[1], F_SETFD,
			      fcntl(pfd[1], F_GETFD)|FD_CLOEXEC);

			exec_perf(pfd, perfpid, perfout, parent);
		case -1:
			err(1, "Forking");
		default:
			/* Parent.  Wait for child. */
			close(pfd[1]);
			if (read(pfd[0], perfpid, 1) == 1)
				exit(1);
			fprintf(stderr, "Perf recording into %s\n", perfout);
			sleep(1);
			exit(0);
		}
	}
	errx(1, "Unknown name: am I perfme-start or perfme-stop?");
}

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-18 22:03:31 +01:00
Rusty Russell
ee06c71efc gossipd: make routing far less chatty.
# 100,000 nodes
$ ./gossipd/test/run-bench-find_route 100000 1 > /tmp/out
=> 5357 msec

# 1M nodes:
$ /gossipd/test/run-bench-find_route 1000000 1 > /tmp/out
=> 55064 msec

# 1M nodes with CDEBUGFLAGS='-std=gnu11 -g -fstack-protector -O3 -flto' and manual linking with '-flto':
$ /gossipd/test/run-bench-find_route 1000000 1 > /tmp/out
=> 33501 msec

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-18 22:03:31 +01:00
Rusty Russell
4a54884d4b gossipd: routing benchmark.
Initial run (100,000 nodes):

$ ./gossipd/test/run-bench-find_route 100000 1 > /tmp/out
=> 15646 msec

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-18 22:03:31 +01:00
Rusty Russell
745c7b223a gossipd/routing: make local functions static.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-18 22:03:31 +01:00
Christian Decker
a8a6d1d669 channel: Directly send announcements and updates to gossipd
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-17 02:44:20 +00:00
Rusty Russell
94711969f9 gossipd: hand out gossip_index to other daemons.
When gossipd sends a message, have a gossip_index.  When it gets back a
peer, the current gossip_index is included, so it can know exactly where
it's up to.

Most of this is mechanical plumbing through openingd, channeld and closingd,
even though openingd and closingd don't (currently) read gossip, so their
gossip_index will be unchanged.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-17 02:44:20 +00:00
Rusty Russell
6afc7dcc09 gossipd: don't increment broadcast_index until *after* message sent.
If the peer is moved from remote to local, this may be lost; it's more
secure to increment after we've sent the broadcast.
2017-12-17 02:44:20 +00:00
Rusty Russell
67aa95c194 gossipd: hand back peer, don't hand a new peer.
All peers come from gossipd, and maintain an fd to talk to it.  Sometimes
we hand the peer back, but to avoid a race, we always recreated it.

The race was that a daemon closed the gossip_fd, which made gossipd
forget the peer, then master handed the peer back to gossipd.  We stop
the race by never closing the gossipfd, but hand it back to gossipd
for closing.

Now gossipd has to accept two fds, but the handling of peers is far
clearer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-17 02:44:20 +00:00
Rusty Russell
8832925122 gossipd: don't hand length to route code, it's implied.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-17 02:44:20 +00:00
Rusty Russell
36746bf65a Makefile: make gossipd objects depend correctly on its own headers.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-17 02:44:20 +00:00
Rusty Russell
bcfbc24308 gossipd: split peer structure to clearly separate local and remote fields.
We should also go through and use consistent nomenclature on functions which
are used with a local peer ("lpeer_xxx"?) and those with a remote peer
("rpeer_xxx"?) but this is minimal.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-17 02:44:20 +00:00
Christian Decker
a0724f45af routing: Do not set an empty channel_announcement if none is given
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-17 02:44:20 +00:00
Christian Decker
4a51de44c0 gossip: Forward when we don't have a valid node_announcement yet
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-17 02:44:20 +00:00
Christian Decker
580a2612fd routing: Add local and sigfail to trace when receiving cannounce
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-17 02:44:20 +00:00
Christian Decker
ba45ccffd8 routing: Return boolean from handle_channel_announcement
This will later be used to determine whether or not we should announce
ourselves as a node.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-17 02:44:20 +00:00
Christian Decker
b5f737e563 gossip: Passing alias, color and wireaddrs through to gossipd
First step towards `gossipd` managing the `node_announcement`.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-17 02:44:20 +00:00
Christian Decker
19c030ea1f routing: Make routing_state aware of its own ID
This is used to identify our own announcements.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-17 02:44:20 +00:00
Christian Decker
8d6281175d gossip: Remove HSM_FD from handshake 2017-12-17 02:44:20 +00:00
Christian Decker
61852b4603 cleanup: Use check_act* in handshake and remove unused static inline
This was tripping up `clang`.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-12 02:31:03 +00:00
practicalswift
61c47c09d0 Fix typos 2017-12-08 13:07:20 +01:00
Christian Decker
2d9cd4759e hsm: Make the master simply a client with special capabilities 2017-12-03 17:06:13 +01:00
Rusty Russell
40516941af channeld: fill in channel_update field in errors by asking gossipd.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-02 12:54:12 +01:00
Rusty Russell
933c1794a1 gossipd: gossip_get_update / gossip_get_update_reply from channeld.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-02 12:54:12 +01:00
Rusty Russell
af7e6158af Makefile: clean needs to do more, distclean should remove everything.
I checked this with git status --ignored after a full build and 'make distclean'.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-24 13:29:03 +01:00
Rusty Russell
40315bfb91 test: fix dependencies.
Test objects must be added to $(ALL_OBJS) so they correctly depend on
CCAN headers etc.

Also, each test in a subdir must depend on headers and src in the parent
directory, as it will often #include them directly.

Reported-by: Christian Decker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-22 19:40:46 +01:00
Rusty Russell
a9002eac52 Update to latest BOLT.
And nail "make check-source" to that specific version (which is a commit id,
not a branch name, so needs a different syntax for git).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-01 01:09:23 +00:00
Rusty Russell
68dae5648d gossipd: route correctly using final CLTV value.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-31 14:36:50 +01:00
Rusty Russell
0c7ca9ab7c gossipd: call to return all connected peers.
And we report these through the getpeers JSON RPC again (carefully: in
our reconnect tests we can get duplicates which this patch now filters
out).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-26 21:01:09 +00:00
Rusty Russell
78cd25d620 ipaddr: rename to wireaddr.
In future it will have TOR support, so the name will be awkward.

We collect the to/fromwire functions in common/wireaddr.c, and the
parsing functions in lightningd/netaddress.c.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-26 21:01:09 +00:00
Rusty Russell
bd1cac34ce netaddr: remove.
We use ipaddr everywhere now, so we can remove this.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-26 21:01:09 +00:00
Rusty Russell
dfd60a2047 gossipd: tell the master the peer's address.
This will let us remove peer->netaddr.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-26 21:01:09 +00:00
Rusty Russell
33bfc2326a gossipd: pass addr of peer though handshake.
We need to derive this from the fd when they connect in, but we already
know it if we're connecting out.

We want this so we can tell (in next few patches) master the peer's address.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-26 21:01:09 +00:00
Rusty Russell
3f84ca1052 gossipd: really fix peer handoff.
954a3990fa had two errors:
1) We created the handoff message *before* we sent the final packet, meaning
   that the cryptostate was out-of-sync.
2) We called io_wait() on the output side of a duplex connection: it has
   to be io_wait_out().

This time, stress testing for 2 hours revealed no more problems.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-26 13:03:51 +02:00
Rusty Russell
8d9818ff9c gossipd: receive global/local features the right way around
Fixes: #323
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-26 12:49:56 +02:00
Rusty Russell
954a3990fa gossipd: don't send a peer to master with half-written or half-read packet.
In this case, it was a gossip message half-sent, when we asked the peer
to be released.  Fix the problem in general by making send_peer_with_fds()
wait until after the next packet.

test_routing_gossip/lightning-4/log:
	b'lightning_openingd(8738): TRACE: First per_commit_point = 02e2ff759ed70c71f154695eade1983664a72546ebc552861f844bff5ea5b933bf'
	b'lightning_openingd(8738): TRACE: Failed hdr decrypt with rn=11'
	b'lightning_openingd(8738): STATUS_FAIL_PEER_IO: Reading accept_channel: Success'

test_routing_gossip/lightning-5/log:

	b'lightning_gossipd(8461): UPDATE WIRE_GOSSIP_PEER_NONGOSSIP'
	b'lightning_gossipd(8461): UPDATE WIRE_GOSSIP_PEER_NONGOSSIP'
	b'lightningd(8308): Failed to get netaddr for outgoing: Transport endpoint is not connected'

The problem occurs here on release, but could be on any place where we hand
a peer over when using ccan/io.  Note the other case (channel.c).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-25 18:34:35 +02:00
Rusty Russell
7e022b522c gossipd: don't try to handle padding inside fromwire_ipaddr.
It makes it impossible to embed an ipaddr in another structure, since we
always try to skip over any zeroes, which may swallow a following field.

Do the skip specially for the case where we're parsing routing messages:
we never use padding for our own internal messages anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-24 16:12:22 +02:00
Rusty Russell
74e684cc0d is_all_channels: rename to channel_id_is_all
Suggested-by: Christian Decker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-22 16:24:10 +02:00
Rusty Russell
474887512d gossipd: rewrite to do the handshake internally.
Now the flow is much simpler from a lightningd POV:

1. If we want to connect to a peer, just send gossipd `gossipctl_reach_peer`.
2. Every new peer, gossipd hands up to lightningd, with global/local features
   and the peer fd and a gossip fd using `gossip_peer_connected`
3. If lightningd doesn't want it, it just hands the peerfd and global/local
   features back to gossipd using `gossipctl_handle_peer`
4. If a peer sends a non-gossip msg (eg `open_channel`) the gossipd sends
   it up using `gossip_peer_nongossip`.
5. If lightningd wants to fund a channel, it simply calls `release_channel`.

Notes:
* There's no more "unique_id": we use the peer id.
* For the moment, we don't ask gossipd when we're told to list peers, so
  connected peers without a channel don't appear in the JSON getpeers API.
* We add a `gossipctl_peer_addrhint` for the moment, so you can connect to
  a specific ip/port, but using other sources is a TODO.
* We now (correctly) only give up on reaching a peer after we exchange init
  messages, which changes the test_disconnect case.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-20 18:31:32 +02:00
Rusty Russell
a88ac22711 gossipd: include ccan/io version of handshake code, with tests.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-20 18:31:32 +02:00
Rusty Russell
4fa36c585d gossipd: receive hsm fd from master.
We'll need this soon.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-20 18:31:32 +02:00
Rusty Russell
f172be71dc gossipd: fail peer for the master daemon.
This fixes the only case where the master currently has to write directly
to the peer: re-sending an error.  We make gossipd do it, by adding
a new gossipctl_fail_peer message.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-20 18:31:32 +02:00
Rusty Russell
2394c9a2e7 crypto_state: move to its own file.
In particular, the main daemon needs to pass it about (marshal/unmarshal)
but it won't need to actually use it after the next patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-20 18:31:32 +02:00
Rusty Russell
399b5f61bc gossipd: rename fail_peer to drop_peer.
We don't actually send it a failure message, we just close it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-20 18:31:32 +02:00
Rusty Russell
32631b4278 generate-wire.py: add --bolt arg, use size->type hacks only when that's specified.
For our own internal comms CSVs, we should always name explicit types.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 14:40:34 +02:00
Rusty Russell
c850e3c0a4 gossipd: add debugging into io_loop.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 10:20:08 +09:30
Rusty Russell
72b215f6fe Make all internal message numbers unique.
We were sending a channeld message to onchaind, which was v. confusing
due to overlap.  We make all the numbers distinct, which means we can
also add an assert() that it's valid for that daemon, which catches
such errors immediately.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell
ef28b6112c status: use common status codes for all the failures.
This change is really to allow us to have a --dev-fail-on-subdaemon-fail option
so we can handle failures from subdaemons generically.

It also neatens handling so we can have an explicit callback for "peer
did something wrong" (which matters if we want to close the channel in
that case).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-12 23:00:53 +02:00
Rusty Russell
ee9e300da0 gossip: fix address descriptor handling.
1. The code to skip over padding didn't take into account max.
2. It also didn't use symbolic names.
3. We are not supposed to fail on unknown addresses, just stop parsing.
4. We don't use the read_ip/write_ip code, so get rid of it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-03 02:01:54 +02:00
Rusty Russell
6933db04b5 gossipd/routing: remove/static unused functions.
I missed these when I removed the legacy daemon.  We also remove the
min_blocks field which was always 0.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-03 02:01:54 +02:00
Rusty Russell
4be7e94e0b gossip: make rpc responses correct.
It's not fee_per_kw, it's fee-per-millionth and a base in msatoshi.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-03 02:01:54 +02:00
Rusty Russell
28553308e2 gossip: make channel_announce-set fields unset until channel announce.
Use a negative timestamp as the flag for this, making the test simple.
This allows valgrind to detect that we're accessing them prematurely,
including across the wire on gossip_getchannels_entry.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-03 02:01:54 +02:00
Rusty Russell
9c35603275 gossipd/routing: free everything at end of tests.
valgrind complains, but using a destructor on the node map is a good
idea anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-03 02:01:54 +02:00
Rusty Russell
ffaab09043 gossipd/test: fix flaky test.
The test is could actually go each way, since for 1000000 the fee is
the same either way.

Increase to 300000, and add an extra test when the alternate path
is disabled.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-03 02:01:54 +02:00
Rusty Russell
c8aa50a382 gossipd: fix routing issue.
I had a routing problem, and wrote a simple unit test which passed.  So
I wrote one which copied the failure case (and importantly, had a non-1
fee factor), which triggerd it.

In that real example, we underflowed which resulted in us not finding
a route.  Simply don't consider routes which are infinite.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-03 02:01:54 +02:00
Rusty Russell
66a07461d3 gossipd: don't ignore first update if timestamp == 0
Since we initialize last_timestamp to 0, we ignore any initial update
with this timestamp.  Don't compare it if we don't already have an
update, and don't initialize it, so valgrind can tell us if we use
it accidentally.

	b'lightning_gossipd(3368): TRACE: Received channel_update for channel 6892:2:1(0)'
	b'lightning_gossipd(3368): TRACE: Ignoring outdated update.'
	b'lightning_gossipd(3368): TRACE: Received channel_update for channel 6893:2:1(1)'
	b'lightning_gossipd(3368): TRACE: Channel 6893:2:1(1) was updated.'

The same logic applies to node_updates, so we do the same there.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-03 02:01:54 +02:00
Rusty Russell
bbed5e3411 Rename subdaemons, move them into top level.
We leave the *build* results in lightningd/ for ease of in-place testing though.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00