Commit Graph

311 Commits

Author SHA1 Message Date
Rusty Russell
eee84b198b listnodes: display global_features.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-25 02:13:52 +00:00
Mark Beckwith
b876c601a6 Modern param style for chaintopology.c, ...
connect_control.c, dev_ping.c, gossip_control.c, invoice.c.

This converts about 50% of all calls of `json_get_params` to `param`.

After trying (and failing) to squash and rebase #1682 I just made a new branch
from a patch file and closed #1682.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-07-17 14:33:14 +02:00
Rusty Russell
28c3706f87 hsmd: fix missing status messages.
I crashed the HSMD, and it gave no output at all.  That's because we
were only reading the status fd when we were waiting for a reply.

Fix this by using a separate request fd and status fd, which also means
that hsm_sync_read() is no longer required.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-17 12:32:00 +02:00
Rusty Russell
f9e5dc7ee8 hsmd: take dbid for client.
We need this later, to generate its seed.  When we switch to lnd's key system,
we'll only need this, and not peerid.

Note also that the peerid is not just for messages any more, too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-17 12:32:00 +02:00
Christian Decker
ceef61dbbd gossip: Pass use_dns option down to gossipd
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-21 11:21:16 +02:00
Christian Decker
6298ce3b03 gossip: Don't ask bitcoind for outpoints we should know
Compares the `blocknum` in the `short_channel_id` with the range of blocks we
store in the database and abort if we should have known about it. Avoids
bombarding `bitcoind` with requests for channels that have already been spent or
were invalid in the first place.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-06 03:30:02 +00:00
Rusty Russell
1bb7713274 gossipd: minor cleanups.
Suggested-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell
9e51e196c1 gossipd: dev-set-max-scids-encode-size to artificially force "full" replies.
We cap each reply at a single one, which forces the code into our
recursion logic.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell
118f099dd8 gossip: dev-query-channel-range to test query_channel_range.
We keep a crappy bitmap, and finish when their replies cover
everything we asked.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell
c34b49c356 gossipd: add dev-send-timestamp-filter command for testing timestamp filtering.
Since we currently only (ab)use it to send everything, we need a way to
generate boutique queries for testing.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell
c633cbe2ee tests: add dev-query-scids
And write the test for it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell
4d8b29089b gossipd: wire up infrastructure to generate query_short_channel_ids msg.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Christian Decker
8e278044e3 gossip: Disable channels when we lose the connection to the peer
We're telling gossipd about disconnections anyway, so let's just use that signal
to disable both sides of the channel.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-31 02:30:27 +00:00
Christian Decker
9982e24a1c gossip: Add local_channel_close message to disable channels upon close
This was failing some of our integration tests, i.e., the ones closing a channel
and not waiting for sigexchange. The remote node would often not be quick enough
to send us its disabling channel_update, and hence we'd still remember the
incoming direction. That could then be sent out as part of an invoice, and fail
subsequently. So just set both directions to be disabled and let the onchain
spend clean up once it happens.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-31 02:30:27 +00:00
Mark Beckwith
7f437715d5 Added error code parameter to command_fail
Until now, `command_fail()` reported an error code of -1 for all uses.
This PR adds an `int code` parameter to `command_fail()`, requiring the
caller to explicitly include the error code.

This is part of #1464.

The majority of the calls are used during parameter validation and
their error code is now JSONRPC2_INVALID_PARAMS.

The rest of the calls report an error code of LIGHTNINGD, which I defined to
-1 in `jsonrpc_errors.h`.  The intention here is that as we improve our error
reporting, all occurenaces of LIGHTNINGD will go away and we can eventually
remove it.

I also converted calls to `command_fail_detailed()` that took a `NULL` `data`
parameter to use the new `command_fail()`.

The only difference from an end user perspecive is that bad input errors that
used to be -1 will now be -32602 (JSONRPC2_INVALID_PARAMS).
2018-05-26 12:17:36 +02:00
Rusty Russell
fca5a9ef30 channeld: tell gossipd to generate channel_updates.
This resolves the problem where both channeld and gossipd can generate
updates, and they can have the same timestamp.  gossipd is always able
to generate them, so can ensure timestamp moves forward.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-21 09:17:57 -07:00
Rusty Russell
cca791d1cb routing: clean up channel public/active states.
1. If we have a channel_announcement, the channel is public, otherwise
   it's not.  Not all channels are public, as they can be local: those
   have a NULL channel_announcement.

2. If we don't have a channel_update, we know nothing about that half
   of the channel, and no other fields are valid.

3. We can tell if a half channel is disabled by the flags field directly.

Note that we never send halfchannels without an update over
gossip_getchannels_reply so that marshalling/unmarshalling can be
vastly simplified.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 21:35:53 +02:00
Rusty Russell
89c76a5a78 Move always-use-proxy auto-override to master daemon.
This means it will effect connect commands too (though it's too
late to stop DNS lookups caused by commandline options).

We also warn that this is one case where we allow forcing through Tor
without a proxy set: it just means all connections will fail.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell
1106c40217 tor: add new 'autotor:' address option.
This takes the Tor service address in the same option, rather than using
a separate one.  Gossipd now digests this like any other type.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell
a8c0bca6a8 gossipd: take over negotiation of autogenerated Tor addresses.
For the moment, this is a straight handing of current parameters through
from master to the gossip daemon.  Next we'll change that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell
c3ccc14f19 Tor: remove --tor prefix from SOCKS5 options.
It's usually for Tor, but we can use a socks5 proxy without it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell
11db7ca9e6 options: use NULL for unset Tor settings.
Rename tor_proxyaddrs and tor_serviceaddrs to tor_proxyaddr and tor_serviceaddr:
the 's' at the end suggests that there can be more than one.

Make them NULL or non-NULL, rather than using all-zero if unset.

Hand them the same way to gossipd; it's a bit of a hack since we don't
have optional fields, so we use a counter which is always 0 or 1.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Saibato
877f63e99e Initial TOR v2/v3 support.
This is a rebased and combined patch for Tor support.  It is extensively
reworked in the following patches, but the basis remains Saibato's work,
so it seemed fairest to begin with this.

Minor changes:
1. Use --announce-addr instead of --tor-external.
2. I also reverted some whitespace and unrelated changes from the patch.
3. Removed unnecessary ';' after } in functions.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell
d40d22b68e gossipd: don't try to connect to non-routable addresses.
Someone could try to announce an internal address, and we might probe
it.

This breaks tests, so we add '--dev-allow-localhost' for our tests, so
we don't eliminate that one.  Of course, now we need to skip some more
tests in non-developer mode.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell
af065417e1 gossipd: handle wildcard addresses correctly.
If we're given a wildcard address, we can't announce it like that: we need
to try to turn it into a real address (using guess_address).  Then we
use that address.  As a side-effect of this cleanup, we only announce
*any* '--addr' if it's routable.

This fix means that our tests have to force '--announce-addr' because
otherwise localhost isn't routable.

This means that gossipd really controls the addresses now, and breaks
them into two arrays: what we bind to, and what we announce.  That is
now what we return to the master for json_getinfo(), which prints them
as 'bindings' and 'addresses' respectively.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell
52917ff6c9 More flexible address wildcards, only add wildcard if nothing else.
1. Add special option where an empty host means 'wildcard for IPv4 and/or IPv6'
   which means ':1234' can be used to set only the portnum.
2. Only add this protocol wildcard if --autolisten=1 (default)
   and no other addresses specified.
3. Pass it down to gossipd, so it can handle errors correctly: in most cases,
   it's fatal not to be able to bind to a port, but for this case, it's OK
   if we can only bind to one of IPv4/v6 (fatal iff neither).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell
73cd009a4c gossipd/lightningd: use wireaddr_internal.
This replacement is a little menial, but it explicitly catches all
the places where we allow a local socket.  The actual implementation of
opening a AF_UNIX socket is almost hidden in the patch.

The detection of "valid address" is now more complex:

	p->addr.itype != ADDR_INTERNAL_WIREADDR || p->addr.u.wireaddr.type != ADDR_TYPE_PADDING

But most places we do this, we should audit: I'm pretty sure we can't
get an invalid address any more from gossipd (they may be in db, but
we should fix that too).

Closes: #1323
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell
e6c678e5df gossipd: take over address determination, from master.
It does all the other address handling, do this too.  It also proves useful
as we clean up wildcard address handling.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell
fe96fe10c7 Clean up network options.
It's become clear that our network options are insufficient, with the coming
addition of Tor and unix domain support.

Currently:

1. We always bind to local IPv4 and IPv6 sockets, unless --port=0, --offline,
   or any address is specified explicitly.  If they're routable, we announce.
2. --addr is used to announce, but not to control binding.

After this change:

1. --port is deprecated.
2. --addr controls what we bind to and announce.
3. --bind-addr/--announce-addr can be used to control one and not the other.
4. Unless --autolisten=0, we add local IPv4 & IPv6 port 9735 (and announce if they are routable).
5. --offline still overrides listening (though announcing is still the same).

This means we can bind to as many ports/interfaces as we want, and for
special effects we can announce different things (eg. we're sitting
behind a port forward or a proxy).

What remains to implement is semi-automatic binding: we should be able
to say '--addr=0.0.0.0:9999' and have the address resolve at bind
time, or even '--addr=0.0.0.0:0' and have the port autoresolve too
(you could determine what it was from 'lightning-cli getinfo'.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell
ed466a8523 lightningd: make explicit listen and reconnect flags.
We set no_reconnect with --offline, but that doesn't work if !DEVELOPER.
Make the flag positive, and non-DEVELOPER mode for gossipd.

We also don't override portnum with --offline, but have an explicit
'listen' flag.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell
f083a699e2 gossipd: separate init and activate.
This means gossipd is live and we can tell it things, but it won't
receive incoming connections.  The split also means that the main daemon
continues (eg. loading peers from db) while gossipd is loading from the store,
potentially speeding startup.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-30 12:01:36 +02:00
Rusty Russell
be1f33b265 gossipd: have master explicitly tell us when peer is disconnected.
Currently we intuit it from the fd being closed, but that may happen out
of order with when the master thinks it's dead.

So now if the gossip fd closes we just ignore it, and we'll get a
notification from the master when the peer is disconnected.

The notification is slightly ugly in that we have to disable it for
a channel when we manually hand the channel back to gossipd.

Note: as stands, this is racy with reconnects.  See the next patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-26 05:47:57 +00:00
Rusty Russell
ab9d9ef3b8 gossipd: drain fd instead of passing around gossip index.
(This was sitting in my gossip-enchancement patch queue, but it simplifies
this set too, so I moved it here).

In 94711969f we added an explicit gossip_index so when gossipd gets
peers back from other daemons, it knows what gossip it has sent (since
gossipd can send gossip after the other daemon is already complete).

This solution is insufficient for the more general case where gossipd
wants to send other messages reliably, so replace it with the other
solution: have gossipd drain the "gossip fd" which the daemon returns.

This turns out to be quite simple, and is probably how I should have
done it originally :(

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-26 05:47:57 +00:00
Rusty Russell
72c459dd6c gossipd: keep reaching struct only when we're actively connecting, and don't retry
1. Lifetime of 'struct reaching' now only while we're actively doing connect.
2. Always free after a single attempt: if it's an important peer, retry
   on a timer.
3. Have a single response message to master, rather than relying on
   peer_connected on success and other msgs on failure.
4. If we are actively connecting and we get another command for the same
   id, just increment the counter

The result is much simpler in the master daemon, and much nicer for
reconnection: if they say to connect they get an immediate response,
rather than waiting for 10 retries.  Even if it's an important peer,
it fires off another reconnect attempt, unless it's actively
connecting now.

This removes exponential backoff: that's restored in next patch.  It
also doesn't handle multiple addresses for a single peer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-26 05:47:57 +00:00
Rusty Russell
8c2c1fe1c2 openingd: tell gossipd that the peer is important once funding tx in place.
And on channel_fail_permanent and closing (the two places we drop to
chain), we tell gossipd it's no longer important.

Fixes: #1316
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-26 05:47:57 +00:00
Rusty Russell
c9fa9817f6 gossipd: explicitly track which peers are important.
These don't have a maximum number of reconnect attempts, and ensure
that we try to reconnect when the peer dies.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-26 05:47:57 +00:00
Rusty Russell
5551c161ca gossipd: finish startup before master prints that it's ready.
We're about to remove automatic retrying of connect, and that uncovered
that we actually print out our "Server started" message before we create
the listening socket.

Move the init higher (outside the db transaction) and make it a
request/response, the loop until it's done.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-23 20:18:15 +00:00
Rusty Russell
b0c2e3cd5c gossipd: use a separate CSV file for the gossip_store types.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-11 15:58:18 +02:00
Christian Decker
63f22d70b5 gossip: Store channel deletions so we don't re-add them on restart
If we only remember the actions that added channels then we'd restore them when
re-reading the gossip_store, so put a tombstone in there to remember to delete
it. These will be cleared upon re-writing the store since the announcements wont
be written anymore.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-30 16:35:00 +02:00
Christian Decker
633f893ec4 gossip: Add function to notify gossipd about an outpoint spend
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-30 16:35:00 +02:00
Christian Decker
5571f2143e gossip: Added message to notify gossipd of outpoint spends
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-30 16:35:00 +02:00
Rusty Russell
4139f06780 json_getnodes: escape the alias when we print it out.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-26 00:20:53 +00:00
Christian Decker
0a5ea76d77 gossip: Add message types to store gossip msgs and associate data
Since we may want to extend the on-disk format by adding custom information we
may as well just go the extra mile and reuse the serialization primitives we
already have.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-25 23:56:59 +00:00
ZmnSCPxj
9efe123a0d lightningd/json: Move json helpers specific for lightningd to new module. 2018-03-19 00:27:55 +00:00
practicalswift
b206aed371 Remove redundant cast to same type 2018-03-16 01:17:40 +00:00
Rusty Russell
0a6e3d1e13 utils: remove tal_tmpctx altogether, use global.
In particular, we now only free tmpctx at the end of main().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-16 00:16:10 +00:00
Rusty Russell
46cc7c281e features: more general accessor functions.
As we add more features, the current code is insufficient.

1. Keep an array of single feature bits, for easy switching on and off.
2. Create feature_offered() which checks for both compulsory and optional
   variants.
3. Invert requires_unsupported_features() and unsupported_features()
   which tend to be double-negative, all_supported_features() and
   features_supported().
4. Move single feature definition from wire/peer_wire.h to common/features.h.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-14 02:19:37 +00:00
Christian Decker
74a444eb7a jsonrpc: Return the channel capacity for listchannels
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-12 22:34:51 +00:00
Christian Decker
5690783148 gossip: Pass output value to gossipd
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-12 22:34:51 +00:00
Christian Decker
a8e553098a gossip: Pass output value to gossipd
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-12 22:34:51 +00:00
Christian Decker
cb4ba9bf4f gossip: Use the utxoset to shortcircuit the txout lookup
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-12 22:34:51 +00:00
Isidoro Ghezzi
f24a7b9356 help: listnodes and listchannels,have optional arg 2018-03-11 14:27:09 +01:00
John Barboza
0a1351fd51 gossipd: command to disconnect from a gossiping peer
lightning-cli disconnect <peer id>
2018-03-07 16:14:01 +01:00
Rusty Russell
042d5d13f5 short_channel_id: don't use bitfields.
I leave all the now-unnecessary accessors in place to avoid churn, but
the use of bitfields has been more pain than help.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-01 23:33:56 +01:00
Rusty Russell
cf3f19524e gossip: formalize passing of siphash_seed.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-26 06:35:02 +00:00
ZmnSCPxj
d23650d2ed Explain fuzz initial values. 2018-02-26 02:36:27 +00:00
ZmnSCPxj
2db6ff9186 gossip_control: Change defult fuzz of getroute to 75%. 2018-02-26 02:36:27 +00:00
ZmnSCPxj
6767434ea9 routing: Use siphash24 for route randomization
Primary idea by @rustyrussell
2018-02-26 02:36:27 +00:00
ZmnSCPxj
216c52940d gossipd: Add fuzz and seed to getroute request. 2018-02-26 02:36:27 +00:00
ZmnSCPxj
2b782ee436 gossip_control: Add fuzz and seed parameters.
In preparation for improved route randomization.
2018-02-26 02:36:27 +00:00
practicalswift
91a9c2923f Mark intentionally unused parameters as such (with "UNUSED") 2018-02-22 01:09:12 +00:00
practicalswift
3dbace3421 Remove redundant casts to same type 2018-02-21 13:07:40 +01:00
Rusty Russell
e92b710406 tools/generate-wire.py: remove length argument from fromwire_ routines.
We always hand in "NULL" (which means use tal_len on the msg), except
for two places which do that manually for no good reason.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-20 22:36:21 +01:00
Rusty Russell
a83ff83328 lightningd/connect_control: routines to control connecting (move from peer_control.c)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-20 22:36:21 +01:00
Christian Decker
cb97dfec4e gossip: Fix up some comments on PR #955
Just some minor things that were suggested during review.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-02-11 11:40:59 +01:00
root
73022d91c6 Fix indentation 2018-02-10 21:30:46 +01:00
root
8b0ed27c43 Add optional from id parameter to getroute call. 2018-02-10 21:30:46 +01:00
Rusty Russell
526d3a232e tools/generate_wire.py: generate varlen arrays properly.
These are now logically arrays of pointers.  This is much more natural,
and gets rid of the horrible utxo array converters.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-08 19:07:12 +01:00
ZmnSCPxj
0d0ef2c733 gossipd: Implement gossip_mark_channel_unroutable.
Temporarily marks a channel as unroutable.
2018-02-08 00:51:30 +00:00
Rusty Russell
91a22dc496 jsonprc: make json_get_params() fail the command, for better error reporting.
We move it into jsonrpc where it belongs, and make it fail the command.
This means it can tell us exactly what was wrong.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-02 00:05:00 +01:00
ZmnSCPxj
b255d82ae2 gossipd: Implement gossip_routing_failure for master to report routing failures. 2018-02-01 00:46:06 +00:00
Christian Decker
61e982c6b7 gossip: Implement connection failed handler in lightningd
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-29 06:01:56 +00:00
Christian Decker
f5147f27c9 gossip: Added message for unsuccessful connection attempts
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-29 06:01:56 +00:00
Christian Decker
c6ae4b521f gossip: Add message to enable and disable a channel
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-26 01:09:44 +00:00
practicalswift
5631054152 Start all command descriptions with a verb ("Show", "Set", etc.). Remove unused help texts. 2018-01-23 13:32:44 +01:00
Rusty Russell
06c4f6ddca JSONRPC: add optional short_channel_id argument to listchannels
In order to just list one (though it may return two entries, one for each
channel direction!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-19 22:23:45 +00:00
Rusty Russell
7fdb30cbda JSONRPC: rename getchannels to listchannels
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-19 22:23:45 +00:00
Rusty Russell
e1e7f289fb JSONRPC listnodes: return timestamp, alias and color.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-19 22:23:45 +00:00
Rusty Russell
ced572b973 JSONRPC: getnodes: rename to listnodes.
Like listinvoice, and add optional 'id' parameter to ask about a
specific node.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-19 22:23:45 +00:00
Rusty Russell
4b663e596a lightningd: don't crash when txout lookup fails.
Gossipd already correctly handles the "empty output means lookup failed" case.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-15 19:29:01 +00:00
Christian Decker
7fc8e3a4e2 gossip: Pass --channel-update-interval to gossipd
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-14 23:40:04 +00:00
ZmnSCPxj
83e76e3ac3 features: Move feature-handling code to a common/features.c source. 2018-01-13 11:29:42 +01:00
Rusty Russell
d4c8210a9e gossipd: don't hang if we try to connect to already-connected peer.
Closes: #287
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-11 23:47:47 +01:00
Rusty Russell
e22dd5ad71 lightningd: implement gossip_get_txout/gossip_get_txout_reply.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-11 23:13:23 +01:00
Rusty Russell
7e0bc88e2a gossip: fix default broadcast interval, move option.
This now makes a few more gossip tests time out without --dev-broadcast-interval

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-11 21:39:13 +01:00
Rusty Russell
a8de8a3140 json_getchannels: add public flag.
Fixes: #509
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-08 18:09:31 +01:00
Rusty Russell
7975a1cbfc json_getchannels: short_id -> short_channel_id, use standard format.
We include flags separately, so name this field as per spec.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-08 18:09:31 +01:00
Rusty Russell
8c665835fa jsonrpc: make explicit call to indicate cmd is still pending.
jsonrpc handlers usually directly call command_success or
command_fail; not doing that implies they're waiting for something
async.

Put an explicit call (currently a noop) there, and add debugging
checks to make sure it's used.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 12:43:10 +01:00
Christian Decker
2988e290cf gossip: Added internal gossip message to add a local channel
Couldn't find a good place to put these messages, we probably want to
do the same capability based request routing that we did for the HSM,
but for now this just defines the message in the master messages file.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-20 06:14:53 +00:00
Christian Decker
a8a6d1d669 channel: Directly send announcements and updates to gossipd
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-17 02:44:20 +00:00
Rusty Russell
94711969f9 gossipd: hand out gossip_index to other daemons.
When gossipd sends a message, have a gossip_index.  When it gets back a
peer, the current gossip_index is included, so it can know exactly where
it's up to.

Most of this is mechanical plumbing through openingd, channeld and closingd,
even though openingd and closingd don't (currently) read gossip, so their
gossip_index will be unchanged.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-17 02:44:20 +00:00
Rusty Russell
67aa95c194 gossipd: hand back peer, don't hand a new peer.
All peers come from gossipd, and maintain an fd to talk to it.  Sometimes
we hand the peer back, but to avoid a race, we always recreated it.

The race was that a daemon closed the gossip_fd, which made gossipd
forget the peer, then master handed the peer back to gossipd.  We stop
the race by never closing the gossipfd, but hand it back to gossipd
for closing.

Now gossipd has to accept two fds, but the handling of peers is far
clearer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-17 02:44:20 +00:00
Christian Decker
b5f737e563 gossip: Passing alias, color and wireaddrs through to gossipd
First step towards `gossipd` managing the `node_announcement`.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-17 02:44:20 +00:00
Christian Decker
9ba99d2b2d hsm: Cleanup after merging control and client libraries
Change all calls to use the correct serialization and deserialization
functions, include the correct headers and remove the control
messages.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-03 17:06:13 +01:00
Christian Decker
6ce264d3a2 hsm: Remove special case of hsmfd_ecdh, replaced with client and cap
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-03 17:06:13 +01:00
Rusty Russell
933c1794a1 gossipd: gossip_get_update / gossip_get_update_reply from channeld.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-02 12:54:12 +01:00
Rusty Russell
68dae5648d gossipd: route correctly using final CLTV value.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-31 14:36:50 +01:00
Rusty Russell
0c7ca9ab7c gossipd: call to return all connected peers.
And we report these through the getpeers JSON RPC again (carefully: in
our reconnect tests we can get duplicates which this patch now filters
out).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-26 21:01:09 +00:00
Rusty Russell
78cd25d620 ipaddr: rename to wireaddr.
In future it will have TOR support, so the name will be awkward.

We collect the to/fromwire functions in common/wireaddr.c, and the
parsing functions in lightningd/netaddress.c.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-26 21:01:09 +00:00
Rusty Russell
dfd60a2047 gossipd: tell the master the peer's address.
This will let us remove peer->netaddr.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-26 21:01:09 +00:00
Rusty Russell
ffaa15c7da hsm: remove unique_id.
It was only for error messages, so replace it with pubkey.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-24 16:12:22 +02:00
Rusty Russell
0b953b86fe subd: automatically detect if callback frees subd.
This involves a tricky callback internally, but far less error-prone.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-20 18:31:32 +02:00
Rusty Russell
5a256c724a subd: simplify and cleanup lifetime handling.
There are now only two kinds of subdaemons: global ones (hsmd, gossipd) and
per-peer ones.  We can handle many callbacks internally now.

We can have a handler to set a new peer owner, and automatically do
the cleanup of the old one if necessary, since we now know which ones
are per-peer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-20 18:31:32 +02:00
Rusty Russell
474887512d gossipd: rewrite to do the handshake internally.
Now the flow is much simpler from a lightningd POV:

1. If we want to connect to a peer, just send gossipd `gossipctl_reach_peer`.
2. Every new peer, gossipd hands up to lightningd, with global/local features
   and the peer fd and a gossip fd using `gossip_peer_connected`
3. If lightningd doesn't want it, it just hands the peerfd and global/local
   features back to gossipd using `gossipctl_handle_peer`
4. If a peer sends a non-gossip msg (eg `open_channel`) the gossipd sends
   it up using `gossip_peer_nongossip`.
5. If lightningd wants to fund a channel, it simply calls `release_channel`.

Notes:
* There's no more "unique_id": we use the peer id.
* For the moment, we don't ask gossipd when we're told to list peers, so
  connected peers without a channel don't appear in the JSON getpeers API.
* We add a `gossipctl_peer_addrhint` for the moment, so you can connect to
  a specific ip/port, but using other sources is a TODO.
* We now (correctly) only give up on reaching a peer after we exchange init
  messages, which changes the test_disconnect case.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-20 18:31:32 +02:00
Rusty Russell
79ebb9dfd0 json: helper to parse pubkeys.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-20 18:31:32 +02:00
Rusty Russell
4fa36c585d gossipd: receive hsm fd from master.
We'll need this soon.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-20 18:31:32 +02:00
Rusty Russell
f172be71dc gossipd: fail peer for the master daemon.
This fixes the only case where the master currently has to write directly
to the peer: re-sending an error.  We make gossipd do it, by adding
a new gossipctl_fail_peer message.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-20 18:31:32 +02:00
Rusty Russell
399b5f61bc gossipd: rename fail_peer to drop_peer.
We don't actually send it a failure message, we just close it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-20 18:31:32 +02:00
Rusty Russell
b7bb0be944 subd: remove context arg, as we're always owned by lightningd.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 15:37:43 +02:00
Rusty Russell
ef28b6112c status: use common status codes for all the failures.
This change is really to allow us to have a --dev-fail-on-subdaemon-fail option
so we can handle failures from subdaemons generically.

It also neatens handling so we can have an explicit callback for "peer
did something wrong" (which matters if we want to close the channel in
that case).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-12 23:00:53 +02:00
Rusty Russell
4be7e94e0b gossip: make rpc responses correct.
It's not fee_per_kw, it's fee-per-millionth and a base in msatoshi.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-03 02:01:54 +02:00
Rusty Russell
28553308e2 gossip: make channel_announce-set fields unset until channel announce.
Use a negative timestamp as the flag for this, making the test simple.
This allows valgrind to detect that we're accessing them prematurely,
including across the wire on gossip_getchannels_entry.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-03 02:01:54 +02:00
Rusty Russell
bbed5e3411 Rename subdaemons, move them into top level.
We leave the *build* results in lightningd/ for ease of in-place testing though.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00
Rusty Russell
153c622157 lightningd: remove lightningd_state.
Some fields were redundant, some are simply moved into 'struct lightningd'.
All routines updated to hand 'struct lightningd *ld' now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00
Rusty Russell
a37c165cb9 common: move some files out of lightningd/
Basically all files shared by different daemons.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00
Rusty Russell
8375857116 common: absorb remaining files from daemon/
Also, we split the more sophisticated json_add helpers to avoid pulling in
everything into lightning-cli, and unify the routines to print struct
short_channel_id (it's ':',  not '/' too).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00
Rusty Russell
85ff95e829 common: new directory for any shared objects.
To avoid everything pulling in HTLCs stuff to the opening daemon, we
split the channel and commit_tx routines into initial_channel and
initial_commit_tx (no HTLC support) and move full HTLC supporting versions
into channeld.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00
Rusty Russell
ffddb91e3e gossipd: hand in chain_hash.
This will be needed for gossip messages.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-22 14:56:02 +02:00
Rusty Russell
ee00c2c508 lightningd: remove peer->gossip_client_fd
We don't need to keep this around any more: by handing it to
subdaemons we ensure we'll close it if the peer disconnects, and we
also add code to get a new one on reconnection.

Because getting a gossip_fd is async, we re-check the peer state after
it gets back.  This is kind of annoying: perhaps if we were to hand
the reconnected peer through gossipd (with a flag to immediately
return it) we could get the gossip fd that way and unify the paths?

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-06-27 10:25:53 +09:30
Rusty Russell
4185153d81 gossipd: interface to get a client gossip_fd for a reconnect.
At the moment, master simply keeps the gossip fd open when peer
disconnects.  That's inefficient, and wrong anyway (it may want a
complete new sync, or may not, but we'll currently send all the
messages including stale ones).

This interface will be required for restart anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-06-27 10:25:53 +09:30
Rusty Russell
7213312f57 lightningd: remove fd from struct peer.
With no async calls left, we can just use a stack variable for the fd.

And we're now *always* in the hands of some daemon, unless we're
disconnected, so owner is only NULL in that case.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-06-27 10:25:53 +09:30
Rusty Russell
f2d4309add lightningd/subd: explicit failure reply support.
We had a terrible hack in gossip when a peer didn't exist.  Formalize
a pattern when code+200 is a failure (with no fds passed), and use it
here.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-06-27 10:25:53 +09:30
Rusty Russell
31ff5a49f4 lightningd/peer_control: remove cs from struct peer.
Now we're not async, we don't need to pack it into struct peer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-06-27 10:25:53 +09:30
Rusty Russell
d95adf7f33 subd: hand through fatal messages as well to callback.
This matters in one case: channeld receiving a bad message is a
permenant failure, whereas losing a connection is transient.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-06-23 09:29:42 +09:30
Rusty Russell
ffa5e1c52c peer_fail: differentiate transient and permanent failures.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-06-23 09:29:42 +09:30
Rusty Russell
f7a22b699e lightningd/peer_control: embed id, don't use pointer.
We always know it now we have struct connection, so we don't need a pointer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-06-07 09:19:04 +09:30
Rusty Russell
d1fcc434c8 subd: use array of fd pointers, not fds, and use take().
This lets us specify that we want to keep some fds.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-05-25 14:24:47 +09:30
Rusty Russell
3ab3281a4c lightningd/opening: rename functions and wire messages for clarity.
Each one is either funder or fundee now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-05-25 14:24:47 +09:30
Rusty Russell
1e36a19164 lightningd/peer_control: keep cryptostate.
Like the fd, it's only useful when the peer is not in a daemon, so we
free & NULL it when that happens.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-05-25 14:24:47 +09:30
Rusty Russell
ccad93edb3 gossipd: add fail_peer.
We use this if it reconnects via another fd.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-05-25 14:24:47 +09:30
Rusty Russell
662dfef436 lightningd/gossip: Move INIT message handling to handshake daemon.
We need to do this on every connection, whether reconnecting or not,
so it makes sense for the handshake daemon to handle it and return
the feature fields.

Longer term I'm considering having the handshake daemon handle the
listening and connecting, and simply hand the fds back once the peers
are ready.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-05-25 14:24:47 +09:30
Rusty Russell
f6495d3310 lightningd/peer_control: don't create peer struct until we've connected.
We currently create a peer struct, then complete handshake to find out
who it is.  This means we have a half-formed peer, and worse: if it's
a reconnect we get two peers the same.

Add an explicit 'struct connection' for the handshake phase, and
construct a 'struct peer' once that's done.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-05-25 14:24:47 +09:30
Rusty Russell
34e6e56471 lightningd: introduce peer_state enum.
The actual state names are place holders for now, really.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-05-25 14:24:47 +09:30
Rusty Russell
be9bb5f9cb lightningd: peer_fail helper to fail/reconnect peer.
This will eventually hook into restart logic.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-05-25 14:24:47 +09:30
Christian Decker
e972b208c7 routing: Passing back all addresses on getnodes 2017-05-10 12:37:44 +09:30
Christian Decker
daf8866eb5 gossip: Implement the basic node_announcement
Rather a big commit, but I couldn't figure out how to split it
nicely. It introduces a new message from the channel to the master
signaling that the channel has been announced, so that the master can
take care of announcing the node itself. A provisorial announcement is
created and passed to the HSM, which signs it and passes it back to
the master. Finally the master injects it into gossipd which will take
care of broadcasting it.
2017-05-10 12:37:44 +09:30
Christian Decker
25f1cba3cf routing: Ask gossipd to resolve channel_id and forward HTLCs
Since we now use the short_channel_id to identify the next hop we need
to resolve the channel_id to the pubkey of the next hop. This is done
by calling out to `gossipd` and stuffing the necessary information
into `htlc_end` and recovering it from there once we receive a reply.
2017-05-02 11:49:14 +02:00
Christian Decker
d87ca4121d routing: Returning channel_id to getroute requests
The new onion uses the `channel_id` instead of the `node_id` of the
next hop to identify where to forward the payment. So we return the
exact channel chosen by the routing algo, to avoid having to look it
up again later.
2017-05-02 11:47:52 +02:00
Christian Decker
b4beab6537 gossip: Make the broadcast interval configurable
Adds a new command line flag `--dev-broadcast-interval=<ms>` that
allows us to specify how often the staggered broadcast should
trigger. The value is passed down to `gossipd` via an init message.

This is mainly useful for integration tests, since we do not want to
wait forever for gossip to propagate.
2017-05-02 11:59:24 +09:30
Rusty Russell
6d55a17642 lightningd/dev_ping: expand to cover gossipd.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-04-25 22:00:28 +02:00
Rusty Russell
fed25cc540 lightningd/subd: add a context to requests.
If a peer dies, and then we get a reply, that can cause access after free.
The usual way to handle this is to make the request a child of the peer,
but in fact we still want to catch (and disard) it, so it's a little
more complex internally.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-04-01 23:59:46 +10:30
Rusty Russell
e36a65a189 lightningd/subd: msgcb return -1 to close channel.
They can't free it while we're using it, but they can return a value
to close it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-04-01 23:59:46 +10:30
Christian Decker
638594f3c6 jsonrpc: Implemented getchannels JSON-RPC call 2017-03-24 13:24:58 +10:30
Christian Decker
c8da420a9d routing: Fill in the getroute functionality
Copied the JSON-request parsing from `pay.c`, passing through to
`gossipd`, filling the reply with the `route_hop` serialization, and
serializing as JSON-RPC response.
2017-03-23 13:34:03 +10:30
Christian Decker
9273b615ab gossip: Added scheleton for getroute calls
This includes the request/reply passed from the main daemon to
`gossipd` and the basic handlers.
2017-03-23 13:34:03 +10:30
Rusty Russell
f511012e29 lightningd/gossip: don't hand client fd until release.
The gossip subdaemon previously passed the fd after init: this is
unnecessary for peers which simply want to gossip (and not establish
channels).

Now we hand the gossip fd back with the peer fd.  This adds another
error message for when we fail to create the gossip fds.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-03-20 07:50:53 +10:30
Rusty Russell
7a9df37ef3 lightningd/subd: support multiple fds sent at once in normal messages.
Rather than returning SUBD_NEED_FD, callback returns how many fds it needs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-03-20 07:50:53 +10:30
Rusty Russell
38bffc0f0c lightningd/subd: support multiple fds sent at once in request reply.
Instead of indicating where to place the fd, you say how many: the
fd array gets passed into the callback.

This is also clearer for the users.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-03-20 07:50:53 +10:30
Rusty Russell
de651bf8fb gossip: don't use hand-coded nested messages for getnodes array.
We can simply tell it 'nodes' is 'num_nodes*struct gossip_getnodes_entry'.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-03-16 14:35:26 +10:30
Christian Decker
d6918dc4ef gossip: Implemented getnodes JSON-RPC call 2017-03-15 21:32:55 +01:00
Christian Decker
3045e25624 gossip: Returning a gossip-client upon peer_ready
This fd is handed to the owner of the peer so that it can talk to the
gossip daemon.
2017-03-13 11:26:48 +01:00
Rusty Russell
da5c7e0a08 daemon/subdaemon: remove in favor of daemon/subd.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-03-11 07:19:41 +10:30
Rusty Russell
5dc8cb12be lightningd/gossip: convert to subd.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-03-11 07:19:40 +10:30
Rusty Russell
5a73380e64 lightningd/subdaemon: keep peer pointer for one-per-peer daemons.
This is really useful to map daemon back to peer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-03-07 11:39:03 +10:30
Rusty Russell
fd258fe495 lightningd: wire up opening daemon.
Now we can actually open a channel.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-02-24 16:22:56 +10:30
Rusty Russell
952722a8cf lightningd/gossip: Fix return from gossip.
We should check that the peer it says it's returning is under its control,
we need to take back the peer fd, and use the correct conversion routine
for the packet it sends us.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-02-24 16:22:56 +10:30
Rusty Russell
90737371d0 lightningd/cryptomsg: split raw crypto_state vs peer_crypto_state
Raw crypto_state is what we send across the wire: the peer one is for
use in async crypto io routines (peer_read_message/peer_write_message).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-02-24 16:22:56 +10:30
Rusty Russell
9fd40da38c generate-wire: don't hand unknown structures specially.
It's awkward to handle them differently.  But this change means we
need to expose them to the generated code.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-02-21 15:15:19 +10:30
Rusty Russell
83b156517f lightningd/lightningd: maintain "condition" for each peer.
This lets us return better messages to the connect json command on failure.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-10 15:38:33 +10:30
Rusty Russell
7aaffda779 lightningd/lightningd: finish connect command once gossip started.
This is after the INIT message is received, so we know there are no
incompatible features.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-10 15:38:33 +10:30
Rusty Russell
c536616bee lightningd/lightningd: wire up lightningd_gossip.
Now we hand peers off to the gossip daemon, to do the INIT handshake and
re-transmit/receive gossip.  They may stay there forever if neither we nor
them wants to open a channel.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-01-10 15:38:33 +10:30