Commit Graph

3775 Commits

Author SHA1 Message Date
Rusty Russell
1106c40217 tor: add new 'autotor:' address option.
This takes the Tor service address in the same option, rather than using
a separate one.  Gossipd now digests this like any other type.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell
a8c0bca6a8 gossipd: take over negotiation of autogenerated Tor addresses.
For the moment, this is a straight handing of current parameters through
from master to the gossip daemon.  Next we'll change that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell
e93682e3bf status: make status_io a more generic mechanism.
Currently it's always for messages to peer: make that status_peer_io and
add a new status_io for other IO.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell
5a0bc83b20 Tor: don't do seed queries if we're supposed to always use proxy.
Risks leakage.  We could do lookup via the proxy, but that's a TODO.

There's only one occurance of getaddrinfo (and no gethostbyname), so
we add a flag to the callers.

Note: the use of --always-use-proxy suppresses *all* DNS lookups, even
those from connect commands and the command line.

FIXME: An implicit setting of use_proxy_always is done in gossipd if it
determines that we are announcing nothing but Tor addresses, but that
does *not* suppress 'connect'.

This is fixed in a later patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell
9d8e3cf3da gossip: handle Tor proxy better.
1. Only force proxy use if we don't announce any non-TOR address.
   There's no option to turn it off, so this makes more sense.
2. Don't assume we want an IPv4 socket to reach proxy, use the family
   from the struct addrinfo.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell
c3ccc14f19 Tor: remove --tor prefix from SOCKS5 options.
It's usually for Tor, but we can use a socks5 proxy without it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell
003cd29733 tor: clean up io_tor_connect.
Instead of storing a wireaddr and converting to an addrinfo every
time, just convert once (which also avoids the memory leak in the
current code).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell
d87a6c3a48 wireaddr: more helpers, to convert to addrinfo.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell
2d840706df tor: don't immediately fail if we can't open cookie file.
We might still be offered password authentication, for example.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell
d5e4d52563 common/base32: make this a simple tal-wrapper around ccan/base32.
And use it in wireaddr.

We fix up the double '.onion' in the test case, which seems like an error?

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell
53c6ceeab9 ccan: import base32 module.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell
e91975c22d wireaddr: use fmt_wireaddr_without_port in fmt_wireaddr.
Avoids duplicate code.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell
7900318216 wireaddr: fix tor address lengths.
This variable does NOT include the 2 bytes for the port.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell
7d95dbbd0a wireaddr: remove FQDN_ADDRLEN in favor of LARGEST_ADDRLEN.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell
c1e0a4d572 gossip/tor: rearrange functions to avoid predeclarations.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell
8b0215549c options: check that combinations of TOR options are valid.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell
e229f113b9 gossipd: don't try to reach tor if we don't have a proxy.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell
11db7ca9e6 options: use NULL for unset Tor settings.
Rename tor_proxyaddrs and tor_serviceaddrs to tor_proxyaddr and tor_serviceaddr:
the 's' at the end suggests that there can be more than one.

Make them NULL or non-NULL, rather than using all-zero if unset.

Hand them the same way to gossipd; it's a bit of a hack since we don't
have optional fields, so we use a counter which is always 0 or 1.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell
ef09961713 lightningd: rewrite tor service interaction to be synchronous, robust.
There's no reason to do this async, and far easier to follow using normal
read/write.

The previous parsing was deeply questionable, using substring searches
only, and relying on the fact that a single non-blocking read would get
the entire response.  This is changed to do (somewhat) proper parsing
using ccan/rbuf.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell
57115f4914 ccan: update and import rbuf module.
This is a simple helper for dealing with buffered I/O.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell
85eff42d9a common/tor: move into lightningd.
This is simply the code to set up the automatic hidden service, so move
it into lightningd.

I removed the undefined parse_tor_wireaddr, and added a parameter name
to the create_tor_hidden_service_conn() declaration for update-mocks.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell
d9f13230cf gossip/tor.c: new file for socks proxy code.
All gossipd needs from common/tor is do_we_use_tor_addr(), so move
that and the rest of the tor-specific handshake code into gossip/tor.c

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell
6d69e7b066 netaddress: fix up IsTor()
We don't actually use it, but let's fix it anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Saibato
877f63e99e Initial TOR v2/v3 support.
This is a rebased and combined patch for Tor support.  It is extensively
reworked in the following patches, but the basis remains Saibato's work,
so it seemed fairest to begin with this.

Minor changes:
1. Use --announce-addr instead of --tor-external.
2. I also reverted some whitespace and unrelated changes from the patch.
3. Removed unnecessary ';' after } in functions.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Isidoro Ghezzi
855d0b9401 "sizeof(sun->sun_path)" is not always the same as "sizeof(addr->u.sockname)"
on all platforms;

because of that BUILD_ASSERT was failing on macOS.

(on macOS "sizeof(sun->sun_path) == 104" and
"sizeof(addr->u.sockname) == 108")

[ Linux man page says it can be as small as 92, so let's use the real value.
  I also cleaned up the incorrect comment order on that struct! --RR ]

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-09 09:10:45 +00:00
Rusty Russell
c33bbb2639 test_lightningd.py: wait longer test_permfail for l2 to notice new blocks.
Reproduced this failure locally: l2 hadn't seen the block yet.  Timeout
was too aggressive.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-09 06:13:49 +00:00
Rusty Russell
2caeab506b tests/test_lightningd.py: make test_blockchaintrack safe to re-run.
It gets confused if re-run due to flaky, since:
1. Using the same HSM, it generates the same spend and sees a previous one.
2. The block height numbers are off.

Fixes: #1479
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-09 06:13:49 +00:00
Rusty Russell
ecebbe700b tests/test_lightningd.py: remove unneeded btc variable.
We have a 'bitcoind' global: getting it from inside one of the daemons
was a mistake I've copied widely.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-09 06:13:49 +00:00
ZmnSCPxj
757c059cd4 test_lightningd: Add closingd torture test. 2018-05-09 04:43:21 +00:00
ZmnSCPxj
926b41b7da closingd: Ensure proper closing of TCP socket.
Fixes: #1457
2018-05-09 04:43:21 +00:00
Felix
80b298a15f help - add preimage to param list 2018-05-09 03:47:53 +00:00
Christian Decker
783327622f pytest: Mark test_blockchaintrack as flaky
The flakyness is probably due to timing issues when reorging.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-08 22:06:04 +02:00
Christian Decker
fac1d24ce3 pytest: Use the port from the lightning node
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-08 22:06:04 +02:00
Christian Decker
81dc82de14 gossip: Clean up stale store argument to handle_gossip_msg
This is a leftover from before splitting the `gossip_store` injection path from
the handling of gossip messages.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-08 22:06:04 +02:00
Christian Decker
5f7b0fdf92 pytest: Marking test_permfail as flaky
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-08 22:06:04 +02:00
Mark Beckwith
9153688b77 Added fundchannel man page.
This describes 'fundchannel' as it works today.

It definitely could use better error codes instead of -1 for everything.
2018-05-08 07:11:35 +00:00
ZmnSCPxj
c79b3de4d6 payalgo: Report reason to delay before clearing try memory.
We allocate the reason to delay, if any, from the `pay->try_parent`.
So we should not clear the `pay->try_parent` until after we print
the reason.
2018-05-08 07:08:37 +00:00
Christian Decker
104645ba3a pytest: Fix flaky test_closing_id test
It simply wasn't waiting for us to register the peer before attempting to open a
connection. Moved into a separate test to be able rerun multiple times.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-08 05:43:35 +00:00
Rusty Russell
79902b48c7 test_lightningd.py: remove gratuitous 'addr' option from test_connect_by_gossip.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell
d40d22b68e gossipd: don't try to connect to non-routable addresses.
Someone could try to announce an internal address, and we might probe
it.

This breaks tests, so we add '--dev-allow-localhost' for our tests, so
we don't eliminate that one.  Of course, now we need to skip some more
tests in non-developer mode.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell
af065417e1 gossipd: handle wildcard addresses correctly.
If we're given a wildcard address, we can't announce it like that: we need
to try to turn it into a real address (using guess_address).  Then we
use that address.  As a side-effect of this cleanup, we only announce
*any* '--addr' if it's routable.

This fix means that our tests have to force '--announce-addr' because
otherwise localhost isn't routable.

This means that gossipd really controls the addresses now, and breaks
them into two arrays: what we bind to, and what we announce.  That is
now what we return to the master for json_getinfo(), which prints them
as 'bindings' and 'addresses' respectively.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell
52917ff6c9 More flexible address wildcards, only add wildcard if nothing else.
1. Add special option where an empty host means 'wildcard for IPv4 and/or IPv6'
   which means ':1234' can be used to set only the portnum.
2. Only add this protocol wildcard if --autolisten=1 (default)
   and no other addresses specified.
3. Pass it down to gossipd, so it can handle errors correctly: in most cases,
   it's fatal not to be able to bind to a port, but for this case, it's OK
   if we can only bind to one of IPv4/v6 (fatal iff neither).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell
73cd009a4c gossipd/lightningd: use wireaddr_internal.
This replacement is a little menial, but it explicitly catches all
the places where we allow a local socket.  The actual implementation of
opening a AF_UNIX socket is almost hidden in the patch.

The detection of "valid address" is now more complex:

	p->addr.itype != ADDR_INTERNAL_WIREADDR || p->addr.u.wireaddr.type != ADDR_TYPE_PADDING

But most places we do this, we should audit: I'm pretty sure we can't
get an invalid address any more from gossipd (they may be in db, but
we should fix that too).

Closes: #1323
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell
6b2282fc1d wireaddr_internal: new type for where we can also use a local socket.
This was something @icota implemented, but it fits logically into this
cleanup series.  We create a new type which is the internal generalization
of a wireaddr (which is defined by the spec), and add a case here for
a socket name.

Based-on-the-true-story-by: @icota
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell
e6c678e5df gossipd: take over address determination, from master.
It does all the other address handling, do this too.  It also proves useful
as we clean up wildcard address handling.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell
356e5dcea8 wireaddr: helpers to convert to/from IPv4/v6 addresses.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell
9c0de76019 lightningd: still bind to local ports even if address not public.
Now we only bind to addresses in our wireaddrs array, we would not
autobind to local sockets if they couldn't reach google's nameserver.

That's clearly wrong: we should only not bind if there's a protocol
issue (eg. no IPv6 support).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell
fe96fe10c7 Clean up network options.
It's become clear that our network options are insufficient, with the coming
addition of Tor and unix domain support.

Currently:

1. We always bind to local IPv4 and IPv6 sockets, unless --port=0, --offline,
   or any address is specified explicitly.  If they're routable, we announce.
2. --addr is used to announce, but not to control binding.

After this change:

1. --port is deprecated.
2. --addr controls what we bind to and announce.
3. --bind-addr/--announce-addr can be used to control one and not the other.
4. Unless --autolisten=0, we add local IPv4 & IPv6 port 9735 (and announce if they are routable).
5. --offline still overrides listening (though announcing is still the same).

This means we can bind to as many ports/interfaces as we want, and for
special effects we can announce different things (eg. we're sitting
behind a port forward or a proxy).

What remains to implement is semi-automatic binding: we should be able
to say '--addr=0.0.0.0:9999' and have the address resolve at bind
time, or even '--addr=0.0.0.0:0' and have the port autoresolve too
(you could determine what it was from 'lightning-cli getinfo'.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell
00537fde43 lightningd: deprecate --ipaddr in favor of --addr.
We're going to add sockets, and later onion addresses, so the current name
is bad.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell
ed466a8523 lightningd: make explicit listen and reconnect flags.
We set no_reconnect with --offline, but that doesn't work if !DEVELOPER.
Make the flag positive, and non-DEVELOPER mode for gossipd.

We also don't override portnum with --offline, but have an explicit
'listen' flag.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00