Commit Graph

3805 Commits

Author SHA1 Message Date
Rusty Russell
85eff42d9a common/tor: move into lightningd.
This is simply the code to set up the automatic hidden service, so move
it into lightningd.

I removed the undefined parse_tor_wireaddr, and added a parameter name
to the create_tor_hidden_service_conn() declaration for update-mocks.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell
d9f13230cf gossip/tor.c: new file for socks proxy code.
All gossipd needs from common/tor is do_we_use_tor_addr(), so move
that and the rest of the tor-specific handshake code into gossip/tor.c

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell
6d69e7b066 netaddress: fix up IsTor()
We don't actually use it, but let's fix it anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Saibato
877f63e99e Initial TOR v2/v3 support.
This is a rebased and combined patch for Tor support.  It is extensively
reworked in the following patches, but the basis remains Saibato's work,
so it seemed fairest to begin with this.

Minor changes:
1. Use --announce-addr instead of --tor-external.
2. I also reverted some whitespace and unrelated changes from the patch.
3. Removed unnecessary ';' after } in functions.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Isidoro Ghezzi
855d0b9401 "sizeof(sun->sun_path)" is not always the same as "sizeof(addr->u.sockname)"
on all platforms;

because of that BUILD_ASSERT was failing on macOS.

(on macOS "sizeof(sun->sun_path) == 104" and
"sizeof(addr->u.sockname) == 108")

[ Linux man page says it can be as small as 92, so let's use the real value.
  I also cleaned up the incorrect comment order on that struct! --RR ]

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-09 09:10:45 +00:00
Rusty Russell
c33bbb2639 test_lightningd.py: wait longer test_permfail for l2 to notice new blocks.
Reproduced this failure locally: l2 hadn't seen the block yet.  Timeout
was too aggressive.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-09 06:13:49 +00:00
Rusty Russell
2caeab506b tests/test_lightningd.py: make test_blockchaintrack safe to re-run.
It gets confused if re-run due to flaky, since:
1. Using the same HSM, it generates the same spend and sees a previous one.
2. The block height numbers are off.

Fixes: #1479
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-09 06:13:49 +00:00
Rusty Russell
ecebbe700b tests/test_lightningd.py: remove unneeded btc variable.
We have a 'bitcoind' global: getting it from inside one of the daemons
was a mistake I've copied widely.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-09 06:13:49 +00:00
ZmnSCPxj
757c059cd4 test_lightningd: Add closingd torture test. 2018-05-09 04:43:21 +00:00
ZmnSCPxj
926b41b7da closingd: Ensure proper closing of TCP socket.
Fixes: #1457
2018-05-09 04:43:21 +00:00
Felix
80b298a15f help - add preimage to param list 2018-05-09 03:47:53 +00:00
Christian Decker
783327622f pytest: Mark test_blockchaintrack as flaky
The flakyness is probably due to timing issues when reorging.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-08 22:06:04 +02:00
Christian Decker
fac1d24ce3 pytest: Use the port from the lightning node
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-08 22:06:04 +02:00
Christian Decker
81dc82de14 gossip: Clean up stale store argument to handle_gossip_msg
This is a leftover from before splitting the `gossip_store` injection path from
the handling of gossip messages.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-08 22:06:04 +02:00
Christian Decker
5f7b0fdf92 pytest: Marking test_permfail as flaky
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-08 22:06:04 +02:00
Mark Beckwith
9153688b77 Added fundchannel man page.
This describes 'fundchannel' as it works today.

It definitely could use better error codes instead of -1 for everything.
2018-05-08 07:11:35 +00:00
ZmnSCPxj
c79b3de4d6 payalgo: Report reason to delay before clearing try memory.
We allocate the reason to delay, if any, from the `pay->try_parent`.
So we should not clear the `pay->try_parent` until after we print
the reason.
2018-05-08 07:08:37 +00:00
Christian Decker
104645ba3a pytest: Fix flaky test_closing_id test
It simply wasn't waiting for us to register the peer before attempting to open a
connection. Moved into a separate test to be able rerun multiple times.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-08 05:43:35 +00:00
Rusty Russell
79902b48c7 test_lightningd.py: remove gratuitous 'addr' option from test_connect_by_gossip.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell
d40d22b68e gossipd: don't try to connect to non-routable addresses.
Someone could try to announce an internal address, and we might probe
it.

This breaks tests, so we add '--dev-allow-localhost' for our tests, so
we don't eliminate that one.  Of course, now we need to skip some more
tests in non-developer mode.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell
af065417e1 gossipd: handle wildcard addresses correctly.
If we're given a wildcard address, we can't announce it like that: we need
to try to turn it into a real address (using guess_address).  Then we
use that address.  As a side-effect of this cleanup, we only announce
*any* '--addr' if it's routable.

This fix means that our tests have to force '--announce-addr' because
otherwise localhost isn't routable.

This means that gossipd really controls the addresses now, and breaks
them into two arrays: what we bind to, and what we announce.  That is
now what we return to the master for json_getinfo(), which prints them
as 'bindings' and 'addresses' respectively.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell
52917ff6c9 More flexible address wildcards, only add wildcard if nothing else.
1. Add special option where an empty host means 'wildcard for IPv4 and/or IPv6'
   which means ':1234' can be used to set only the portnum.
2. Only add this protocol wildcard if --autolisten=1 (default)
   and no other addresses specified.
3. Pass it down to gossipd, so it can handle errors correctly: in most cases,
   it's fatal not to be able to bind to a port, but for this case, it's OK
   if we can only bind to one of IPv4/v6 (fatal iff neither).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell
73cd009a4c gossipd/lightningd: use wireaddr_internal.
This replacement is a little menial, but it explicitly catches all
the places where we allow a local socket.  The actual implementation of
opening a AF_UNIX socket is almost hidden in the patch.

The detection of "valid address" is now more complex:

	p->addr.itype != ADDR_INTERNAL_WIREADDR || p->addr.u.wireaddr.type != ADDR_TYPE_PADDING

But most places we do this, we should audit: I'm pretty sure we can't
get an invalid address any more from gossipd (they may be in db, but
we should fix that too).

Closes: #1323
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell
6b2282fc1d wireaddr_internal: new type for where we can also use a local socket.
This was something @icota implemented, but it fits logically into this
cleanup series.  We create a new type which is the internal generalization
of a wireaddr (which is defined by the spec), and add a case here for
a socket name.

Based-on-the-true-story-by: @icota
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell
e6c678e5df gossipd: take over address determination, from master.
It does all the other address handling, do this too.  It also proves useful
as we clean up wildcard address handling.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell
356e5dcea8 wireaddr: helpers to convert to/from IPv4/v6 addresses.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell
9c0de76019 lightningd: still bind to local ports even if address not public.
Now we only bind to addresses in our wireaddrs array, we would not
autobind to local sockets if they couldn't reach google's nameserver.

That's clearly wrong: we should only not bind if there's a protocol
issue (eg. no IPv6 support).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell
fe96fe10c7 Clean up network options.
It's become clear that our network options are insufficient, with the coming
addition of Tor and unix domain support.

Currently:

1. We always bind to local IPv4 and IPv6 sockets, unless --port=0, --offline,
   or any address is specified explicitly.  If they're routable, we announce.
2. --addr is used to announce, but not to control binding.

After this change:

1. --port is deprecated.
2. --addr controls what we bind to and announce.
3. --bind-addr/--announce-addr can be used to control one and not the other.
4. Unless --autolisten=0, we add local IPv4 & IPv6 port 9735 (and announce if they are routable).
5. --offline still overrides listening (though announcing is still the same).

This means we can bind to as many ports/interfaces as we want, and for
special effects we can announce different things (eg. we're sitting
behind a port forward or a proxy).

What remains to implement is semi-automatic binding: we should be able
to say '--addr=0.0.0.0:9999' and have the address resolve at bind
time, or even '--addr=0.0.0.0:0' and have the port autoresolve too
(you could determine what it was from 'lightning-cli getinfo'.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell
00537fde43 lightningd: deprecate --ipaddr in favor of --addr.
We're going to add sockets, and later onion addresses, so the current name
is bad.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell
ed466a8523 lightningd: make explicit listen and reconnect flags.
We set no_reconnect with --offline, but that doesn't work if !DEVELOPER.
Make the flag positive, and non-DEVELOPER mode for gossipd.

We also don't override portnum with --offline, but have an explicit
'listen' flag.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell
6ba9a7cb57 tests/test_lightningd.py: move port into node.
We're about to change the JSONRPC, so let's put an explicit 'port' into
our node class.

We initialize it at startup time: in future I hope to use ephemeral ports
to make our tests more easily parallelizable.

Suggested-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell
7d27501c94 tests/utils.py: remove unused connect routine.
It's broken anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 22:37:28 +02:00
Rusty Russell
99bba2a23c channeld: don't send ANNOUNCEMENT_SIGNATURES if we've send shutdown.
Our closingd doesn't handle it:

lightningd(2968): 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518 chan #1:
 Peer permanent failure in CLOSINGD_SIGEXCHANGE: lightning_closingd: sent ERROR Expected closing_signed:
 0103ff54517293892ec3f214f2343c54cbfbf24aa6ffb8d5585d3bc1b543eae0a272000067000001000146390e0c043c777226927eacd2186a03f064e4bdc30f891cb6e4990af49967d34b338755e99d728987e3d49227815e17f3ab40092434a59e33548e870071176d26d19a4e4d8f7715c13ac2d6bf3238608a1ccf9afd91f774d84d170d9edddebf7460c54d49bd6cd81410bc3eeeba2b7278b1b5f7e748d77d793f31086847d582

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-07 12:48:19 +02:00
Christian Decker
6231e9969f pytest: Give lightningd nodes a numeric ID to prefix logs
This used to be the port, but since we no longer have fixed ports, and we start
them in random order we can't easily distinguish them by the port anymore. Just
use a numeric ID that matches their lightning-dirs.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-07 02:40:50 +00:00
Christian Decker
65cac1ba85 pytest: Add flaky dependency and mark as test_closing_different_fees
This is mainly just a stopgap solution until we get to stabilize the test.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-07 02:40:50 +00:00
Christian Decker
957b32affe contrib: Add the flaky dependency to the builder image
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-07 02:40:50 +00:00
Christian Decker
e00c7012cc pytest: Ensure unique test directories per test even if rerun
We add an attempt number to the test directory to improve the test-isolation and
allow for multiple reruns of the same test, without re-using any of the
lightning-dirs or bitcoin-datadirs.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-07 02:40:50 +00:00
Christian Decker
71857bad8e builder: Add the ephemeral-port-reserve dependency to the builder
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-07 02:40:50 +00:00
Christian Decker
071ef628db pytest: Use random ports for bitcoind and lightningd to allow parallel testing
Adds a new dependency, but totally worth it :-)

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-07 02:40:50 +00:00
Christian Decker
bbeb44faac pytest: Make the directory of the NodeFactory configurable
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-07 02:40:50 +00:00
Christian Decker
727d115296 pytest: Add py.test fixtures and migrate first example test
This is the first example of the py.test style fixtures which should allow us to
write much cleaner and nicer tests.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-07 02:40:50 +00:00
Christian Decker
b307df0002 pytest: Let connect start nodes in parallel
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-07 02:40:50 +00:00
Christian Decker
e3bac12683 pytest: Allow stdout to be dropped for bitcoind
We never really look at the output, and it's rather noisy, so we just stop
writing to the log.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-07 02:40:50 +00:00
Christian Decker
4f63acc77c pytest: Add helper to start a number of nodes in parallel
Especially with valgrind this allows us to safe quite some time on multi-core
machines since startup is about the most expensive operation in the tests. In a
simple test spinning up 3 nodes this gave me about 25% - 30% test time
reduction. The effects will be smaller for single core machines, hoping Travis
handles these gracefully.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-07 02:40:50 +00:00
Christian Decker
81b715aae9 pytest: Move wait_for to utils.py and make fund_channel a member
Slowly moving towards a more pythonic approach to the testing code.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-07 02:40:50 +00:00
Christian Decker
7aa13cc949 channel: Queue a channel_update to the peer upon funding_locked
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-07 01:10:48 +00:00
Christian Decker
9cfd09dc4a gossip: HalfChans are public if we have an update and the Chan is
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-07 01:10:48 +00:00
Christian Decker
b028a363d8 gossip: Make sure we never add a channel twice
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-07 01:10:48 +00:00
practicalswift
8cc02f63bc gossipd: Handle failed lseek(...) 2018-05-06 20:45:10 +02:00
Christian Decker
e308423954 travis: Fix the locale in the builder images
These were spamming the logs and making the log view almost unusable.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-06 15:45:52 +02:00