Commit Graph

3975 Commits

Author SHA1 Message Date
Rusty Russell
8739b4cbe8 lighningd: Remove --debug-subdaemon-io.
We can use SIGUSR1, even in non-developer builds.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-20 02:32:42 +00:00
Rusty Russell
edf1b3cec9 More option cleanups.
Because we have too many which are never used and I don't want to document
them.

1. Remove unused anchor_onchain_wait.  When implemented, it should be
   hardcoded to 100 or more.
2. Remove anchor_confirms_max.  10 always reasonable, and we can readd
   an override option should someone need it.
3. max_htlc_expiry should be the same as locktime_max (which increases
   from 3 to 5 days by default): they're both a limit on how long
   funds can be locked up.
4. channel_update_interval should always be a dev option.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-20 02:32:42 +00:00
Rusty Russell
0aa22741df option cleanup: --dev-override-fee-rates
Make --override-fee-rates a dev option.  We use default-fee-rate in
its place, which (since bitcoind won't give fee estimates in regtest
mode for short chains) gives an effective feerate of 15000/7500/3750.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-20 02:32:42 +00:00
Rusty Russell
ceb1ce0ef5 chaintopology: fix default feerate.
We never hit the guess_feerate() path, because we turned a 0 ("can't
estimate fee") into 253.

This also revealed that we weren't initializing topo->feerate, and
that we were giving spurious updates even if we were using override-fee-rates.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-20 02:32:42 +00:00
Rusty Russell
c8cc8fd83f option cleanup: --dev-bitcoind-poll
Make --bitcoind-poll a dev option.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-20 02:32:42 +00:00
Rusty Russell
adbe02c6be gossip: temporarily allow replacement of updates with same timestamp.
We erroneously create updates with the same timestamps when tests run
quickly, and the second one is ignored.

We've already noted that this should be fixed: gossipd should generate
all the updates, as it already has to do the case where channeld
crashed, for example.  But that's a bigger change.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-19 15:52:56 -04:00
Rusty Russell
85a062164a test_lightningd: test_closing_torture timeout increase.
I hit this timeout on pytest and valgrind with -n5; use the timeout variable
from utils.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-19 15:52:56 -04:00
Rusty Russell
f92712f22c channeld: always send either a temporary or final update if we can.
If we hit depth 6, we would start exchanging announcement signatures.
However, we should still send a temporary update while waiting for the
reply; make the logic clear in this case that we should always send
one or the other.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-19 15:52:56 -04:00
Rusty Russell
00d4c04208 channeld: fix conditions under which we can send a channel update.
The condition in send_channel_update is wrong: it needs to match the
conditions under which we send announcements.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-19 15:52:56 -04:00
Rusty Russell
c546b1bbb6 gossipd: specify origin of updates in errors.
@cdecker points out that in test_forward, where we manually create a route,
we get an error back which contains an update for an unknown channel.

We should still note this, but it's not an error for testing.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-19 15:52:56 -04:00
Rusty Russell
8ee60e2d8e testing: make sure we don't see gossip in bad order.
This is something which generally shouldn't happen, but we didn't
notice it previously.

We ignore this warning in the case where a channel was deleted: this
happens because one side can send an update while the other notices
that the channel is closed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-19 15:52:56 -04:00
Rusty Russell
177a1fc88e gossipd: handle local channel creation separately from update.
Note: this will break the gossip_store if they have current channels,
but it will fail to parse and be discarded.

Have local_add_channel do just that: the update is logically separate
and can be sent separately.

This removes the ugly 'bool add_to_store' flag.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-19 15:52:56 -04:00
Rusty Russell
f0231b0e6f channeld: don't send temporary announcements more than once.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-19 15:52:56 -04:00
Rusty Russell
f014cbb78c channeld: clean up announcement state tests.
We always call:

	send_temporary_announcement(peer);
	send_announcement_signatures(peer);

We should handle these in one place, since the conditional at the top
of them actually makes sure only one is effective.  We also make the
caller set the peer->have_sigs[LOCAL] flag, instead of doing it
inside send_announcement_signatures().

We were sending announcements at the wrong time (on restart) somtimes.

We also move announce_channel() into the same logic, so it's always
together.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-19 15:52:56 -04:00
Rusty Russell
323472225c channeld: simplify announce/locked-in callback,
Just have a "new depth" callback, and let channeld do the right thing.

This makes the channeld paths a bit more straightforward.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-19 15:52:56 -04:00
Rusty Russell
981ffb83f7 channeld: don't send updates for 0:0:0.
Some paths (eg reconnect) were unconditionally sending a channel_update.
valgrind wasn't catching it because we unmarshal short_channel_ids[LOCAL]
as all-zeroes, so it's technically "initialized".

Create a wrapper to do this, and change the 'bool disabled' flag to be
the explicit disable flag value for clarity.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-19 15:52:56 -04:00
Rusty Russell
540c68d7ca gossipd/gossip_constants.h: Single place for BOLT constants.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-19 15:52:56 -04:00
Rusty Russell
b965ef7d1d routing: make sure we fail if we can't unmarshal announcements.
This is how we notice if the gossip store is corrupt!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-19 15:52:56 -04:00
Rusty Russell
d52564dc05 check-source: add check for tal_free(tmpctx).
My older rebased patches use this, and I want to catch it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-19 15:52:56 -04:00
Rusty Russell
cae25ca5db devtools/print_wire: add return for numerical fields.
Before:
    $ ./devtools/decodemsg 0102c2bd3f4a94ff390ce764caf51925d0ed38fa95b6539945b42124f5c4e625da63351380c79230a05550d0e5def9c2412f4f164478f9f9491140e505f79c0d716506226e46111a0b59caaf126043eb5bbf28c34f3a5e332a1fc7b2b73cf188910f00006700000100015afd0da1000000060000000000000000000000010000000a
    WIRE_CHANNEL_UPDATE:
    signature=3045022100c2bd3f4a94ff390ce764caf51925d0ed38fa95b6539945b42124f5c4e625da630220351380c79230a05550d0e5def9c2412f4f164478f9f9491140e505f79c0d7165
    chain_hash=0f9188f13cb7b2c71f2a335e3a4fc328bf5beb436012afca590b1a11466e2206
    short_channel_id=103:1:1
    timestamp=1526533537flags=0cltv_expiry_delta=6htlc_minimum_msat=0fee_base_msat=1fee_proportional_millionths=10

After:

    WIRE_CHANNEL_UPDATE:
    signature=3045022100c2bd3f4a94ff390ce764caf51925d0ed38fa95b6539945b42124f5c4e625da630220351380c79230a05550d0e5def9c2412f4f164478f9f9491140e505f79c0d7165
    chain_hash=0f9188f13cb7b2c71f2a335e3a4fc328bf5beb436012afca590b1a11466e2206
    short_channel_id=103:1:1
    timestamp=1526533537
    flags=0
    cltv_expiry_delta=6
    htlc_minimum_msat=0
    fee_base_msat=1
    fee_proportional_millionths=10

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-19 15:52:56 -04:00
Rusty Russell
bd1c16d284 update-ccan: clean up the two unwanted imports.
We don't import these, though they're technically dependencies.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-19 15:52:56 -04:00
Rusty Russell
6fe2c037dc Add .rej files to gitignore.
They turn up for me when I force-push a guilt patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-19 15:52:56 -04:00
Christian Decker
e604891910 pytest: Do not set test-groups unless explicitly requested
Setting these options could fail the `pytest` target if `pytest-test-groups` is
not installed.

Fixes #1508

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-05-16 19:42:44 -04:00
ZmnSCPxj
b79bb1dbe6 gitignore: Ignore run-features test. 2018-05-16 04:59:29 +00:00
Rusty Russell
7bfb9978de wire: remove --check-alignment arg from spec parsing tool.
It's no longer functional, and is being removed upstream.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-15 23:39:09 +00:00
practicalswift
ba20ace064 Do not use potentially uninitialized value needed_dns 2018-05-15 15:05:16 +02:00
Rusty Russell
4359f46ffd features: fix features_supported() function.
1. We need to test all bits, not all bytes.
2. Both local and global features need to be supported.
3. Untested code is broken code.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-15 15:04:34 +02:00
Rusty Russell
149e967ba3 features: fix endian.
Closes: https://github.com/lightningnetwork/lightning-rfc/issues/424

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-15 15:04:34 +02:00
Saibato
8659745ca6 update TOR.md to latest changes
Signed-off-by: Saibato <saibato.naga@protonmail.com>
2018-05-15 07:16:43 +00:00
nicolas.dorier
59154ebd34 Fix build on linux-musl
[ Done by updating CCAN with the ccan/endian fix --RR ]
2018-05-15 05:54:15 +00:00
practicalswift
3abd9ad4f6 wireaddr: Avoid memory corruption in fmt_wireaddr_without_port() 2018-05-15 05:30:43 +00:00
practicalswift
fab3b214b4 Avoid static analyzer warning about integer wraparound 2018-05-15 05:26:29 +00:00
Rusty Russell
1125682ceb wireaddr: new type, ADDR_INTERNAL_FORPROXY, use it if we can't/wont resolve.
Tor wasn't actually working for me to connect to anything, but it worked
for 'ssh -D' testing.

Note that the resulting 'netaddr' is a bit weird, but I guess it's honest.

    $ ./cli/lightning-cli connect 021f2cbffc4045ca2d70678ecf8ed75e488290874c9da38074f6d378248337062b
    {
      "id": "021f2cbffc4045ca2d70678ecf8ed75e488290874c9da38074f6d378248337062b"
    }
    $ ./cli/lightning-cli listpeers
    {
      "peers": [
        {
          "state": "GOSSIPING", 
          "id": "021f2cbffc4045ca2d70678ecf8ed75e488290874c9da38074f6d378248337062b", 
          "netaddr": [
            "ln1qg0je0lugpzu5ttsv78vlrkhteyg9yy8fjw68qr57mfhsfyrxurzkq522ah.lseed.bitcoinstats.com:9735"
          ], 
          "connected": true, 
          "owner": "lightning_gossipd"
        }
      ]
    }

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-11 09:15:54 +00:00
Rusty Russell
2a0acd3492 tor: log proxy communications using status_io.
Good for debugging (you have to send SIGUSR1 to lightning_gossipd to turn
it on though, and --log-level=io on the lightningd cmdline to have it
output IO messages by default).

I also noticed that io_tor_connect_after_req_host() does a useless
test on reach->buffer[0] after it's *written*: remove it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-11 09:15:54 +00:00
Rusty Russell
570283bc76 gossipd: don't use fake addrhint for non-addrhint resolutions.
Use a wireaddr_internal directly (which is what we want).

Also, don't hardcode 9735, use DEFAULT_PORT internally in
seed_resolve_addr().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-11 09:15:54 +00:00
Rusty Russell
de063edb54 gossip: extract function to derive seedname.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-11 09:15:54 +00:00
Rusty Russell
0d23f4fb4a gossipd: hand io_tor_connect the host as a string.
Previously it converted the wireaddr to a string internally: to support
unresolved names we need that done externally.

We actually tell the SOCKS5 proxy to do a domain lookup already, even
though we give use IP/IPv6 address, so this change is sufficient to
support connect-by-name.

Note replacement of assert() with an explicit case statement, which
has the benefit that the compiler complains when we add new
ADDR_INTERNAL types.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-11 09:15:54 +00:00
Rusty Russell
a1dc4eef56 wireaddr: tell caller that we failed due to wanting DNS lookup, don't try.
This is useful for the next patch, where we want to hand the unresolved
name through to the proxy.

This also addresses @Saibato's worry that we still called getaddrinfo()
(with the AI_NUMERICHOST option) even if we didn't want a lookup.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-11 09:15:54 +00:00
Rusty Russell
5345e43354 gossipd: rename use_tor to use_proxy,
Not all of them, but it's really about using the SOCKS proxy rather than
really using Tor at this level.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-11 09:15:54 +00:00
Rusty Russell
bcb047a729 gossipd: fix uninitialized var.
We assert() that it's set by one of the branches (it should be!) but
if we don't hit one it's uninitialized, not NULL.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-11 09:15:54 +00:00
Rusty Russell
011d75c618 lightningd: don't allow --announce-addr with a wildcard address.
It doesn't make sense.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-11 09:15:54 +00:00
Rusty Russell
1ab9a5def6 wireaddr: fix parsing of empty address with no port.
Port should become the default in not specified.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-11 09:15:54 +00:00
Rusty Russell
cca791d1cb routing: clean up channel public/active states.
1. If we have a channel_announcement, the channel is public, otherwise
   it's not.  Not all channels are public, as they can be local: those
   have a NULL channel_announcement.

2. If we don't have a channel_update, we know nothing about that half
   of the channel, and no other fields are valid.

3. We can tell if a half channel is disabled by the flags field directly.

Note that we never send halfchannels without an update over
gossip_getchannels_reply so that marshalling/unmarshalling can be
vastly simplified.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 21:35:53 +02:00
Rusty Russell
9d1e496b11 gossipd: use a real update in local_add_channel.
We generate one now, so let's use it.  That lets us simplify the
code, too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 21:35:53 +02:00
Rusty Russell
c71e16f784 broadcast: invert ownership of messages.
Make the update/announce messages own the element in the broadcast map
not the other way around.

Then we keep a pointer to the message, and when we free it
(eg. channel closed, update replaces it), it gets freed from the
broadcast map automatically.

The result is much nicer!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 21:35:53 +02:00
Rusty Russell
8940528bdb gossipd: don't include private announcements into broadcast map.
Basically, if we don't have an announcement for the channel, stash it,
and once we get an announcement, replay if necessary.

Fixes: #1485
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 21:35:53 +02:00
Rusty Russell
cdded4f53d tests/utils.py: make default timeout configurable via $TIMEOUT
This is useful when you're expecting a test to fail, you've just added
diagnostics.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 21:35:53 +02:00
Rusty Russell
d1b28f832d gossipd: when reconnecting, make sure we free old connection.
Looks like old connection got a callback, and we blew up since
the old peer was freed:

2018-05-06T10:57:11.865Z lightning_gossipd(14387): ...will try again in 300 seconds
2018-05-06T10:57:16.397Z lightning_gossipd(14387): peer_out WIRE_INIT
2018-05-06T10:57:16.405Z lightning_gossipd(14387): peer_in WIRE_INIT
2018-05-06T10:57:16.406Z lightning_gossipd(14387): peer 03b30e131241fe28fc923d74a060a8c7abfcc91323c485f8a9cf964575cb4fd3f4: reconnect for local peer
2018-05-06T10:57:16.406Z lightning_gossipd(14387): peer 03b30e131241fe28fc923d74a060a8c7abfcc91323c485f8a9cf964575cb4fd3f4 now remote
2018-05-06T10:57:16.406Z lightning_gossipd(14387): UPDATE WIRE_GOSSIP_PEER_CONNECTED
2018-05-06T10:57:16.406Z lightning_gossipd(14387): UPDATE WIRE_GOSSIP_PEER_CONNECTED
2018-05-06T10:57:16.406Z lightning_gossipd(14387): Handing back peer 03b30e131241fe28fc923d74a060a8c7abfcc91323c485f8a9cf964575cb4fd3f4 to master
2018-05-06T10:57:16.420Z lightning_gossipd(14387): hand_back_peer 03b30e131241fe28fc923d74a060a8c7abfcc91323c485f8a9cf964575cb4fd3f4: now local again
2018-05-06T10:57:16.420Z lightning_gossipd(14387): FATAL SIGNAL 11
2018-05-06T10:57:16.420Z lightning_gossipd(14387): backtrace: common/daemon.c:42 (crashdump) 0x416991
2018-05-06T10:57:16.420Z lightning_gossipd(14387): backtrace: (null):0 ((null)) 0x7f70cf57a4af
2018-05-06T10:57:16.420Z lightning_gossipd(14387): backtrace: common/msg_queue.c:38 (msg_dequeue) 0x418232
2018-05-06T10:57:16.421Z lightning_gossipd(14387): backtrace: gossipd/gossip.c:816 (peer_pkt_out) 0x404ac4
2018-05-06T10:57:16.421Z lightning_gossipd(14387): backtrace: ccan/ccan/io/io.c:59 (next_plan) 0x4316db
2018-05-06T10:57:16.421Z lightning_gossipd(14387): backtrace: ccan/ccan/io/io.c:427 (io_do_always) 0x4322ce
2018-05-06T10:57:16.421Z lightning_gossipd(14387): backtrace: ccan/ccan/io/poll.c:228 (handle_always) 0x433abd
2018-05-06T10:57:16.421Z lightning_gossipd(14387): backtrace: ccan/ccan/io/poll.c:249 (io_loop) 0x433b48
2018-05-06T10:57:16.421Z lightning_gossipd(14387): backtrace: gossipd/gossip.c:2407 (main) 0x4093aa
2018-05-06T10:57:16.421Z lightning_gossipd(14387): backtrace: (null):0 ((null)) 0x7f70cf56582f
2018-05-06T10:57:16.421Z lightning_gossipd(14387): backtrace: (null):0 ((null)) 0x402ad8
2018-05-06T10:57:16.421Z lightning_gossipd(14387): backtrace: (null):0 ((null)) 0xffffffffffffffff
2018-05-06T10:57:16.421Z lightning_gossipd(14387): STATUS_FAIL_INTERNAL_ERROR: FATAL SIGNAL

Fixes: #1469
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 21:11:00 +02:00
Rusty Russell
2bc9d7594c doc/TOR.md: update for new options.
Currently the Tor port for autotor is always 9735; the --port option
is deprecated, though we could append an optional '[@portnum]' to
'autotor' if people want it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00
Rusty Russell
89c76a5a78 Move always-use-proxy auto-override to master daemon.
This means it will effect connect commands too (though it's too
late to stop DNS lookups caused by commandline options).

We also warn that this is one case where we allow forcing through Tor
without a proxy set: it just means all connections will fail.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-10 02:28:44 +00:00