Commit Graph

6776 Commits

Author SHA1 Message Date
Rusty Russell
4b13c92802 seeker: use hash table for unknown short_channel_ids.
Instead of a linear array which is fairly inefficient if it turns out
we know nothing at all.

We remove the gossip_missing() call by changing the api to
remove_unknown_scid() to include a flag as to whether the scid turned
out to be real or not.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-10 21:48:52 -05:00
Rusty Russell
83575f27a1 seeker: add code to check range of scids.
Once we've finished streaming gossip from the first peer, we ask a
random peer (maybe the same one) for all short_channel_ids in the last
6 blocks from the latest channel we know about.

If this reveals new channels we didn't know about, we expand the probe
by a factor of 2 each time.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-10 21:48:52 -05:00
Rusty Russell
521c7f7121 seeker: take over gossip control.
The seeker starts by asking a peer (the first peer!) for all gossip
since a minute before the modified time of the gossip store.

This algorithm is enhanced in successive patches.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-10 21:48:52 -05:00
Rusty Russell
55323ec385 gossipd: move gossip seeking routines into new file seeker.c
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-10 21:48:52 -05:00
Rusty Russell
79ca9bf998 gossipd: use per-peer information to make messages clearer.
We can (usually) indicate what peer caused the bad gossip error.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-10 21:48:52 -05:00
Rusty Russell
0091300ee3 gossipd: track what peer gave us gossip msgs so we can credit it.
Since we have to validate, there can be a delay (and peer might
vanish) between receiving the gossip and actually confirming it, hence
the use of softref.

We will use this information to check that the peers are making progress
as we start asking them for specific information.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-10 21:48:52 -05:00
Rusty Russell
d5eca470dc utils: add set_softref() / clear_softref().
We often want a pointer which will turn to NULL if the pointed-to thing is
freed.  This is possible with tal objects, so create it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-10 21:48:52 -05:00
Rusty Russell
296868daf4 gossipd: have gossip_store_load() return a timestamp.
This is the modified-time of the file.  We have to store it internally
since we overwrite the gossip file with compaction on startup.

This means the "are we behind on gossip?" heuristic is no longer inside
gossip_store.c, which is cleaner.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-10 21:48:52 -05:00
Rusty Russell
d75302deba gossipd: random_peer() selector.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-10 21:48:52 -05:00
Rusty Russell
8a3abdd5ff pytest: fix test_restart_many_payments
It sometimes fail with a bad_gossip error because the sending node
might not have found out about the channel when it gets a
channel_update.  Make sure the whole network knows everything before
we start.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-10 21:48:52 -05:00
Rusty Russell
213263ca3f pytest: filter out gossip_timetamp_filter should we receive it.
We completely rework test_node_reannounce: it's assumes we always ask for
all gossip and that assumption will be broken in future patches too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-10 21:48:52 -05:00
Rusty Russell
a3f6ce1f8f pytest: add filters arg to query_gossip()
Code changes mean we're going to get gossip_timestamp_filter messages from
peers.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-10 21:48:52 -05:00
Rusty Russell
9d336763ff devtools/dump-gossipstore: get offets correct when we have DELETED entries.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-10 21:48:52 -05:00
Rusty Russell
247fb6fa6a common/gossip_store: fix lseek argument order on gossip_store reload.
We don't currently use this, but it's badly wrong.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-10 21:48:52 -05:00
Rusty Russell
2b7e5f7f5a dev_disconnect: make it more reliable.
I have seen some strange flakiness (under VALGRIND), which I have
traced down to dev-disconnect "+" not working as expected.  In
particular, the message is not sent out before closing the fd.

This seems to fix it on Linux, though it's so intermittant that it's hard
to be completely sure.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-10-10 21:48:52 -05:00
lisa neigut
6e7cfcc76e walletrpc: explicitly note we discard utx on unsuccessful broadcast
If we can't broadcast the transaction, you're going to have to start
over regardless.  It's a tal child of cmd, which we free here.
2019-10-10 05:57:45 +00:00
lisa neigut
af4ffe5fcd dev-forget-channel: accept passing in channel_id
This patch adds a channel_id parameter to allow for specifying
channels that are lacking a short_channel_id.

Useful in the case where a peer has 1) multiple channels (ONCHAIN etc)
and 2) a channel where the funding transaction hasn't been
broadcast/mined.
2019-10-10 05:57:45 +00:00
lisa neigut
cbfa045f91 tx: remove input size assertion
this is unnecessary, and actually severely limits the functionality
of `wally_tx_add_input`, which will expand the allocated input
length if there's not enough room for the additional input

```external/libwally-core/src/transaction.c
    if (tx->num_inputs >= tx->inputs_allocation_len) {
        /* Expand the inputs array */
        struct wally_tx_input *p;
        p = realloc_array(tx->inputs, tx->inputs_allocation_len,
                          tx->num_inputs + 1, sizeof(*tx->inputs));
        ...
        tx->inputs = p;
        tx->inputs_allocation_len += 1;
```
2019-10-10 05:57:45 +00:00
lisa neigut
3458ee2a6e openingd: small formatting cleanups 2019-10-10 05:57:45 +00:00
lisa neigut
c6338573d6 nits: align codes 2019-10-10 05:57:45 +00:00
lisa neigut
496d2cae5f varint: Add helper function for getting varlen size
Needed to calculate somethings for building dual funding txs
2019-10-10 05:57:45 +00:00
lisa neigut
cb2cad8c94 script: add helper for checking if a 'known type'
To be used for verifying remote inputs' validity
2019-10-10 05:57:45 +00:00
lisa neigut
6f85088b2b tools: add handy debug error 2019-10-10 05:57:45 +00:00
lisa neigut
3f1f075421 tools: add ability to wrap wire messages with ifs
Makes it possible to hide wire messages behind EXPERIMENTAL_FEATURES
flag.
2019-10-10 05:57:45 +00:00
lisa neigut
9e29a47cf8 contrib: add aliases for logs
Add aliases 'l1-log' and 'l2-log' to startup_regtest script
2019-10-10 05:57:45 +00:00
darosior
fac5faacc0 Document hsm_secret encryption 2019-10-09 22:00:38 -05:00
darosior
1c27545f9e pytest: test hsm_secret encryption 2019-10-09 22:00:38 -05:00
darosior
c9982f9f49 pytest: allow to set stdin, stdout and stderr at lightningd startup
And also allow to not wait for it to be started.
Passing stderr=subprocess.STDOUT can be useful to wait_for_log() also on
stderr messages.
2019-10-09 22:00:38 -05:00
darosior
6f9030168e lightningd: prevent hsmd from interpretating an encrypted hsm_secret as not encrypted 2019-10-09 22:00:38 -05:00
darosior
d393cda215 hsmd: encrypt hsm_secret if 'lightningd' pass an encryption key
This splits maybe_create_hsm_secret() in two parts (either encrypted
or in clear) for clarity, and adds an encryption detection in load_hsm().
There are actually three cases if an encryption key is passed:
- There is no hsm_secret => just create it and store the encrypted seed
- There is an encrypted hsm_secret => the provided key should be able to
decrypt the seed, if the wrong key is passed libsodium will nicely error
and hsmd will exit() to not throw a backtrace (using status_failed() as for
other errors) at the face of an user who mistyped its password.
- There is a non-encrypted hsm_secret => load the seed, delete the
hsm_secret, create the hsm_secret, store the encrypted seed.
2019-10-09 22:00:38 -05:00
darosior
62896566d0 lightningd: pass the hsm_secret encryption key in hsm_init 2019-10-09 22:00:38 -05:00
darosior
a698395bf0 lightningd: '--encrypted-hsm', a new startup option
Add a new startup option which will, if set, prompt the user for a
password to derive a key from. This key will later be used to encrypt
and/or decrypt `hsm_secret`.

This was made a noarg option even if it would have been preferable to
let the user the choice of how to specify the password. Since we have
to chose, better to not let the password in the commands history.
2019-10-09 22:00:38 -05:00
darosior
fe2543d8dc daemons: initialize libsodium at setup
According to the doc (https://download.libsodium.org/doc):
"sodium_init() initializes the library and should be called before
any other function provided by Sodium. [...]
the function ensures that the system's random number generator has
been properly seeded.".
2019-10-09 22:00:38 -05:00
lisa neigut
2c69ece7fe close: update short help text
Update short help text to match undeprecated close API
2019-10-09 21:04:16 -05:00
trueptolemy
9d4cccda19 pytest: Add the test for compact of deprecated close style 2019-10-09 21:04:16 -05:00
trueptolemy
bb5aa774e2 CHANGELOG: Add the entry about destination in close 2019-10-09 21:04:16 -05:00
trueptolemy
e53d065230 doc: Update the doc about destination of close 2019-10-09 21:04:16 -05:00
trueptolemy
63af8632b7 pytest: Test close to the specified destination 2019-10-09 21:04:16 -05:00
trueptolemy
019c052123 JSON-API: Allow close channel to specified address
Command format: close id [unilateraltimeout] [destination]

Close the channel with peer {id}, forcing a unilateral
close after {unilateraltimeout} seconds if non-zero, and
the to-local output will be sent to {destination}. If
{destination} isn't specified, the default is the address
of lightningd.

Also change the pylightning:
update the `close` API to support `destination` parameter
2019-10-09 21:04:16 -05:00
trueptolemy
71b606e050 lightningd: Add a new field shutdown_scriptpubkey[NUM_SIDES]
`shutdown_scriptpubkey[REMOTE]` is original remote_shutdown_scriptpubkey;
`shutdown_scriptpubkey[LOCAL]` is the script used for "to-local" output when `close`. Add the default is generated form `final_key_idx`;

Store `shutdown_scriptpubkey[LOCAL]` into wallet;
2019-10-09 21:04:16 -05:00
trueptolemy
a1204a0b94 Closed: Cleanup for wire_closing_init
`final_scriptpubkey` is repeated with `funding_pubkey[LOCAL]`.
2019-10-09 21:04:16 -05:00
trueptolemy
75e946d256 json: Move param_bitcoin_address from wallet/walletrpc.c to lightningd/json.c
It's a useful helper, and it will be used to prase address in `close` command.
2019-10-09 21:04:16 -05:00
trueptolemy
8346d5c353 josn: Move json_tok_address_scriptpubkey from lightningd/jsonrpc.c to lightningd/json.c
Its declaration is in lightningd/json.h, so we should move it to lightningd/json.c.
2019-10-09 21:04:16 -05:00
darosior
7fd2f6db6d gossipd: query_messages: fail the connection if peer says it does not have up-to-date infos
It is most likely not on the same network, and in any case not a good peer to gossip with.
2019-10-09 16:54:39 -05:00
darosior
2638947ddc gossipd: query_scid: respond with complete to 0 on wrong chain_hash 2019-10-09 16:54:39 -05:00
darosior
d3c8225968 gossipd: add a BOLT#7 comment when wrong chain_hash in 'query_channel_range'
And correct some typos
2019-10-09 16:54:39 -05:00
trueptolemy
d59015d4ca CHANGELOG: Add the entries about the new parameter name amount 2019-10-09 16:51:42 -05:00
trueptolemy
e4cd5df06c doc: Use amount as the parameter name for fundchannel 2019-10-09 16:51:42 -05:00
trueptolemy
fdb90a765e pytest: Test compact of the old-style fundchannel_start 2019-10-09 16:51:42 -05:00
trueptolemy
d149ba2f3a JSON-API: fundchannel_start uses amount fieldname to replace satoshi 2019-10-09 16:51:42 -05:00