Commit Graph

4563 Commits

Author SHA1 Message Date
lisa neigut
3ce53ab9ed tools/gen-wire: 3th -> 3rd
Small grammar fix in wire gen tool
2018-09-22 19:06:06 +02:00
Rusty Russell
8455b12781 Revert "gossipd: handle premature node_announcements in the store."
This reverts commit e2f426903d.

With the new store version, this can't happen.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-21 17:56:15 +02:00
Rusty Russell
48de77d56e gossipd: invalidate old gossip_stores.
Incrementing version number means stores which were prior to the previous
commit will be removed, and refreshed.  The simplest fix, if not the most
efficient.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-21 17:56:15 +02:00
Rusty Russell
0855422110 gossip_control: when searching for a txout, make sure it's not spent!
There's no reason for the db to ever return non-NULL if it's spent.  And there's
only one caller, for which that is definitely true.

Suggested-by: @cdecker
Fixes: #1934
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-21 17:56:15 +02:00
Rusty Russell
24c386c086 test_gossip: gossip retransmit on spent UTXO test.
Aka #1934.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-21 17:56:15 +02:00
Rusty Russell
6228c25643 test_gossip: basic 'node notices close' test.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-21 17:56:15 +02:00
Rusty Russell
d16c3dcdc7 devtools/decodemsg: take series of msgs from stdin.
Useful in combination with gossipwith.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-21 17:56:15 +02:00
Rusty Russell
47f5bc4deb gossipwith: add ability to send message.
Just cmdline for now, rather than a proper stdin io loop.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-21 17:56:15 +02:00
Rusty Russell
0925daa087 gossipwith: simple tool to snarf gossip from a node.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-21 17:56:15 +02:00
Rusty Russell
1b80cb4269 Makefile: don't rebuild all the time
The code to regenerate the local BOLT copy was causing eternal rebuild.

So only build if it's wrong.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-21 17:55:51 +02:00
Rusty Russell
edb74260f4 pytest: more reliable onchain tests.
We extract the tx from the logs, and then we wait until that hits
the mempool.  This is more reliable than 'sendrawtx' in the logs,
which might catch a previous sendrawtx; it's also more explicit
that we expect that tx exactly.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-21 00:28:44 +00:00
lisa neigut
b1ceaf9910 gossipd: Update BOLT-split flags in channel_update
BOLT 7's been updated to split the flags field in `channel_update`
into two: `channel_flags` and `message_flags`. This changeset does the
minimal necessary to get to building with the new flags.
2018-09-21 00:24:12 +00:00
lisa neigut
b1f15c2605 BOLT updates: broken link fixes
See a9195a84d0
2018-09-21 00:24:12 +00:00
lisa neigut
73ea6d0038 BOLT 2 updates for fix placment of chain_hash req
See 4b62d26af9
2018-09-21 00:24:12 +00:00
lisa neigut
b287f2f007 BOLT 11 human-readable formatting changes 2018-09-21 00:24:12 +00:00
lisa neigut
2327f04b54 HACKING.md: Update rec'd test cmd to full-check
Otherwise you won't run `check-source` and your Travis build will
fail on you.

Also remove comment about how it's currently disabled cuz that's a lie.
2018-09-21 00:21:20 +00:00
Rusty Russell
5b2e829b4f hsmd: remove master pointer.
We used to use it to complain about bad requests, but we use the status conn
now, so it's unused except for tests and asserts.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-20 09:49:39 +02:00
Rusty Russell
019bc4fcd7 hsmd: reorder functions (MOVEONLY).
We don't need to pre-declare any more, but I left it in the previous
patch for review simplicity.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-20 09:49:39 +02:00
Rusty Russell
b0769d9c0c hsmd: don't use daemon_conn for clients.
It offers them a DoS vector, if they don't read the replies.  We really want
to use raw ccan/io so we can avoid buffering for this.

It makes the handing of fds for new clients a bit more complex
(callback based), but it's not too bad.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-20 09:49:39 +02:00
Rusty Russell
573f2f065a hsmd: document as part II of our journey.
Thanks greatly to the four people who I *know* have read this:

	@wythe, @ZmnSCPxj, @SimonVrouwe, and @cdecker

Your feedback will help future developers seeking enlightenment!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-20 09:49:39 +02:00
Rusty Russell
04c77f4853 lightningd: use hsm_get_client_fd() helper for global daemons too.
We couldn't use it before because it asserted dbid was non-zero.  Remove
assert and save some code.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>



Header from folded patch 'fixup!_lightningd__use_hsm_get_client_fd()_helper_for_global_daemons_too.patch':

fixup! lightningd: use hsm_get_client_fd() helper for global daemons too.

Suggested-by: @ZmnSCPxj
2018-09-20 09:49:39 +02:00
Rusty Russell
cc48e794c9 hsmd: extract and use common sign_all_inputs() helper.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-20 09:49:39 +02:00
Rusty Russell
e012e94ab2 hsmd: rename hsm_client_wire_csv to hsm_wire.csv
That matches the other CSV names (HSM was the first, so it was written
before the pattern emerged).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-20 09:49:39 +02:00
Rusty Russell
8f1f1784b3 hsmd: remove hsmd/client.c
It was only used by handshake.c.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-20 09:49:39 +02:00
Rusty Russell
f8df069536 hsmd: move HTLC TX signing next to commitment TX signing.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-20 09:49:39 +02:00
Rusty Russell
6b6b7eac61 hsmd: use status_conn to send bad_request messages, make handlers uniform.
The current code sends hsmstatus_client_bad_request via the req fd;
this won't work, since lightningd uses that synchronously and only
expects a reply to its commands.  So send it via status_conn.

We also enhance hsmstatus_client_bad_request to include details, and
create convenience functions for it.  Our previous handling was ad-hoc;
we sometimes just closed on the client without telling lightningd,
and sometimes we didn't tell lightningd *which* client was broken.

Also make every handler the exact same prototype, so they now use the
exact same patterns (hsmd *only* handles requests, makes replies).

I tested this manually by corrupting a request to hsmd.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-20 09:49:39 +02:00
Rusty Russell
da9d92960d lightningd: accept hsmstatus_client_bad_request messages (and log!)
We currently just ignore them.  This is one reason the hsm (in some places)
explicitly calls log_broken so we get some idea.

This was the only subdaemon which had a NULL msgcb and msgname, so eliminate
those checks in subd.c.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-20 09:49:39 +02:00
Rusty Russell
1e4e476c9e hsmd: implement bitcoin_key() to subsume bitcoin_pubkey and bitcoin_keypair.
This mirrors the node_key() interface we already have.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-20 09:49:39 +02:00
Rusty Russell
3e63d88ad1 hsmd: rename per-peer to per-channel.
And remove cut&paste of derive_peer_seed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-20 09:49:39 +02:00
Rusty Russell
72e7856bf3 hsmd: reorder functions (MOVEONLY).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-20 09:49:39 +02:00
Rusty Russell
e41e1a177e pytest: wait until mock is called for set_feerates.
Got a spurious failure in test_no_fee_estimate; we fired too soon from the logs (presumably
we raced in on the first response, but estimatesmartfee gets called 3 times).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-20 09:12:28 +02:00
Saibato
d5c3626fa7 parse autotor: address before separate_address_and_port
this enables addr like --addr=autotor:127.0.0.1 or
--addr=autotor:localhost to just use the default tor service port

Signed-off-by: Saibato <Saibato.naga@pm.me>
2018-09-20 09:09:11 +02:00
Rusty Russell
4198cb34a2 CHANGELOG.md: catchup with changes so far.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-20 05:24:40 +00:00
Simon Vrouwe
94e42f2384 openingd: prioritize incoming peer traffic over handling (and sending out) gossip
- reduces probability for a deadlock where we block on sending data because
  the other peer cannot receive because it blocks on sending data etc.
- when either side sends so much data that it fills up the kernel/network buffer
- however sending out gossip can still block when (malicious) peer never receives
2018-09-20 03:28:42 +00:00
Christian Decker
674d176087 doc: Update docs to remove 100% upper bound
Actual change is in the previous commit.
2018-09-20 02:41:28 +00:00
Christian Decker
cf52b7161f json-rpc: Remove upper limit for percentage
The `json_tok_percentage` parser is used for the `fuzzpercent` in `getroute` and
`maxfeepercent` in `pay`. In both cases it seems reasonable to allow values
larger than 100%. This has bitten users in the past when they transferred single
satoshis to things like satoshis.place over a route longer than 2 hops.
2018-09-20 02:41:28 +00:00
Saibato
60b51d29b6 set the tor port also in struct wireaddr
fixes also #1939

Signed-off-by: Saibato <Saibato.naga@pm.me>
2018-09-19 13:48:05 +02:00
Rusty Russell
f6fb120e4a lightningd: allow more than one bitcoind request at once, run multiple queues.
With the previous patch, we could still get stuck behind a low-prio
request.  Generalize it into separate queues, and allow more than one
request in parallel.

Worth noting that the test time for `VALGRIND=0 pytest -vx tests/ -n 10`
doesn't change measurably.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-19 13:21:21 +02:00
Rusty Russell
e7a0ffca05 lightningd: verbose debugging for bitcoind commands.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-19 13:21:21 +02:00
Rusty Russell
9b8c8f652b lightningd: make bcli_args() helper take ctx.
Otherwise we can get leak complaints: all callers now use tmpctx.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-19 13:21:21 +02:00
Rusty Russell
2cdc5fb964 lightningd: make some bitcoind requests high priority.
fiatjaf has a cheap VPS, connecting remotely to his home bitcoind node.
    fiatjaf's latency on bitcoin-cli getblock is between 10 and 37 seconds.
    fiatjaf's c-lightning node is getting one block per hour.
    fiatjaf is sad.

We single-file our bitcoind requests, because bitcoind has a limited
thread pool and it *fails* rather than queueing if you upset it.  We
probably be fine using separate queues for each command type, but simply
allowing some requests to cut in line should prove my theory that we're
getting stuck behind gossip verification requests.

    fiatjaf now gets one block per 2 minutes.
    fiatjaf is less sad.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-19 13:21:21 +02:00
Rusty Russell
252bbe1d2d pytest: don't wait for sendrawtx, wait for expected tx.
In particular, test_no_fee_estimate was flaky due to seeing the funding
tx being sent.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-19 13:04:01 +02:00
Rusty Russell
7744c41521 listpeers: add 'scratch_txid' for the tx we would broadcast if necessary.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-19 13:04:01 +02:00
Rusty Russell
5f059ef3fe CHANGELOG.md: add Unreleased section at the top.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-19 13:04:01 +02:00
Christian Decker
36eab5de26 pytest: Disable early abort if we run in parallel 2018-09-16 00:05:34 +02:00
Christian Decker
f29f92a5fe pytest: Clean up bitcoind_cmd_override, it's no longer used 2018-09-16 00:05:34 +02:00
Christian Decker
9e5d7dacb0 pytest: Use the mock bitcoind everywhere 2018-09-16 00:05:34 +02:00
Christian Decker
16869e3fe6 pytest: Use the bitcoind proxy to mock feerates 2018-09-16 00:05:34 +02:00
Christian Decker
aa80a330f1 pytest: Remove auto-proxying in favor of a per-node btc proxy 2018-09-16 00:05:34 +02:00
Christian Decker
2dabc5af93 pytest: Set correct header in mock bitcoind 2018-09-16 00:05:34 +02:00