Commit Graph

618 Commits

Author SHA1 Message Date
Rusty Russell
0baa5f7071 gossipd: send node announcement on startup.
I suspect this fixes #1660 too, but checking would be good.

Fixes: #1781
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-28 18:20:17 +02:00
Rusty Russell
cbd1d1d0f2 pytest: test that we reannounce node after restart.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-28 18:20:17 +02:00
Rusty Russell
c8c3294a23 json-invoice: add routeboost, warnings.
We split json_invoice(), as it now needs to round-trip to the gossipd,
and uniqueness checks need to happen *after* gossipd replies to avoid
a race.

For every candidate channel gossipd gives us, we check that it's in
state NORMAL (not shutting down, not still waiting for lockin), that
it's connected, and that it has capacity.  We then choose one with
probability weighted by excess capacity, so larger channels are more
likely.

As a side effect of this, we can tell if an invoice is unpayble (no
channels have sufficient incoming capacity) or difficuly (no *online*
channels have sufficient capacity), so we add those warnings.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-28 15:03:42 +02:00
Rusty Russell
9455331575 json: use bolt naming for features arrays in listnodes, listpeers.
Deprecate the old names.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-28 04:14:28 +00:00
Rusty Russell
3358437062 connectd: don't log every time a peer disconnects.
Great for a few of our tests, but generally spammy.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-28 04:14:28 +00:00
Rusty Russell
a90c200fcb pytest: wait_channel_quiescent helper to wait for resolved htlcs.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-27 22:57:19 +02:00
Rusty Russell
704f8b5b0e pytest: make test_autocleaninvoice more reliable.
We're about to slow down the invoice rpc (esp. under valgrind), which
breaks the delicate timing of the autocleaninvoice test.

Change that so the autocleaner (and timestamp) starts after the invoices
are added.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-27 22:57:19 +02:00
Christian Decker
d590302523 pytest: Fix benchmarks after the fixture migration 2018-09-27 00:18:24 +00:00
Rusty Russell
59bdba6bac pytest: don't test crashing under valgrind at all.
Travis failures:

valgrind: m_scheduler/sema.c:104 (vgModuleLocal_sema_down): Assertion 'sema->owner_lwpid != lwpid' failed.
host stacktrace:
==1296==    at 0x38083F48: ??? (in /usr/lib/valgrind/memcheck-amd64-linux)
==1296==    by 0x38084064: ??? (in /usr/lib/valgrind/memcheck-amd64-linux)
==1296==    by 0x380841F1: ??? (in /usr/lib/valgrind/memcheck-amd64-linux)
==1296==    by 0x38135DAE: ??? (in /usr/lib/valgrind/memcheck-amd64-linux)
==1296==    by 0x380D328D: ??? (in /usr/lib/valgrind/memcheck-amd64-linux)
==1296==    by 0x3809A4AC: ??? (in /usr/lib/valgrind/memcheck-amd64-linux)
==1296==    by 0x3809AE43: ??? (in /usr/lib/valgrind/memcheck-amd64-linux)
==1296==    by 0x380988CF: ??? (in /usr/lib/valgrind/memcheck-amd64-linux)
sched status:
  running_tid=0
Thread 1: status = VgTs_WaitSys (lwpid 1296)
==1296==    at 0x5729730: __poll_nocancel (syscall-template.S:84)
==1296==    by 0x4348DF: daemon_poll (daemon.c:78)
==1296==    by 0x4169E7: io_poll_lightningd (lightningd.c:543)
==1296==    by 0x471ECD: io_loop (poll.c:282)
==1296==    by 0x416E06: main (lightningd.c:744)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-26 16:32:08 -07:00
Rusty Russell
e450c6bbdb gossipd: remove time-delayed local channel_update, produce DISABLE on-demand.
We have a lot of infrastructure to delay local channel_updates to
avoid spamming on each peer reconnect; we had to keep tracking of
pending ones though, in case we needed the very latest for sending an
error when failing an HTLC.

Instead, it's far simpler to set the local_disabled flag on a channel
when we disconnect, but only send a disabling channel_update if we
actually fail an HTLC.

Note: handle_channel_update() TAKES update (due to tal_arr_dup), but we
didn't use that before.  Now we do, add annotation.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-26 03:21:35 +00:00
Rusty Russell
16e16a725e gossipd: apply private updates to announce channel.
We trade channel_update before channel_announce makes the channel
public, and currently forget them when we finally get the
channel_announce.  We should instead apply them, and not rely on
retransmission (which we remove in the next patch!).

This earlier channel_update means test_gossip_jsonrpc triggers too
early, so have that wait for node_announcement.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-26 03:21:35 +00:00
Mark Beckwith
cbde3e20f7 cli: help command now also prints usage
The help command now adds command usage to its output by calling each
command handler in CMD_USAGE mode.

Instead of seeing, for example:

	decodepay
	    Decode {bolt11}, using {description} if necessary

we see:

	decodepay bolt11 [description]
	    Decode {bolt11}, using {description} if necessary

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-09-25 15:11:45 +02:00
Rusty Russell
8455b12781 Revert "gossipd: handle premature node_announcements in the store."
This reverts commit e2f426903d.

With the new store version, this can't happen.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-21 17:56:15 +02:00
Rusty Russell
48de77d56e gossipd: invalidate old gossip_stores.
Incrementing version number means stores which were prior to the previous
commit will be removed, and refreshed.  The simplest fix, if not the most
efficient.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-21 17:56:15 +02:00
Rusty Russell
0855422110 gossip_control: when searching for a txout, make sure it's not spent!
There's no reason for the db to ever return non-NULL if it's spent.  And there's
only one caller, for which that is definitely true.

Suggested-by: @cdecker
Fixes: #1934
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-21 17:56:15 +02:00
Rusty Russell
24c386c086 test_gossip: gossip retransmit on spent UTXO test.
Aka #1934.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-21 17:56:15 +02:00
Rusty Russell
6228c25643 test_gossip: basic 'node notices close' test.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-21 17:56:15 +02:00
Rusty Russell
0925daa087 gossipwith: simple tool to snarf gossip from a node.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-21 17:56:15 +02:00
Rusty Russell
edb74260f4 pytest: more reliable onchain tests.
We extract the tx from the logs, and then we wait until that hits
the mempool.  This is more reliable than 'sendrawtx' in the logs,
which might catch a previous sendrawtx; it's also more explicit
that we expect that tx exactly.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-21 00:28:44 +00:00
lisa neigut
b1ceaf9910 gossipd: Update BOLT-split flags in channel_update
BOLT 7's been updated to split the flags field in `channel_update`
into two: `channel_flags` and `message_flags`. This changeset does the
minimal necessary to get to building with the new flags.
2018-09-21 00:24:12 +00:00
Rusty Russell
da9d92960d lightningd: accept hsmstatus_client_bad_request messages (and log!)
We currently just ignore them.  This is one reason the hsm (in some places)
explicitly calls log_broken so we get some idea.

This was the only subdaemon which had a NULL msgcb and msgname, so eliminate
those checks in subd.c.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-20 09:49:39 +02:00
Rusty Russell
e41e1a177e pytest: wait until mock is called for set_feerates.
Got a spurious failure in test_no_fee_estimate; we fired too soon from the logs (presumably
we raced in on the first response, but estimatesmartfee gets called 3 times).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-20 09:12:28 +02:00
Rusty Russell
252bbe1d2d pytest: don't wait for sendrawtx, wait for expected tx.
In particular, test_no_fee_estimate was flaky due to seeing the funding
tx being sent.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-19 13:04:01 +02:00
Christian Decker
f29f92a5fe pytest: Clean up bitcoind_cmd_override, it's no longer used 2018-09-16 00:05:34 +02:00
Christian Decker
9e5d7dacb0 pytest: Use the mock bitcoind everywhere 2018-09-16 00:05:34 +02:00
Christian Decker
16869e3fe6 pytest: Use the bitcoind proxy to mock feerates 2018-09-16 00:05:34 +02:00
Christian Decker
aa80a330f1 pytest: Remove auto-proxying in favor of a per-node btc proxy 2018-09-16 00:05:34 +02:00
Christian Decker
2dabc5af93 pytest: Set correct header in mock bitcoind 2018-09-16 00:05:34 +02:00
Christian Decker
74f228deb8 btcproxy: Unpack batched JSON-RPC calls and issue them separately 2018-09-16 00:05:34 +02:00
Christian Decker
88186020e0 pytest: Implement method mocking for ProxiedBitcoinRpc 2018-09-16 00:05:34 +02:00
Christian Decker
e132dffa0b pytest: Add an RPC proxy inbetween bitcoind and bitcoin-cli
This is a simple reverse proxy that `bitcoin-cli` can talk to when invoked by
`lightningd`. It allows us to trace `bitcoin-cli` calls, and intercept calls to
mock the replies, better than the current bash-script based method.
2018-09-16 00:05:34 +02:00
Christian Decker
f505a9418b pytest: Fix lint error 2018-09-16 00:05:34 +02:00
Christian Decker
f1e931f7bb pytest: Fix flaky test_logging
File was rotated away but didn't wait for the first line to be actually written.
2018-09-14 21:19:50 +02:00
Christian Decker
79da1b9aa2 pytest: Keep the test directory even if the failure is in the fixtures 2018-09-14 21:19:50 +02:00
Rusty Russell
cefb6925b2 db: save and restore last_sent_commit correctly.
It's an array: we were only saving the single element; if there was more than
one changed HTLC we'd get a bad signature!

The report in #1907 is probably caused by the other side re-requesting
something we considered already finalized; to avoid this particular error,
we should set the field to NULL if there's no last_sent_commit.

I'm increasingly of the opinion we want to just save all the update
packets to the db and blast them out, instead of doing this
second-guessing dance.

Fixes: #1907
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-04 14:43:27 +02:00
Rusty Russell
db12a1452f pytest: reproduce problem with restarting and retransmitting multiple outgoing htlcs
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-04 14:43:27 +02:00
Rusty Russell
e2f426903d gossipd: handle premature node_announcements in the store.
These happen after we compact the store; every log I've seen of a
restart on a real node has a message about truncating the store,
because node_announcements predate channel_announcements.

I extracted one such case from testnet, and reduced it to test here.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-04 14:36:05 +02:00
Rusty Russell
312209ad60 pytest: support simple subdaemon debugging.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-04 14:36:05 +02:00
Rusty Russell
7e1fd9c38a pytest: make test_no_fee_estimate more reliable.
1. Wait for a 'sendrawtransaction' *after* the dev-fail message; don't be
   fooled by a previous one.
2. Turning on estimate fee sets fees exactly; just wait for it to be processed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-30 16:33:35 +02:00
Rusty Russell
33c6285787 feerates: turn it into a simple query API, remove setting.
It's probably unnecessary to have this weird way of injecting results
now we have explicit feerate args.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-30 16:33:35 +02:00
Rusty Russell
db3c387264 feerate: allow names 'urgent' 'normal' and 'slow'.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-30 16:33:35 +02:00
Rusty Russell
e0952ceff2 feerate: use suffix, not separate argument.
And, reluctantly, default to bitcoind style.
"It's wrong to be right too soon."

Suggested-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-30 16:33:35 +02:00
Rusty Russell
14dc1c37ab fundchannel / withdraw: allow explicit feerate setting.
These are the two cases where we'll refuse without a fee estimate.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-30 16:33:35 +02:00
Rusty Russell
e2d4b7cc8d cleanup: extract and formalize feerate conversion.
I didn't want to create a new file for this now, as that would totally
break #1880.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-30 16:33:35 +02:00
Rusty Russell
af4fa9a359 feerates: rename sipa/bitcoind to perkw/perkb.
Suggested-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-25 00:33:12 +00:00
Rusty Russell
a4b952ebc7 feerate: include rough estimates of actual tx costs.
We could refine this later (based on existing wallet, for example), but
this gives some estimate.

[ Rename onchain_estimates -> onchain_fee_estimates Suggested-by: @SimonVrouwe ]
[ Factor of 1000 fix Reported-by: @SimonVrouwe ]
Suggested-by: @molxyz
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-25 00:33:12 +00:00
Rusty Russell
14294642d2 feerates: consider last three raw values for min/max.
We don't know what our peer is doing, but if we see those values, maybe
they did too, and for longer.  And add the min/max acceptable values
into our JSON API.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-25 00:33:12 +00:00
Rusty Russell
c7c5affa3f feerates: new command to inject/query fee estimates.
This is useful mainly in the case where bitcoind is not giving estimates,
but can also be used to bias results if you want.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-25 00:33:12 +00:00
Rusty Russell
9d37b78088 cleanup: lowercase name of feerates, immediate -> urgent.
This is only used for logging now, but it gets more important as it
enters the RPC API.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-25 00:33:12 +00:00
Rusty Russell
9d517ddc1d options: remove default-fee-rate now we don't use it.
And no more filtering out messages, as we should no longer spam the
logs with them (the 'Connected json input' one was removed some time
ago).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-25 00:33:12 +00:00