Commit Graph

3056 Commits

Author SHA1 Message Date
Rusty Russell
37373f2c16 wallet: provide better comments on wallet_first_blocknum.
ZmnSCPxj queried the unilateral close case, so make that clearer.
Christian raise concerns about existing channels, so make it clear
what we're doing there too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-16 13:07:12 +01:00
Rusty Russell
ef9c6cb8c3 wallet: don't scan from worst-case start on first use.
We only need to do that if it's possible there's something to find:
either we have an unspent output from a unilateral close, or we've
ever handed out an address.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-16 13:07:12 +01:00
Rusty Russell
df4c669a60 test: test for funds sent while we were offline.
As described by @lvaccaro in #990.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-16 13:07:12 +01:00
Rusty Russell
6620305606 wallet: use last_processed_block to determine scan start.
With fallback depending on chainparams: this means the first upgrade
will be slow, but after that it'll be fast.

Fixes: #990
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-16 13:07:12 +01:00
Rusty Russell
21849329dd wallet: store last block number we searched for UTXOs.
We already go back 100 from this in case of reorgs, so the block number
itself is sufficient.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-16 13:07:12 +01:00
Rusty Russell
256bdc12ff lightningd: activate crashlog later.
We error out for all kinds of reasons early on (eg. bitcoind down),
and printing a backtrace for them is pretty confusing.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-16 13:02:41 +01:00
Rusty Russell
eb17d6af71 lightningd: implement --daemon.
Includes closing off stdout and stderr.  We don't do it directly in the
arg parser, as we want to interact normally (eg with other errors) before
we turn off stdout/stderr.

Fixes: #986
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-16 13:02:41 +01:00
Rusty Russell
65593d4a98 chaintopology: don't start fee estimation loop until we're fully active.
This interacts badly with --daemon (next patch) which then tries to
reap a child it didn't create, which took me a couple of hours to
figure out.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-16 13:02:41 +01:00
Rusty Russell
c5cc5c5c1b ccan: import ccan/daemonize.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-16 13:02:41 +01:00
Rusty Russell
ccd0e5db54 ccan: update so we get exposed path constants.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-16 13:02:41 +01:00
Rusty Russell
c8f9ea7bf7 fixup: test_multirpc does not terminate.
We need to set expiry, otherwise waitinvoice would take 1 hr, and we
can't read once for every cmd, since each read may consume more than
a single result, and we block.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-16 12:56:48 +01:00
Christian Decker
cab7e3e43a pytest: Add failing test with JSON-RPC requests crossing each other
Suggested-By: @ZmnSCPxj
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-02-16 12:56:48 +01:00
Rusty Russell
a08bcfdbd3 jsonrpc: don't crash on multiple commands at once.
Once we read a command, we are supposed to io_wait until it finishes.
However, we are actually woken in two places: when it's complete
(which is correct), and when it's written out (which is wrong).

We don't care when it's written out, only when it's finished:
refactor to make json_done() free and NULL the old ->current,
rather than have the callers do it.  Now it's clear that it's
ready for both new output and new input.

Fixes: #934
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-16 12:56:48 +01:00
Douglas Schilling Landgraf
cb1a4a5180 pylightning: make flake8 happy
Trivial changes to make the code pythonic

Signed-off-by: Douglas Schilling Landgraf <dougsland@gmail.com>
2018-02-16 12:55:04 +01:00
Christian Decker
e48a97ffd4 pylightning: Filter out None arguments in JSON-RPC calls
Passing them in doesn't hurt, but better to avoid ambivalences.

Suggested-by: @graphite
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-02-15 23:56:38 +00:00
Christian Decker
db5c0aa6c1 pytest: Make pep8 happy(er)
A number of fixes to make our code more pythonic
Signed-off-by: Christian Decker <decker.christian@gmail.com>
Co-authored-by: Guido Dassori @gdassori
2018-02-15 23:56:38 +00:00
Christian Decker
0e40b9b08c pylightning: Switch to dict-based params instead of positional
Signed-off-by: Christian Decker <decker.christian@gmail.com>
Co-authored-by: Guido Dassori @gdassori
2018-02-15 23:56:38 +00:00
Christian Decker
ed2fee8977 pylightning: Added explicit logger to log to
Signed-off-by: Christian Decker <decker.christian@gmail.com>
Co-authored-by: Guido Dassori @gdassori
2018-02-15 23:56:38 +00:00
ZmnSCPxj
33b25b6a9b travis: Correctly process $SOURCE_CHECK_ONLY.
The `!` outside of parentheses was being cut by travis.
2018-02-15 08:48:35 +00:00
Luca Vaccaro
4dac2da8fc Fix litecoin mainnet & testnet chainparams 2018-02-14 15:26:27 +01:00
Rusty Russell
8f7a19d1a3 onchain: handle case where multiple HTLCs exist for same payment_hash.
We will have probably failed the others, but either way, don't try to
fulfill an HTLC we've already failed.

Fixes: #394
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-14 11:57:42 +01:00
Rusty Russell
55d962046b Rename (almost) all destructors to destroy_<type>.
We usually did this, but sometimes they were named after what they did,
rather than what they cleaned up.

There are still a few exceptions:
1. I didn't bother creating destroy_xxx wrappers for htable routines
   which already existed.
2. Sometimes destructors really are used for side-effects (eg. to simply
   mark that something was freed): these are clearer with boutique names.
3. Generally destructors are static, but they don't need to be: in some
   cases we attach a destructor then remove it later, or only attach
   to *some* cases.  These are best with qualifiers in the destroy_<type>
   name.

Suggested-by: @ZmnSCPxj
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-14 11:31:58 +01:00
Rusty Russell
6a3ccafaf9 wallet: don't implicitly remove peers, but do it explicitly.
This provides a sanity check that we are in sync, and also keeps the
logic in the program and out of the SQL.

Since the destructor now doesn't clean up the peer, there are some
wider changes to be made when cleaning up.  Most notably we create
lots of channels in run-wallet.c and they previously freed the peer:
now we need free the peer explicitly, so we need to free them first.

Suggested-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-14 11:31:58 +01:00
Rusty Russell
98de10b842 channel: rename free_channel to delete_channel.
free_channel() sounds like a destructor.

Suggested-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-14 11:31:58 +01:00
Rusty Russell
d822ba1eee lightningd: allow a new channel open from peer if no *active* channels.
And return the correct error message for the channel they give, if
they try to re-establish on an error channel.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-14 11:31:58 +01:00
Rusty Russell
8f48a72d74 lightningd: remove peer->log in favor of channel->log.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-14 11:31:58 +01:00
Rusty Russell
e20fff9340 lightningd: remove almost all other peer2channel / channel2peer shims.
This final sweep only keepl peer2channel within peer_control.c for
the reconnect case.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-14 11:31:58 +01:00
Rusty Russell
0e93fb932a lightningd: bitcoind and topology routines take channel, not peer.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-14 11:31:58 +01:00
Rusty Russell
329e31bbe7 lightningd/peer_htlcs: remove remaining peer_ shims.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-14 11:31:58 +01:00
Rusty Russell
cf7c399cc5 htlc: keep channel pointer, not peer pointer.
And move the no-remaining-htlcs check from the peer destructor to the
channel destructor.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-14 11:31:58 +01:00
Rusty Russell
409fef582d subd: keep pointer to channel, not peer.
This rolls through many other functions, making them take channel not peer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-14 11:31:58 +01:00
Rusty Russell
b7680412e3 lightningd: rename peer_fail functions to channel_fail.
And move them into channel.c.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-14 11:31:58 +01:00
Rusty Russell
8c084d57ff lightningd: channels own the peer.
Channels are within the peer structure, but the peer is freed only
when the last channel is freed.

We also implement channel_set_owner() and make peer_set_owner() a temporary
wrapper.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-14 11:31:58 +01:00
Rusty Russell
32411de90e lightningd: split struct peer into struct peer and struct channel.
Much like the database; peer contains id, address, channel contains
per-channel information.  Where we create a channel, we always create
the peer too.

For the moment, peer->log and channel->log coexist side-by-side, to
reduce some of the churn.

Note that this changes the API to dev-forget-channel: if we have more
than one channel, we insist they specify the short-channel-id.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-14 11:31:58 +01:00
Rusty Russell
6b71654351 lightningd: create new structure channel to hold per-channel info.
This is not connected yet; during the transition, there will be a 1:1
mapping from channel to peer, so we can use channel2peer and peer2channel
to shim between them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-14 11:31:58 +01:00
Rusty Russell
65c09c895d wallet: properly handle case where peer has no address when saving channel.
In practice, it currently always does, so we've never hit an error.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-14 11:31:58 +01:00
Rusty Russell
81ca1db347 wallet: delete peers with no channels.
ON DELETE CASCADE goes the other way: we should clean up peers with no
channels from db.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-14 11:31:58 +01:00
Rusty Russell
38a313af0d wallet: delete channels in state OPENINGD.
Both when we forget about an opening peer, and at startup.  We're
going to be relying on this, and the next patch, as we refactor
peer/channel handling to mirror the db.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-14 11:31:58 +01:00
Rusty Russell
15eaf56d79 wallet: add ld pointer.
This will be required to give it direct access to the ld->peers list.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-14 11:31:58 +01:00
Rusty Russell
fcffbd0f20 channeld: rename new_channel to new_full_channel.
This avoids clashing with the new_channel we're about to add to lightningd,
and also matches its counterpart new_initial_channel.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-14 11:31:58 +01:00
Rusty Russell
7eea02e846 db: don't allow newer db versions.
Clearly we could do more damage if we continue.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-14 11:31:58 +01:00
Rusty Russell
26617d1d64 gossipd/test: update mocks.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-14 11:31:58 +01:00
practicalswift
e97ee3d14e Fix typo in error message (funding_signed ids don't match) 2018-02-13 19:59:05 +01:00
Rusty Russell
e76a0b4ddc gossipd: fix race where we can handoff peer with bad cryptostate.
DEBUG:root:lightningd(16333): 2018-02-08T02:12:21.158Z lightningd(8262): lightning_openingd(0382ce59ebf18be7d84677c2e35f23294b9992ceca95491fcf8a56c6cb2d9de199): Failed hdr decrypt with rn=2

We only hand off the peer if we've not started writing, but that was
insufficient: we increment the sn twice on encrypting packet, so there's
a window before we've actually started writing where this is now
wrong.

The simplest fix is only to hand off from master when we've just written,
and have the read-packet path simply wake the write-packet path.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-13 12:10:35 +01:00
Rusty Russell
fd04fca771 jsonrpc: make lifetimes in error path clearer.
Saving one allocation isn't worth the confusion.

Reported-by: @ZmnSCPxj
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-13 12:01:27 +01:00
Rusty Russell
dec86e0115 jsonrpc: split error and success cases, use json_add_string_escape.
Combining the two was just awkward, so it's clearer to have separate
functions.  And we make the lower-level functions do the escaping.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-13 12:01:27 +01:00
Christian Decker
b4ce4d228d JSON-RPC: Fix unquoted error string when parser fails
Fixes #963

Reported-by: @shesek
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-02-13 12:01:27 +01:00
practicalswift
fe670b9aaa Improve log message when receiving unknown message type 2018-02-12 09:31:00 +01:00
ZmnSCPxj
4e382ebd94 doc: Mention keyword arguments, update pay to mention use of null. 2018-02-12 09:30:16 +01:00
Rusty Russell
6757300a0e Travis: slightly more finegrained, should be under 17 minutes now.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-12 09:27:49 +01:00