Commit Graph

86 Commits

Author SHA1 Message Date
Simon Vrouwe
a235c1fa67 add more detail to log messages about feerate estimates 2018-07-29 20:12:21 +02:00
Simon Vrouwe
a80622edab at startup initialize smoothed feerate to polled feerate
fix indentation
2018-07-26 19:08:13 +02:00
Mark Beckwith
b876c601a6 Modern param style for chaintopology.c, ...
connect_control.c, dev_ping.c, gossip_control.c, invoice.c.

This converts about 50% of all calls of `json_get_params` to `param`.

After trying (and failing) to squash and rebase #1682 I just made a new branch
from a patch file and closed #1682.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-07-17 14:33:14 +02:00
Hiroki Gondo
070aa08709 fix: compile error with clang 2018-07-17 05:08:29 +00:00
SimonVrouwe
f2ffb6d03e improves exponential smoothing of feerate estimates (#1699)
- fixes problem with polling interval > 150 * 0.9
- fixes log message 'feerate hit floor' at every feerate change
- smoothed fee now reaches 90% of (exp weighted) fee estimates polled in last
120s, independent of polling interval
- only apply smoothing when effect > 10 percent so it doesn't correct forever
- fix indentation
2018-07-15 18:30:43 +02:00
Rusty Russell
fed5a117e7 Update ccan/structeq.
structeq() is too dangerous: if a structure has padding, it can fail
silently.

The new ccan/structeq instead provides a macro to define foo_eq(),
which does the right thing in case of padding (which none of our
structures currently have anyway).

Upgrade ccan, and use it everywhere.  Except run-peer-wire.c, which
is only testing code and can use raw memcmp(): valgrind will tell us
if padding exists.

Interestingly, we still declared short_channel_id_eq, even though
we didn't define it any more!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-04 23:57:00 +02:00
Christian Decker
582ea1a33b jsonrpc: Remove dev-blockheight in favor of getinfo
`getinfo` has been providing the blockheight for a good while and doesn't
require the `DEVELOPER=1` flag during compilation, so it should be the preferred
method to retrieve the blockchain height.
2018-07-04 00:08:14 +00:00
Christian Decker
fe405f49be bitcoind: Smooth fee changes over a number of estimates
Implements an EWMA for the fee estimation. Achieves 90% influence of the newer
fee after 5 minutes, and adjusts to the polling rate that is configured.
2018-07-02 01:41:42 +00:00
Rusty Russell
0e6c0dbba2 bitcoin: expose feerate_floor.
Onchaind will want it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-21 13:43:32 +02:00
Christian Decker
37327d31de topo: Remove obsolete FIXME marker
This was addressed in bdb87aa994

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-09 13:39:27 +02:00
Christian Decker
2415f48723 topo: Tell chain_topology about the min and max block height
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-06 03:30:02 +00:00
Christian Decker
0d4b7eaa2c topo: Have chain_topology track both min and max block heights
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-06-06 03:30:02 +00:00
Mark Beckwith
7f437715d5 Added error code parameter to command_fail
Until now, `command_fail()` reported an error code of -1 for all uses.
This PR adds an `int code` parameter to `command_fail()`, requiring the
caller to explicitly include the error code.

This is part of #1464.

The majority of the calls are used during parameter validation and
their error code is now JSONRPC2_INVALID_PARAMS.

The rest of the calls report an error code of LIGHTNINGD, which I defined to
-1 in `jsonrpc_errors.h`.  The intention here is that as we improve our error
reporting, all occurenaces of LIGHTNINGD will go away and we can eventually
remove it.

I also converted calls to `command_fail_detailed()` that took a `NULL` `data`
parameter to use the new `command_fail()`.

The only difference from an end user perspecive is that bad input errors that
used to be -1 will now be -32602 (JSONRPC2_INVALID_PARAMS).
2018-05-26 12:17:36 +02:00
ZmnSCPxj
097a8e72d1 channel_control: Forget if unconfirmed for a long time and we are fundee.
We should forget this as it is a potential DoS if we remember every
funding txid that an attacker gave in a `funding_created` but never
broadcasted.
2018-05-23 14:37:32 -07:00
Rusty Russell
0aa22741df option cleanup: --dev-override-fee-rates
Make --override-fee-rates a dev option.  We use default-fee-rate in
its place, which (since bitcoind won't give fee estimates in regtest
mode for short chains) gives an effective feerate of 15000/7500/3750.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-20 02:32:42 +00:00
Rusty Russell
ceb1ce0ef5 chaintopology: fix default feerate.
We never hit the guess_feerate() path, because we turned a 0 ("can't
estimate fee") into 253.

This also revealed that we weren't initializing topo->feerate, and
that we were giving spurious updates even if we were using override-fee-rates.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-20 02:32:42 +00:00
Rusty Russell
c8cc8fd83f option cleanup: --dev-bitcoind-poll
Make --bitcoind-poll a dev option.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-20 02:32:42 +00:00
Rusty Russell
c6af2a8cb2 lightningd: loosen feerate minimum.
We're getting spurious closures, even on mainnet.  Using --ignore-fee-limits
is dangerous; it's slightly less so to lower the minimum (which is the
usual cause of problems).

So let's halve it, but beware the floor.

This is a workaround, until we get independent feerates in the spec.

Fixes: #613
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-01 18:45:04 +02:00
ZmnSCPxj
d5a67ec87a chaintopology: Protect against underflow when computing first_blocknum.
Fixes: #1423

(Hopefully)

Reported-by: @NicolasDorier
2018-04-26 11:40:43 +00:00
Christian Decker
96352858d6 chaintopology: Simplify rescan offset computation
Simplification of the offset calculation to use the rescan parameter, and rename
of `wallet_first_blocknum`. We now use either relative rescan from our last
known location, or absolute if a negative rescan was given. It's all handled in
a single location (except the case in which the blockcount is below our
precomputed offset), so this should reduce surprises.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-25 14:33:38 +02:00
Christian Decker
244d4e49e1 onchaind: Store channeltxs so we can restore later
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-25 14:33:38 +02:00
Christian Decker
f27cd3e43f topo: Remove in-memory txs from the block struct
The only use for these was to compute their txids so we could notify depth
in case of reorgs.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-13 00:04:37 +02:00
Christian Decker
23984ecde4 chaintopology: Use the DB to locate transactions and rebroadcast txs
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-13 00:04:37 +02:00
Christian Decker
86b6402e5c chaintopology: Refactor get_tx_depth to use the DB backed tx store
We are slowly hollowing out the in-memory blockchain representation to make
restarts easier.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-13 00:04:37 +02:00
Christian Decker
50600ae241 wallet: Store transactions we are watching, broadcast or own
This will later allow us to determine the transaction confirmation count, and
recover transactions for rebroadcasts.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-13 00:04:37 +02:00
Christian Decker
9ab28d1659 topology: Two off-by-one error when catching up with the blockchain
There are two very hard problems in software engineering:

 1. Off-by-one errors

In this case we were rolling back further than needed and we were starting the
catchup one block further than expected.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-06 06:55:41 +00:00
Christian Decker
633f893ec4 gossip: Add function to notify gossipd about an outpoint spend
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-30 16:35:00 +02:00
Christian Decker
a8d587c418 wallet: Return any eventual outpoint scid when marking it spent
Just return the short_channel_id matching the outpoint that we just marked as
spent.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-30 16:35:00 +02:00
Christian Decker
ba7341ec87 cleanup: Make blockheights unsigned
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-27 23:17:17 +00:00
practicalswift
98f49c0837 Remove include in file foo.c that is already included in foo.h 2018-03-25 23:54:21 +00:00
Rusty Russell
8966a175ee chaintopology: add debug message when we're adjusting due to feerate floor.
But only if we're actually going to change the feerate, otherwise we'd
log every time.

Suggested-by: @ZmnSCPxj
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-20 16:08:45 +01:00
Rusty Russell
2e687b9b35 chaintopology: set a fee floor to avoid creating unrelayable txs.
Naively, this would be 250 satoshi per sipa, but it's not since bitcoind's
fee calculation was not rewritten to deal with weight, but instead bolted
on using vbytes.

The resulting calculations made me cry; I dried my tears on the thorns
of BUILD_ASSERT (I know that makes no sense, but bear with me here as I'm
trying not to swear at my bitcoind colleagues right now).

Fixes: #1194
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-20 16:08:45 +01:00
ZmnSCPxj
7bbc1948f7 chaintopology: Add missing block_map_del.
We would `block_map_add` inside `add_tip`, but we never
`block_map_del` inside `remove_tip`, which is dangerous as
we actually `tal_free` the block inside `remove_tip`.

Our CI did not reliably trap this problem since block
hashes are random and rerunning the `test_blockchaintrack`
often passed spuriously.
2018-03-16 01:45:08 +00:00
Rusty Russell
de5d84097e lightningd: remove ltmp context now we have tmpctx.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-16 00:16:10 +00:00
Rusty Russell
9d9c1c3db6 wallet: wrap missing last_processed_block handling in COMPAT_V052.
Our testing also reveals a bug: we start lightningd and shut it down
before fully processing the blockchain, so we don't set
last_processed_block.  Fix that by setting it immediately once we have
a block: worst case it goes backwards a little.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-07 18:55:51 +01:00
Christian Decker
e44ea0b363 topology: Add new P2WSH outpoints to the wallet utxoset
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Christian Decker
ae30942e3f chaintopo: Record outpoint spends for owned outputs
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Christian Decker
227dc36146 utxo: Add blockheight and spendheight to outputs to track state
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
Christian Decker
1aa21498c3 topology: Track blockchain changes in the DB
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-03-06 18:59:27 +01:00
practicalswift
0046ace318 Remove unused parameter topo in txowatch_fire(struct chain_topology *topo, ...) 2018-02-22 10:46:30 +01:00
practicalswift
f15868a94d Remove unused function json_dev_broadcast(...) 2018-02-22 09:45:22 +00:00
practicalswift
91a9c2923f Mark intentionally unused parameters as such (with "UNUSED") 2018-02-22 01:09:12 +00:00
practicalswift
3dbace3421 Remove redundant casts to same type 2018-02-21 13:07:40 +01:00
Rusty Russell
bdd11e07fe chaintopology: fix 100 block subtraction.
We do a complicated dance because we don't know the current block
height before setting up the topology.

If we're starting at a particular block, we want to go back 100 blocks
before that to cover any reorgs.

If we're not (fresh startup), we still want to go back 100 blocks
because we don't bother handling a reorg which removes all the blocks
we know.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-16 13:07:12 +01:00
Rusty Russell
6620305606 wallet: use last_processed_block to determine scan start.
With fallback depending on chainparams: this means the first upgrade
will be slow, but after that it'll be fast.

Fixes: #990
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-16 13:07:12 +01:00
Rusty Russell
21849329dd wallet: store last block number we searched for UTXOs.
We already go back 100 from this in case of reorgs, so the block number
itself is sufficient.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-16 13:07:12 +01:00
Rusty Russell
65593d4a98 chaintopology: don't start fee estimation loop until we're fully active.
This interacts badly with --daemon (next patch) which then tries to
reap a child it didn't create, which took me a couple of hours to
figure out.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-16 13:02:41 +01:00
Rusty Russell
55d962046b Rename (almost) all destructors to destroy_<type>.
We usually did this, but sometimes they were named after what they did,
rather than what they cleaned up.

There are still a few exceptions:
1. I didn't bother creating destroy_xxx wrappers for htable routines
   which already existed.
2. Sometimes destructors really are used for side-effects (eg. to simply
   mark that something was freed): these are clearer with boutique names.
3. Generally destructors are static, but they don't need to be: in some
   cases we attach a destructor then remove it later, or only attach
   to *some* cases.  These are best with qualifiers in the destroy_<type>
   name.

Suggested-by: @ZmnSCPxj
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-14 11:31:58 +01:00
Rusty Russell
0e93fb932a lightningd: bitcoind and topology routines take channel, not peer.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-14 11:31:58 +01:00
alaniz
b7856e3512 Onboarding: tweak grammar conventions for RPC responses 2018-02-02 06:16:23 +00:00