There are hardly any lightningd-specific JSON functions: all that's left
are the feerate ones, and there's already a comment that we should have
a lightningd/feerate.h.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We have them split over common/param.c, common/json.c,
common/json_helpers.c, common/json_tok.c and common/json_stream.c.
Change that to:
* common/json_parse (all the json_to_xxx routines)
* common/json_parse_simple (simplest the json parsing routines, for cli too)
* common/json_stream (all the json_add_xxx routines)
* common/json_param (all the param and param_xxx routines)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We noticed bogus behavior where (with 200 blocks added at once)
lightningd didn't see a DF channel open: this is because we told
lightningd about the new blocks (and it timed out channel) before
the watches for the transaction are fired.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Looks like we woke one of the startup io_loops early, and thus
we thought we'd finished connectd_activate and we hadn't. This
caused us to use an uninitialized ld->announceable array, and
finally caused an assert fail in the main loop.
Make *every* loop assert that it was exited for the correct reason,
so if it happens again, we can maybe figure out what part of
the code to look at.
```
lightningd: lightningd/lightningd.c:1186: main: Assertion `io_loop_ret == ld' failed.
lightningd: FATAL SIGNAL 6 (version 4df66fa)
...
------------------------------- Valgrind errors --------------------------------
Valgrind error file: valgrind-errors.895509
==895509== Conditional jump or move depends on uninitialised value(s)
==895509== at 0x22C58E: to_tal_hdr_or_null (tal.c:184)
==895509== by 0x22D531: tal_bytelen (tal.c:637)
==895509== by 0x1F10B6: towire_gossipd_init (gossipd_wiregen.c:100)
==895509== by 0x13AC6E: gossip_init (gossip_control.c:254)
==895509== by 0x1497EC: main (lightningd.c:1090)
==895509==
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fire off a snapshot of current account balances (node wallet + every
'active' channel) after we've caught up to the chain tip for the *first*
time (in other words, on start).
We're not going to do 'spend tracks' any more; instead we'll emit an
event whenever an output is included in a broadcast tx
(even if the broadcast fails!!)
And turn "" includes into full-path (which makes it easier to put
config.h first, and finds some cases check-includes.sh missed
previously).
config.h sets _GNU_SOURCE which really needs to be done before any
'#includes': we mainly got away with it with glibc, but other platforms
like Alpine may have stricter requirements.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Before:
Ten builds, laptop -j5, no ccache:
```
real 0m36.686000-38.956000(38.608+/-0.65)s
user 2m32.864000-42.253000(40.7545+/-2.7)s
sys 0m16.618000-18.316000(17.8531+/-0.48)s
```
Ten builds, laptop -j5, ccache (warm):
```
real 0m8.212000-8.577000(8.39989+/-0.13)s
user 0m12.731000-13.212000(12.9751+/-0.17)s
sys 0m3.697000-3.902000(3.83722+/-0.064)s
```
After:
Ten builds, laptop -j5, no ccache: 8% faster
```
real 0m33.802000-35.773000(35.468+/-0.54)s
user 2m19.073000-27.754000(26.2542+/-2.3)s
sys 0m15.784000-17.173000(16.7165+/-0.37)s
```
Ten builds, laptop -j5, ccache (warm): 1% faster
```
real 0m8.200000-8.485000(8.30138+/-0.097)s
user 0m12.485000-13.100000(12.7344+/-0.19)s
sys 0m3.702000-3.889000(3.78787+/-0.056)s
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Not necessary yet, but it will be once shutdown starts waiting for
plugins to respond: we don't want these to try to access the bcli
plugin once it's freed.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It's a legacy from when it didn't have an ld pointer to access ld's
timer structure. Now it's just confusing.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This actually caused the flake in test_funding_reorg_private, where
l1 and l2 might not mark the original channel disabled. In fact, they
should *remove* it as it gets reorged out.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Useful for parsing a passed in feerate before calling lightningd with
it, e.g. when you need to know what the feerate is for a fundpsbt before
calling fundpsbt
Changelog-Added: JSON-RPC: new command `parsefeerate` which takes a feerate string and returns the calculated perkw/perkb
Interestingly, we required that "sendrawtx" include "errmsg" field even
on success, otherwise we crashed in broadcast_remainder.
We only actually insist on an "errmsg" if success is false. And this
logic here is weird (the !success) was added by darosior in
947f5ddde1, which makes the msg checks redundant.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
I mistakenly assumed the block would be freed after processing completed. That
is not true since chaintopology keeps headers and stubs around for reorgs. So
we need to remove the precomputed txids along with the full_txs.
Fixing the following error by changing 'enum feerate' to int.
lightningd/bitcoind.c:183:29: error: result of comparison of constant
8 with expression of type 'enum feerate' is always true [-Werror,
-Wtautological-constant-out-of-range-compare]
for (enum feerate f = 0; f < NUM_FEERATES; f++) {
Changelog-Fixed: compile error on macos
Changelog-Deprecated: plugin: `bcli` replacements should note that `sendrawtransaction` now has a second required Boolean argument, `allowhighfees`, which if `true`, means ignore any fee limits and just broadcast the transaction. Use `--deprecated-apis` to use older `bcli` replacement plugins that only support a single argument.
We defer the notification to gossipd till the end of the spends. By itself not
a huge change, but it allows us to later migrate to doing updates blindly and
using the DB as our ground truth. It also allows us to simplify the
`wallet_outpoint_spend` semantics to not return two values in the next commit.
We're not going to mutate transactions in a block, so computing the txids
every time we need them is a waste, let's compute them upfront and use them
afterwards.
HTLC fees increase (larger weight), and the fee paid by the opener
has to include the anchor outputs (i.e. 660 sats).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is what txsend does, only we have a psbt so we have
to change the db interface to take a wally_tx.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We were basing the 2016 block timeout on the blockchain height that we had
processed at the time completed the funding, which could lag considerably
behind the network-wide blockheight. For example this could happen if we
started with `--rescan` from a low height, taking some time to catch up.
This means that we could end up forgetting channels even before reaching the
network blockheight. This patch instead uses the headercount reported by the
backend plugin if we aren't caught up yet. While the chances of this happening
are still there, the window it might happen are much reduced, since headers
can be synced in a couple of minutes.
Reported-by: Riccardo Masutti
Changelog-Changed: Funding timeout is now based on the header count reported by the bitcoin backend instead of our current blockheight which might be lower.
Technically an API break, but nobody relies on these I hope!
Note that the feerates warning was buried inside the style object:
it should be top-level.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Our existing coin_moves tracking logic assumed that any tx we had an
input in belonged to *all* of our wallet (not a bad assumption as long
as there was no way to update a tx that spends our wallets)
Now that we've got `signpsbt` implemented, however, we need to be
careful about how we account for withdrawals. For now we do a best guess
at what the feerate is, and lump all of our spent outputs as a
'withdrawal' when it's impossible to disambiguate
This moves the notification for our coin spends from when it's
successfully submited to the mempool to when they're confirmed in a
block.
We also add an 'informational' notice tagged as `spend_track` which
can be used to track which transaction a wallet output was spent in.
This adapts our fee estimations requests to the Bitcoin backend to the
new semantic, and batch the requests.
This makes our request for fees much simpler, and leaves some more
flexibility for a plugin to do something smart (it could still lie before
but now it's explicit, at least.) as we don't explicitly request
estimation for a specific mode and a target.
Changelog-Changed: We now batch the requests for fee estimation to our Bitcoin backend.
Changelog-Changed: We now get more fine-grained fee estimation from our Bitcoin backend.
We kept track of an URGENT, a NORMAL, and a SLOW feerate. They were used
for opening (NORMAL), mutual (NORMAL), UNILATERAL (URGENT) transactions
as well as minimum and maximum estimations, and onchain resolution.
We now keep track of more fine-grained feerates:
- `opening` used for funding and also misc transactions
- `mutual_close` used for the mutual close transaction
- `unilateral_close` used for unilateral close (commitment transactions)
- `delayed_to_us` used for resolving our output from our unilateral close
- `htlc_resolution` used for resolving onchain HTLCs
- `penalty` used for resolving revoked transactions
We don't modify our requests to our Bitcoin backend, as the next commit
will batch them !
Changelog-deprecated: The "urgent", "slow", and "normal" field of the `feerates` command are now deprecated.
Changelog-added: The fields "opening", "mutual_close", "unilateral_close", "delayed_to_us", "htlc_resolution" and "penalty" have been added to the `feerates` command.
This adds `getchaininfo` and `getrawblockbyheight` handling lightningd-side,
and use them in setup_topology().
We then remove legacy bitcoind_getblockcount() (we already get the count in
`getchaininfo`), bitcoind_getblockchaininfo() (it was only used in setup_topology()),
and wait_for_bitcoind() (this was specific to bitcoin-core and we assume our Bitcoin
backend to be functional if the plugin responds to `init`).
If the same memory gets reallocated, our "has the tip changed?" test
gets a false negative. This happened for me about one time in 10,
causing tests/test_misc.py::test_funding_reorg_remote_lags to fail.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
