Before:
Ten builds, laptop -j5, no ccache:
```
real 0m36.686000-38.956000(38.608+/-0.65)s
user 2m32.864000-42.253000(40.7545+/-2.7)s
sys 0m16.618000-18.316000(17.8531+/-0.48)s
```
Ten builds, laptop -j5, ccache (warm):
```
real 0m8.212000-8.577000(8.39989+/-0.13)s
user 0m12.731000-13.212000(12.9751+/-0.17)s
sys 0m3.697000-3.902000(3.83722+/-0.064)s
```
After:
Ten builds, laptop -j5, no ccache: 8% faster
```
real 0m33.802000-35.773000(35.468+/-0.54)s
user 2m19.073000-27.754000(26.2542+/-2.3)s
sys 0m15.784000-17.173000(16.7165+/-0.37)s
```
Ten builds, laptop -j5, ccache (warm): 1% faster
```
real 0m8.200000-8.485000(8.30138+/-0.097)s
user 0m12.485000-13.100000(12.7344+/-0.19)s
sys 0m3.702000-3.889000(3.78787+/-0.056)s
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We were getting off-by-one for the total amount that the change is for,
since it rounds the fee *down*, independent of the total weight of the
entire tx.
We fix this by using the diff btw the fee of the total weight (w/ and
w/o the change output)
libwally has a bug which results in it failing to parse the 'empty tx'
cHNidP8BAAoAAAAAAAAAAAAAAA==. While we wait for the patch to land in
libwally, we patch over it.
Fix at: https://github.com/ElementsProject/libwally-core/pull/273
wally offers up `wally_clone_psbt` but it's a bit clunky (requires
checking return value, starting/stopping the wally_allocation context)
Helper method wraps this all up nice + neat!
We should actually be including this (as it may define _GNU_SOURCE
etc) before any system headers. But where we include <assert.h> we
often didn't, because check-includes would complain that the headers
included it too.
Weaken that check, and include config.h in C files before assert.h.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Our new "decode" command will also handle bolt11. We make a few cleanups:
1. Avoid type_to_string() in JSON, instead use format functions directly.
2. Don't need to escape description now that JSON core does that for us.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
In a couple of places we accept arrays of strings and don't validate
them. If we forward them, e.g., call a JSON-RPC method from the
plugin, we end up embedding the unverified string in the JSON-RPC
call without escaping, which then leads to invalid JSON being passed
on.
This at least partially causes #4238
We assert() this in onchaind while grinding fees; better to free newtx.
Before this we hit 530MB, after a mere 2.5MB.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: onchaind uses much less memory on unilateral closes for old channels.
Avoids much cut & paste. Some tests don't need any of it, but most
want at least some of this infrastructure.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
If a tx is larger than 2k, libwally will do an alloc:
```
lightning_hsmd: common/setup.c:11: wally_tal: Assertion `wally_tal_ctx' failed.
0x11c283 wally_tal
common/setup.c:11
0x15ebd1 wally_malloc
../../../libwally-core/src/internal.c:233
0x171e9e tx_to_bip143_bytes
../../../libwally-core/src/transaction.c:1918
0x172cda tx_to_bytes
../../../libwally-core/src/transaction.c:2086
0x1759df tx_get_signature_hash
../../../libwally-core/src/transaction.c:2776
0x175afd wally_tx_get_signature_hash
../../../libwally-core/src/transaction.c:2800
0x175b62 wally_tx_get_btc_signature_hash
../../../libwally-core/src/transaction.c:2810
0x1297d9 bitcoin_tx_hash_for_sig
bitcoin/signature.c:139
0x1298ca sign_tx_input
bitcoin/signature.c:161
0x10e701 handle_sign_remote_commitment_tx
hsmd/hsmd.c:1011
0x110f7f handle_client
hsmd/hsmd.c:1968
0x147a71 next_plan
ccan/ccan/io/io.c:59
0x1485ee do_plan
ccan/ccan/io/io.c:407
0x14862c io_ready
ccan/ccan/io/io.c:417
0x14a7f2 io_loop
ccan/ccan/io/poll.c:445
0x111125 main
hsmd/hsmd.c:2040
```
I reduced that constant in libwally to 200, and ran the entire
test suite, and found no other places.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Note that check-whitespace and check-bolt already do this, so we
can eliminate redundant lines in common/Makefile and bitcoin/Makefile.
We also include the plugin headers in ALL_C_HEADERS so they get
checked.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We force use of tal_wally_start/tal_wally_end around every wally
allocation, and with "end" make the caller choose where to reparent
everything.
This is particularly powerful where we allocate a tx or a psbt: we
want that tx or psbt to be the parent of the other allocations, so
this way we can reparent the tx or psbt, then reparent everything
else onto it.
Implementing psbt_finalize (which uses a behavior flag antipattern)
was tricky, so I ended up splitting that into 'psbt_finalize' and
'psbt_final_tx', which I think also makes the callers clearer.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
If we've already got a scriptSig field filled out for a PSBT input, we
use that instead of 'deriving' the scriptSig from the redeemscript
(finalizing a PSBT removes the redeemscript field)
