Commit graph

276 commits

Author SHA1 Message Date
Rusty Russell
5bc61e77cb onchaind: speed up HTLC matching by skipping identical CLTVs.
We try signatures to see which HTLC (we can have many) is the right one;
we can trivially match htlcs against commitment tx outputs, but the CTLV
can vary, and that's inside the htlc tx itself.

By sorting them, it's easy to skip comparing duplicates:

Time before: 2m32.547s
Time after: 1m6.984s

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-07 14:23:59 +01:00
Rusty Russell
f22d719d2c onchaind: test stressful input.
In particular, this made valgrind OOM, and chewed much CPU.  I dumped the
input and output into a file to allow easy replay.

This will break as soon as we change onchaind's wire format, but it will
serve its purpose until then!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-07 14:23:59 +01:00
Rusty Russell
ff8830876d wire/tlvstream: add tlv_make_fields helper to populate ->fields array.
This is vital for calculating merkle trees; I previously used
towire+fromwire to get this!

Requires generation change so we can magic the ARRAY_SIZE var (the C
pre-processor can't uppercase things).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-04 20:16:54 -06:00
Rusty Russell
5c167d16ab tools/generate-wire.py: use helpers.
This was terrible cut & paste.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-04 20:16:54 -06:00
Rusty Russell
dc83e64003 tools/generate-wire: don't use void * pointers for tlv fromwire.
And fix up the one place which got it wrong.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-04 20:16:54 -06:00
Rusty Russell
5e865ce42b Makefile: unify generated files definition.
We change gen_ to _gen, because filtering on gen_% doesn't work if they're
in subdirectories :(

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-22 12:14:34 +10:30
niftynei
4508584b21 dualfund: rearrange things so that the wire-dependent calls are separate
There's a few structs/wire calls that only exist under experimental features.
These were in a common file that was shared/used a bunch of places but
this causes problems. Here we move one of the problematic methods back
into `openingd`, as it's only used locally and then isolate the
references to the `witness_stack` in a new `common/psbt_internal` file.

This lets us remove the iff EXP_FEATURES inclusion switches in most of
the Makefiles.
2020-10-20 14:27:19 +10:30
niftynei
41ebf71e26 psbt: new methods for generating serial_ids for an input/output
we need to do this elsewhere later, pull it out so we can use it
2020-10-20 12:50:31 +10:30
niftynei
82c0b48215 wires: towire/fromwire for wally_tx
We're eventually moving away from 'bitcoin_tx
2020-10-20 12:50:31 +10:30
Rusty Russell
f37f2b6193 common/memleak: simplify and document API.
1. Rename memleak_enter_allocations to memleak_find_allocations.
2. Unify scanning for pointers into memleak_remove_region / memleak_remove_pointer.
3. Document the functions.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-23 13:52:49 +09:30
Matt Whitlock
eab14768a8 update SHA256STAMPs using sorted dependencies 2020-09-17 10:23:40 +09:30
niftynei
5cd06227d7 build: exclude dualopend from non-experimental builds 2020-09-09 19:54:20 +09:30
niftynei
303263d381 psbt: clean up interface for setting metadata on PSBT inputs
it's just neater if it's not all wrapped up together, simplifies the
interface a smidge
2020-09-09 19:54:20 +09:30
niftynei
8d429ecd06 df: add needed info to any PSBT we produce
dual funding needs the max-witness-len and utxo fields set for every
input. we should add them when we create a 'fundpsbt', so that every
psbt that c-lightning generates is dual-funding ready
2020-09-09 19:54:20 +09:30
Rusty Russell
0e805427dc tools/generate-wire.py: strip trailing whitespace on lines, fix bolt quotes.
There's a lot of it, and it means we can't `make check-source` on
these files.

Also bring bolt quotes up-to-date.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-09 16:23:58 +09:30
ZmnSCPxj jxPCSnmZ
06b44f00a3 onchaind/onchaind.c: Implement scorch-the-earth.
Fixes: #3832

Changelog-Changed: onchaind: We now scorch the earth on theft attempts, RBFing up our penalty transaction as blocks arrive without a penalty transaction getting confirmed.
2020-09-09 12:38:19 +09:30
ZmnSCPxj jxPCSnmZ
abc585b7a4 onchaind/onchain_wire.csv: Allow suppression of auto-rebroadcasting for RBF txes created by onchaind. 2020-09-09 12:38:19 +09:30
ZmnSCPxj jxPCSnmZ
34bf0133f2 onchaind/onchaind_wire.csv: Propagate minimum relay fee to onchaind. 2020-09-09 12:38:19 +09:30
Rusty Russell
d8e8426b52 Makefile: remove EXPERIMENTAL_FEATURES marker from generated files
We're going to make experimental versions of these completely separate files.

Also remove the dependency on the Makefile itself: it simply causes
unnecessary churn.  We can always force-rebuild when we change a rule.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-08 09:42:00 +09:30
Rusty Russell
82f2f43425 Generated files: more merge fallout.
We changed the generation templates!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-02 11:10:21 +09:30
Rusty Russell
3c6af3efb4 Makefile: commit and preserve all the wiregenerated files.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Build: we no longer require extra Python modules to build.
2020-08-31 21:33:26 -05:00
Rusty Russell
1746406e41 Makefile: normalize all the Makefiles
We create ALL_PROGRAMS, ALL_TEST_PROGRAMS, ALL_C_SOURCES and
ALL_C_HEADERS.  Then the toplevel Makefile knows which are
autogenerated (by wildcard), so it can have all the rules to clean
them or check the source as necessary.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-31 21:33:26 -05:00
Rusty Russell
8150d28575 Makefile: use generic rules to make spec-derived sources.
Now we use the same Makefile rules for all CSV->C generation.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-31 21:33:26 -05:00
Rusty Russell
3e52d4100d common: convert to new wire generation style.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-25 12:53:13 +09:30
Rusty Russell
fe8564555f closingd: convert to new wire generation style.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-25 12:53:13 +09:30
Rusty Russell
1702c7a69a hsmd: convert to new wire generation style.
Note that other directories were explicitly depending on the generated
file, instead of relying on their (already existing) dependency on 
$(LIGHTNINGD_HSM_CLIENT_OBJS), so we remove that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-25 12:53:13 +09:30
Rusty Russell
6e4e267f2c doc: update BOLT now option_anchor_outputs is merged.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-24 20:55:47 +09:30
Rusty Russell
c753561d24 onchaind: recognize the to-remote output if option_anchor_outputs.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-14 11:51:14 +09:30
Rusty Russell
a730cbdc96 onchaind: recognize (and ignore) anchor outputs.
Without this, onchaind fails to identify them.  So onchaind now needs
to know the funding pubkeys.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-14 11:51:14 +09:30
Rusty Russell
a5d0c14d4d option_anchor_outputs: wire into all the subdaemons.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-14 11:51:14 +09:30
Rusty Russell
d8d1014ca4 channeld: implement htlc sig checking changes for option_anchor_outputs.
This is best done by passing `struct bitcoin_signature` around instead
of raw signatures.  We still save raw sigs to the db, and of course the
wire protocol uses them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-14 11:51:14 +09:30
Rusty Russell
85e3b43176 channeld, openingd: take into account option_anchor_outputs for fees.
HTLC fees increase (larger weight), and the fee paid by the opener
has to include the anchor outputs (i.e. 660 sats).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-14 11:51:14 +09:30
Rusty Russell
7a2f964d43 hsmd: take option_anchor_outputs flag for HTLC signatures.
They are to be signed with SIGHASH_SINGLE|SIGHASH_ANYONECANPAY.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-14 11:51:14 +09:30
Rusty Russell
05c3a9bf12 script: change htlc output scripts if option_anchor_outputs.
For the moment, everyone passes "false".

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-14 11:51:14 +09:30
Rusty Russell
edf60b4f9e openingd: remember if we negotiated option_anchor_outputs, to put in the db.
And hand it through to channeld just like option_static_remotekey.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-14 11:51:14 +09:30
Rusty Russell
93d04d08d0 wire: update to latest version of the spec.
The main change here is that the previously-optional open/accept
fields and reestablish fields are now compulsory (everyone was
including them anyway).  In fact, the open/accept is a TLV
because it was actually the same format.

For more details, see lightning-rfc/f068dd0d8dfa5ae75feedd99f269e23be4777381

Changelog-Removed: protocol: support for optioned form of reestablish messages now compulsory.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-23 18:49:15 +02:00
niftynei
f9300e8480 tx: add setter for tx locktime
We need to update the psbt's global transaction simultaneously, so we
wrap access to the locktime in a method which will handle both
2020-06-11 13:13:13 +02:00
niftynei
dc868630a8 tx-psbt: pass in the witness script (if known) when adding an input
Update the `bitcoin_tx_add_input` interface to accept a witness script
and or scriptPubkey.

We save the amount + witness script + witness program (if known) to
the PSBT object for a transaction when creating an input.
2020-06-11 13:13:13 +02:00
niftynei
a04f0fe250 psbt: remove input_amounts from bitcoin tx
Instead we will stash them into the PSBT as a utxo/witness record (which
includes the amount)
2020-06-11 13:13:13 +02:00
Rusty Russell
48853ec954 onchaind: use tx_parts for initial tx.
For the moment it's a complete tx, but in future designs we might only
be given the specific input which closes the channel.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-27 10:12:03 +09:30
Rusty Russell
abba59057c onchaind: remove chainparams args in favor of global.
Otherwise this creates noise for the next patch which switches the initial
`struct bitcoin_tx` into a `struct tx_parts`.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-27 10:12:03 +09:30
Rusty Russell
48f397ee19 onchaind: receive a tx_parts instead of a tx when a tx is seen onchain.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-27 10:12:03 +09:30
niftynei
559f88faa1 psbt: add serialize to/from wire for psbts 2020-05-21 18:45:07 +09:30
niftynei
b076f40cf3 psbt: move witness script storage into the psbt
we can now keep associated witness data with the output in the psbt
struct, so we do that.
2020-05-21 18:45:07 +09:30
Rusty Russell
b920d4d21b wire: make fromwire_fail return non-const, use it more
It returns NULL, so you can simply `return fromwire_fail(...)`
if you want to return NULL in this case.  Use that more.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-20 11:36:13 +02:00
niftynei
fbe50e087a setup: create a common setup which will handle the wally-context
Since we now over-write the wally malloc/free functions, we need to do
so for tests as well. Here we pull up all of the common setup/teardown
logic into a separate place, and update the tests that use libwally to
use the new common_setup core

Changelog-None
2020-05-19 13:35:42 +09:30
Rusty Russell
cfb320c972 wire: move remaining bitcoin functions out to bitcoin/ files.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-18 14:51:12 +02:00
Rusty Russell
197d1bcef2 wire: move towire/fromwire_short_channel_id out to bitcoin/short_channel_id.c
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-18 14:51:12 +02:00
Rusty Russell
13356b75bf wire: move towire/fromwire_amount from wire/ into common/amount.c
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-18 14:51:12 +02:00
Rusty Russell
4dbfce6057 wire: move towire/fromwire_preimage out to bitcoin/preimage.c.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-18 14:51:12 +02:00