Commit Graph

8073 Commits

Author SHA1 Message Date
niftynei
185fe722be update-mocks: make it a bit easier to tell what step is happening
Adding a small explainer before printing the filename makes it a bit
clearer what's going on when parsing make logs
2020-05-30 15:36:56 +02:00
niftynei
76c57595c3 tools-make: add mock parser for clang ld output
According to #3226, it looks like clang's LD error format has changed.
This patch adds the new format so we can parse the mocks successfully.

Apple clang version 11.0.3 (clang-1103.0.32.29)

```
checking for ANSI C header files... Undefined symbols for architecture x86_64:
  "_fromwire_amount_msat", referenced from:
      _fromwire_tlv_test_n1_tlv3 in ccj4zKdV.o
  "_fromwire_bool", referenced from:
      _fromwire_test_msg in ccj4zKdV.o
      _fromwire_test_msg_option_short_id in ccj4zKdV.o
      _fromwire_test_msg_option_one in ccj4zKdV.o
      _fromwire_test_msg_option_two in ccj4zKdV.o
  "_fromwire_test_enum", referenced from:
      _fromwire_test_msg in ccj4zKdV.o
      _fromwire_test_msg_option_short_id in ccj4zKdV.o
      _fromwire_test_msg_option_one in ccj4zKdV.o
      _fromwire_test_msg_option_two in ccj4zKdV.o
  "_fromwire_tlvs", referenced from:
      _fromwire_test_tlv1 in ccj4zKdV.o
      _fromwire_test_tlv2 in ccj4zKdV.o
      _fromwire_test_tlv3 in ccj4zKdV.o
  "_fromwire_tu32", referenced from:
      _fromwire_tlv_test_n2_tlv2 in ccj4zKdV.o
  "_fromwire_tu64", referenced from:
      _fromwire_tlv_test_n1_tlv1 in ccj4zKdV.o
      _fromwire_tlv_test_n2_tlv1 in ccj4zKdV.o
  "_fromwire_u16", referenced from:
      _fromwire_test_features in ccj4zKdV.o
      _fromwire_subtype_var_assign in ccj4zKdV.o
      _fromwire_subtype_arrays in ccj4zKdV.o
      _fromwire_tlv_test_n1_tlv4 in ccj4zKdV.o
      _fromwire_tlv_test_n3_tlv3 in ccj4zKdV.o
      _fromwire_test_msg in ccj4zKdV.o
      _fromwire_test_tlv1 in ccj4zKdV.o
      ...
  "_fromwire_u32", referenced from:
      _fromwire_tlv_test_n3_tlv3 in ccj4zKdV.o
      _fromwire_test_msg in ccj4zKdV.o
      _fromwire_test_msg_option_short_id in ccj4zKdV.o
      _fromwire_test_msg_option_one in ccj4zKdV.o
      _fromwire_test_msg_option_two in ccj4zKdV.o
  "_fromwire_u64", referenced from:
      _fromwire_test_short_id in ccj4zKdV.o
      _fromwire_tlv_test_n3_tlv3 in ccj4zKdV.o
  "_fromwire_u8", referenced from:
      _fromwire_subtype_var_assign in ccj4zKdV.o
      _fromwire_subtype_var_len in ccj4zKdV.o
      _fromwire_subtype_varlen_varsize in ccj4zKdV.o
      _fromwire_tlv_test_n3_tlv3 in ccj4zKdV.o
  "_fromwire_u8_array", referenced from:
      _fromwire_test_features in ccj4zKdV.o
      _fromwire_subtype_arrays in ccj4zKdV.o
      _fromwire_tlv_test_n3_tlv3 in ccj4zKdV.o
      _fromwire_test_msg in ccj4zKdV.o
      _fromwire_test_msg_option_short_id in ccj4zKdV.o
      _fromwire_test_msg_option_one in ccj4zKdV.o
      _fromwire_test_msg_option_two in ccj4zKdV.o
      ...
```

Changelog-Fixed: Build for macOS Catalina / Apple clang v11.0.3 fixed
2020-05-30 15:36:56 +02:00
niftynei
3640befc10 tools-make: also be quiet if --quiet is flagged
Missed a update-mocks.sh call
2020-05-30 15:36:56 +02:00
niftynei
24e186bc20 test: use correct deps for channeld tests
LIGHTNING_CHANNELD_* doesn't exist; the actual name is
LIGHTNINGD_CHANNEL_*
2020-05-29 16:20:23 -05:00
niftynei
78d95b51aa nit: align spacing for SQL stmts 2020-05-29 16:20:23 -05:00
fiatjaf
96452eafb7 sort listinvoices and listsendpays by order of creation. 2020-05-29 15:30:59 +02:00
Rusty Russell
ce9e559aed features: do not set option_support_large_channel in channel_announcement.
Spec is wrong (it says it should be compulsory), and Eclair doesn't set it
at all, leading to an error when they send their announcement_signatures.

Fixes: #3703
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-changed: large-channels: negotiate successfully with Eclair nodes.
2020-05-26 19:57:29 -05:00
Rusty Russell
b3eef81f19 pytest: add test for channel_announcement feature bits.
And neaten current feature mangling code now we have a wumbo flag for
expected_peer_features().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-26 19:57:29 -05:00
Rusty Russell
4d917e2566 gossipd: add test showing how we and Eclair disagree on channel_announcement features for wumbo channels.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-26 19:57:29 -05:00
Rusty Russell
f4f8a363dd pytest: fix feature mask for EXPERIMENTAL_FEATURES, add wumbo support.
There are various places where our tests failed with
--enable-expimental-features.  And our plugin test overlapped an
existing feature.

We make our expected_feature functions more generic, and use them
everywhere.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-26 19:57:29 -05:00
Rusty Russell
0b89664076 gossipd: fix cut & paste in error messages.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-26 19:57:29 -05:00
Rusty Russell
0b3040b9a6 bitcoin/tx: insist input amounts all be populated across the wire.
If you need to send a tx where you don't know this info, send a tx_parts.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-27 10:12:03 +09:30
Rusty Russell
1e889eeaf7 lightningd: have sign_last_tx populate the input amounts.
With this change, all bitcoin_tx we send across the wire have
their inputs_amounts populated.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-27 10:12:03 +09:30
Rusty Russell
48853ec954 onchaind: use tx_parts for initial tx.
For the moment it's a complete tx, but in future designs we might only
be given the specific input which closes the channel.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-27 10:12:03 +09:30
Rusty Russell
abba59057c onchaind: remove chainparams args in favor of global.
Otherwise this creates noise for the next patch which switches the initial
`struct bitcoin_tx` into a `struct tx_parts`.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-27 10:12:03 +09:30
Rusty Russell
48f397ee19 onchaind: receive a tx_parts instead of a tx when a tx is seen onchain.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-27 10:12:03 +09:30
Rusty Russell
dafaf854c5 bitcoin/tx_parts: infrastructure for partial bitcoin txs.
`struct tx_parts` is just a txid and a bunch of inputs and outputs,
some of which may be NULL.

This is both a nod towards a future where we (or our peer) can combine
HTLCs or (in an eltoo world) commitments, although for the moment all
our tx_parts will be complete.

It also matches our plan to split `bitcoin_tx` into two types: this
`struct tx_parts` where we don't know input amounts etc, and `psbt`
where we do.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-27 10:12:03 +09:30
Rusty Russell
7e48f77d43 bitcoin/tx: expose functions for raw wally_txs.
These are useful for tx_parts:

1. wally_txid.
2. linearize_wtx.
3. wally_tx_input_spends
4. wally_tx_output_get_amount
5. wally_tx_input_get_txid

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-27 10:12:03 +09:30
Glen Cooper
8d98b99bde Update TOR.md
typo: referes, changed to refer
2020-05-26 12:21:40 +09:30
Antoine Poinsot
5dfd2436db wallet: show input txid in little endian for json_listtransactions
Changelog-Fixed: jsonrpc: `listtransactions` now displays all txids as little endian

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-05-22 15:11:51 +02:00
Antoine Poinsot
1061cd95c9 pytest: add a sanity check for the listtransactions command
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-05-22 15:11:51 +02:00
lisa neigut
1fb9a078b6 txprepare: return psbt serialized version of tx as well
Changelog-Added: JSON-API: `txprepare` returns a psbt version of the created transaction
2020-05-21 18:45:07 +09:30
niftynei
bf4cac7fb8 tx: strip out witscript
now that witness script data is saved into the tx/psbt which is
serialized across the wire, there's no reason to use witscript to do
this. good bye witscript!
2020-05-21 18:45:07 +09:30
niftynei
559f88faa1 psbt: add serialize to/from wire for psbts 2020-05-21 18:45:07 +09:30
niftynei
5d0fc176e8 psbt: create new_psbt method
Move all psbt creation into single method, new_psbt

note that if a psbt is init'd for a transaction that's
deserialized with scripts etc already attached, then set_global_tx
will fail. instead, we empty all of this out first.

if the tx is being re-init'd from a tx source that had a psbt attached
(e.g. fromwire_) then the script/witness data will get populated
appropriatel from there.
2020-05-21 18:45:07 +09:30
niftynei
a848df67f1 psbt: populate scriptsigs + witnesses
Pass scriptSig + witness info down to the PSBT struct
2020-05-21 18:45:07 +09:30
niftynei
2055b53425 psbt: update global tx output amount also 2020-05-21 18:45:07 +09:30
niftynei
cf9de86dba psbt: add transaction inputs to the psbt struct
Make sure that we permute them also!

Fixes weird spacing also
2020-05-21 18:45:07 +09:30
niftynei
a1e1073752 tx: move to tal_dup_arr 2020-05-21 18:45:07 +09:30
niftynei
b076f40cf3 psbt: move witness script storage into the psbt
we can now keep associated witness data with the output in the psbt
struct, so we do that.
2020-05-21 18:45:07 +09:30
niftynei
2d5c61dfc1 psbt: methods to mutate psbt in place
add the missing psbt helpers for adding and removing an input and output
2020-05-21 18:45:07 +09:30
niftynei
7a0624797e psbt: add psbt to bitcoin tx struct 2020-05-21 18:45:07 +09:30
niftynei
24ecb3e2b9 Remove trailing line in fromwire.c 2020-05-21 18:45:07 +09:30
Rusty Russell
06ae03f487 external/libwally-core: update to latest version.
Includes an important PSBT fix (https://github.com/ElementsProject/libwally-core/pull/190)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-21 18:45:07 +09:30
Rusty Russell
f1426bad8e bitcoin/tx: fix bug in fromwire_bitcoin_tx.
If we fail to unmarshal the tx (shouldn't happen, but...) we must
not dereference NULL.

Also tighten the check: towire_ must send 0 or all inputs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-20 11:36:13 +02:00
Rusty Russell
b920d4d21b wire: make fromwire_fail return non-const, use it more
It returns NULL, so you can simply `return fromwire_fail(...)`
if you want to return NULL in this case.  Use that more.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-20 11:36:13 +02:00
Antoine Poinsot
0088147d9e pytest: add sanity check for --commit-fee
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-05-20 06:09:24 +09:30
Antoine Poinsot
d67743d511 bcli cleanups
Correct some comments wrt estimatefees, add a constructor for bitcoind

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-05-20 06:09:24 +09:30
Antoine Poinsot
f598caa60d config: don't ignore the --commit-fee option.
We did not take the value of --commit-fee into account : this removes
the unused option from lightningd and instead registers it in bcli,
where we set the actual feerate of commitment transactions. This also
corrects the documentation.

Changelog-Fixed: config: we now take the --commit-fee parameter into account.
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-05-20 06:09:24 +09:30
niftynei
47c701cbe6 make: have update-mocks actually be quiet
Shhhh
2020-05-19 20:25:23 +09:30
Rusty Russell
7ed3e72b88 Makefile: don't print out what we're doing if --quiet/-s
Reported-by: @niftynei
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-19 20:25:23 +09:30
Rusty Russell
4bb92178b1 Update lightningd/lightningd.c
Co-authored-by: neil saitug <niftynei@gmail.com>
2020-05-19 11:46:11 +02:00
Rusty Russell
e0517a1022 doc: lightningd comment refers to obsolete bitcoin_tx.
It's now a wrapper for wally_tx, so update example.

Reported-by: @niftynei
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-19 11:46:11 +02:00
niftynei
9845eb41a9 contrib: allow stderr printing for startup_regtest
Moves the 'daemon'ization from c-lightning to the process level, so that
stderr print messages appear in the terminal. Easier debugging!

Changelog-None
2020-05-19 11:44:54 +02:00
niftynei
632b42da40 hsmd: fix missing return bug
One must `return` the bad_req, otherwise you continue onward, usually
with bad or unexpected results.

introduced in 6b6b7eac61
2020-05-19 11:44:00 +02:00
niftynei
66f59659a7 setup: add setup to make checks
- we've moved tmpctx management to setup.c from daemon.c, so we update
the `check-tmpctx`
- `common_setup(char *)` is now a valid analog for `setup_locale`, so we
check for either in check-setup_locale
2020-05-19 13:35:42 +09:30
niftynei
fbe50e087a setup: create a common setup which will handle the wally-context
Since we now over-write the wally malloc/free functions, we need to do
so for tests as well. Here we pull up all of the common setup/teardown
logic into a separate place, and update the tests that use libwally to
use the new common_setup core

Changelog-None
2020-05-19 13:35:42 +09:30
niftynei
c5a0d8cfbf wally: override wally to use tal-context
See attached. This lets us do fancy things with tal_steal
2020-05-19 13:35:42 +09:30
Rusty Russell
cfb320c972 wire: move remaining bitcoin functions out to bitcoin/ files.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-18 14:51:12 +02:00
Rusty Russell
197d1bcef2 wire: move towire/fromwire_short_channel_id out to bitcoin/short_channel_id.c
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-18 14:51:12 +02:00