Commit Graph

1473 Commits

Author SHA1 Message Date
Rusty Russell
62a5183fb5 common: add ability for gossip_store to track by timestamp.
We'll use this to keep a two-hour-ago "recent gossip offset" as an
optimization.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-15 21:18:29 +09:30
adi2011
286d6c3165 tools/gen: Always return bool! 2022-07-14 12:24:48 -05:00
adi2011
e42ba8366b common: Add scb_wire for serializing the static_chan_backup 2022-07-14 12:24:48 -05:00
Michael Schmoock
f67ab2a86e test: run-gossmap update test store instructions
Changelog-None
2022-07-12 13:59:46 -05:00
Rusty Russell
1217f479df sendpay: allow route to contain both amount_msat and (deprecated) msatoshi.
Since it's a deprecation, we simply ignore one, rather than properly
checking they match etc.

Fixes: #5386
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-12 12:42:05 -05:00
Christian Decker
9ff9d3d4e3 tests: Fix the canned gossmap
The gossip_store version was bumbed to 10 in PR #5239 and #5375 has
apparently not been rebased on top of that when it was merged, causing
the `run-gossmap_canned` test to fail when parsing the v9 gossip
store.

Changelog-None
2022-07-11 14:24:10 -05:00
Rusty Russell
ae71a87c40 ccan: update to latest htable fixes, and update gossmap to meet new assertions.
Updating ccan to stricter htable revealed we were trying to put
(void *)1 in the htable, which is forbidden:

```
topology: ccan/ccan/htable/htable.c:382: htable_add_: Assertion `entry_is_valid((uintptr_t)p)' failed.
topology: FATAL SIGNAL 6 (version 1358d7f)
0x55f30c689c34 send_backtrace
	common/daemon.c:33
0x55f30c689ce0 crashdump
	common/daemon.c:46
0x7f5d150fe51f ???
	./signal/../sysdeps/unix/sysv/linux/x86_64/libc_sigaction.c:0
0x7f5d15152828 __pthread_kill_implementation
	./nptl/pthread_kill.c:44
0x7f5d15152828 __pthread_kill_internal
	./nptl/pthread_kill.c:80
0x7f5d15152828 __GI___pthread_kill
	./nptl/pthread_kill.c:91
0x7f5d150fe475 __GI_raise
	../sysdeps/posix/raise.c:26
0x7f5d150e47b6 __GI_abort
	./stdlib/abort.c:79
0x7f5d150e46da __assert_fail_base
	./assert/assert.c:92
0x7f5d150f5e25 __GI___assert_fail
	./assert/assert.c:101
0x55f30c6adbe4 htable_add_
	ccan/ccan/htable/htable.c:382
0x55f30c65f303 chanidx_htable_add
	common/gossmap.c:35
0x55f30c6605ed new_channel
	common/gossmap.c:337
0x55f30c6609cf add_channel
	common/gossmap.c:425
0x55f30c661101 map_catchup
	common/gossmap.c:607
0x55f30c66221e gossmap_refresh
	common/gossmap.c:927
0x55f30c66e3e9 get_gossmap
	plugins/topology.c:27
0x55f30c66f939 listpeers_done
	plugins/topology.c:369
0x55f30c671f46 handle_rpc_reply
	plugins/libplugin.c:558
0x55f30c672a19 rpc_read_response_one
	plugins/libplugin.c:726
0x55f30c672b4f rpc_conn_read_response
	plugins/libplugin.c:746
0x55f30c6ae35e next_plan
	ccan/ccan/io/io.c:59
0x55f30c6aef93 do_plan
	ccan/ccan/io/io.c:407
0x55f30c6aefd5 io_ready
	ccan/ccan/io/io.c:417
0x55f30c6b1371 io_loop
	ccan/ccan/io/poll.c:453
0x55f30c67587c plugin_main
	plugins/libplugin.c:1559
0x55f30c6708eb main
	plugins/topology.c:701
0x7f5d150e5fcf __libc_start_call_main
	../sysdeps/nptl/libc_start_call_main.h:58
0x7f5d150e607c __libc_start_main_impl
	../csu/libc-start.c:409
0x55f30c65d894 ???
	???:0
0xffffffffffffffff ???
	???:0
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-10 20:48:53 -05:00
Rusty Russell
32af92145b update-mocks: handle missing deprecated_apis.
This expands update-mocks to be able to handle (simple!) missing
symbols which are not functions.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-09 09:59:52 +09:30
Alex Myers
cbafc0fa33 gossip_store: add flag for spam gossip, update to v10
This will be used to decouple internal use of gossip from what is
passed to gossip peers. Updates GOSSIP_STORE_VERION to 10.

Changelog-Changed: gossip_store updated to version 10.
2022-07-06 14:31:19 +09:30
Christian Decker
d115d01105 zeroconf: Add channel_type variant support
If zeroconf was negotiated we'll add it to the basic channel
type. Similarly we'll accept it if it was negotiated too.
2022-07-04 22:14:06 +02:00
Christian Decker
a07797c166 features: Add function to unset a featurebit
Needed so we can blank optional bits when comparing channel_types.
2022-07-04 22:14:06 +02:00
Christian Decker
e4511452ac bolt: Reflect the zeroconf featurebits in code 2022-07-04 22:14:06 +02:00
Christian Decker
6d07f4ed85 json: Add parser for u32 params 2022-07-04 22:14:06 +02:00
Greg Sanders
9f953b5efb No funding_wscript arg in initial_commit_tx 2022-07-01 13:30:19 -05:00
Vincenzo Palazzo
7ff62b4a00 lightnind: removeDEFAULT_PORT global definition
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2022-06-28 06:09:01 +09:30
Rusty Russell
2b7915359c gossmap: handle case where private channel turns into public.
Usually we won't see this, since private is deleted.  But we could
have already read the private channel before that.  Handle it properly.

(Tested by removing the gossip_store deletion code and making sure
this worked).

We have to fix up the test, which announces a channel twice!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-06-27 17:21:35 +09:30
Rusty Russell
a196c77fe6 common: downgrade LND 'internal error' properly.
Thanks to @zerofeerouting for another report.

"desc" here is the sanitized message, eg:
"ERROR error channel 0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef: internal error"

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-06-26 13:52:14 +09:30
Rusty Russell
e48c0dda85 common: downgrade "internal error" errors from lnd.
Prior to 0.11.0 we had cases where we would treat errors
as warnings: regretfully, this is still needed.  This message
in particular has been widely reported, and it now causes
channel force closes.

Downgrade and log.  I did insert some snarky log message earlier,
but hey, I'm sure CLN has done worse things to our peers!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Protocol: treat LND "internal error" as warnings, not force close events (as we did in v0.10).
2022-06-26 13:52:14 +09:30
Rusty Russell
0b7897302e common/gossip_store: optimize case where entries are filtered out.
@whitslack complained of large CPU usage by connectd at startup;
I ran perf record on connectd on my machine (which sees a little spike, only)
and I see the cost of reading and discarding the entries:

```
-   95.52%     5.24%  lightning_conne  lightning_connectd  [.] gossip_store_next
   - 90.28% gossip_store_next
      + 40.27% tal_alloc_arr_
      + 22.78% tal_free
      + 11.74% crc32c
      + 9.32% fromwire_peektype
      + 4.10% __libc_pread64 (inlined)
        1.70% be32_to_cpu
```

Much of this is caused by the search for our own gossip: keeping this separately
would be even better, but this fix is minimal.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: connectd: reduce initial CPU load when connecting to peers.
2022-06-24 10:33:04 +09:30
Rusty Russell
60bd70be85 JSON: deprecate printing msat fields as strings.
This changes many fields: in non-deprecated mode, they're now raw integers.
This was always the intention, but the transition was never completed.

Suggested-By: @ShahanaFarooqui
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: "_msat" fields can be raw numbers, not "123msat" strings (please handle both!)
Changelog-Deprecated: JSON-RPC: "_msat" fields as "123msat" strings (will be only numbers)
2022-06-21 06:52:35 +09:30
Rusty Russell
5531de99de lease_rates: prepare for msats fields as raw numbers.
We would otherwise multiply them by 1000.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-06-21 06:52:35 +09:30
Rusty Russell
c3efba16ff JSON: don't print deprecated amount fields any more
A small change in one routine creates a lot of changes!  We actually
recommended moving away from these in v0.7.0 (2019-02-28), but never
deprecated them formally.

Changelog-Deprecated: JSON-RPC: `pay`, `decode`, `decodepay`, `getroute`, `listinvoices`, `listpays` and `listsendpays` `msatoshi` fields (use `amount_msat`).
Changelog-Deprecated: JSON-RPC: `getinfo` `msatoshi_fees_collected` field (use `fees_collected_msat`).
Changelog-Deprecated: JSON-RPC: `listpeers` `channels`: `msatoshi_to_us`, `msatoshi_to_us_min`, `msatoshi_to_us_max`, `msatoshi_total`, `dust_limit_satoshis`, `our_channel_reserve_satoshis`, `their_channel_reserve_satoshis`, `spendable_msatoshi`, `receivable_msatoshi`, `in_msatoshi_offered`, `in_msatoshi_fulfilled`, `out_msatoshi_offered`, `out_msatoshi_fulfilled`, `max_htlc_value_in_flight_msat` and `htlc_minimum_msat` (use `to_us_msat`, `min_to_us_msat`, `max_to_us_msat`, `total_msat`, `dust_limit_msat`, `our_reserve_msat`, `their_reserve_msat`, `spendable_msat`, `receivable_msat`, `in_offered_msat`, `in_fulfilled_msat`, `out_offered_msat`, `out_fulfilled_msat`, `max_total_htlc_in_msat` and `minimum_htlc_in_msat`).
Changelog-Deprecated: JSON-RPC: `listinvoices` and `pay` `msatoshi_received` and `msatoshi_sent` (use `amount_received_msat`, `amount_sent_msat`)
Changelog-Deprecated: JSON-RPC: `listpays` and `listsendpays` `msatoshi_sent` (use `amount_sent_msat`)
Changelog-Deprecated: JSON-RPC: `listforwards` `in_msatoshi`, `out_msatoshi` and `fee` (use `in_msat`, `out_msat` and `fee_msat`)
Changelog-Deprecated: JSON-RPC: `listfunds` `outputs` `value` (use `amount_msat`)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-06-21 06:52:35 +09:30
Rusty Russell
01411d70be common: enforce that msat fields are called "xxx_msat".
Now we've fixed them all, make sure no new ones slip in!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-06-21 06:52:35 +09:30
Rusty Russell
a52bdeee01 common: add msat to sat convert helper.
We don't always want to round down, sometimes we want to fail.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-06-21 06:52:35 +09:30
Rusty Russell
36a2491a89 json: fix up msat amounts in non-_msat fields.
We had json_add_amount_msat_only(), which was designed to be used to
print out msat fields, if we had sats.

However, we misused it, so split it into the three different cases:
1. json_add_amount_sat_msat: We are using it correctly, with a field called
   xxx_msat.
2. json_add_amount_sats_deprecated: We were using it wrong, so deprecate
   the old field and create a new one which does end in _msat.
3. json_add_sats: we were using it to hand sats as a JSON parameter to an
   interface, where "XXXsat".

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: Plugins: `rbf_channel` and `openchannel2` hooks `their_funding` (use `their_funding_msat`)
Changelog-Deprecated: Plugins: `openchannel2` hook `dust_limit_satoshis` (use `dust_limit_msat`)
Changelog-Deprecated: Plugins: `openchannel` hook `funding_satoshis` (use `funding_msat`)
Changelog-Deprecated: Plugins: `openchannel` hook `dust_limit_satoshis` (use `dust_limit_msat`)
Changelog-Deprecated: Plugins: `openchannel` hook `channel_reserve_satoshis` (use `channel_reserve_msat`)
Changelog-Deprecated: Plugins: `channel_opened` notification `amount` (use `funding_msat`)
Changelog-Deprecated: JSON-RPC: `listtransactions` `msat` (use `amount_msat`)
Changelog-Deprecated: Plugins: `htlc_accepted` `forward_amount` (use `forward_msat`)
2022-06-21 06:52:35 +09:30
Rusty Russell
6e2a775ef2 common/param: support renaming options using "|<deprecatedname>".
This is *much* easier to do inside parsing than in the caller!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-06-21 06:52:35 +09:30
Michael Schmoock
55cf413fc3 wireaddr: moves wireaddr_arr_contains to wireaddr.h
...So it can be reused somewhere else

Changelog-None
2022-06-17 20:30:16 +09:30
Rusty Russell
7c8dc62035 channeld: take over gossip_rcvd_filter.c and is_msg_gossip_broadcast.
channeld is the only user of these functions, since it now streams
all gossip itself.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-06-17 14:14:02 +09:30
Jon Griffiths
572942c783 psbt: use DER encoded + sighash byte for PSBT_IN_PARTIAL_SIG items
Per BIP-0171, the signature map is of pubkey to "The signature as would
be pushed to the stack from a scriptSig or witness".

Fixes 5298

Changelog-Fixed: PSBT: Fix signature encoding to comply with BIP-0171.

Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
2022-06-09 18:28:35 +02:00
Christian Decker
74ddc15435 route: Do not require both directions to be active
This likely lead to a number of false errors when attempting to
route. We deemed a channel to be unusable as soon as either direction
isn't usable. This is bad since it excludes not only zeroconf
channels (which have different scids for the two directions), but it
also excludes any channel that we haven't seen an update from
yet. This was likely introduced when attemting to exclude nodes that
haven't sent a disable, but their peer has, but this is not necessary
as the unresponsive node would be marked as isolated by all its peers,
so we don't need to artificially mark a channel direction as disabled
when really we can't even enter the node to traverse the channel in
that direction.

Changelog-Fixed: routing: Fixed an issue where we would exclude the entire channel if either direction was disabled, or we hadn't seen an update yet.
2022-05-25 14:47:05 +09:30
Rusty Russell
685fa25756 Makefile: update bolts to include remote_pubkey change.
Only affects comments.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-05-19 09:47:32 +09:30
Rusty Russell
bf040c398b Makefile: update to BOLTs without zlib.
This contains a typo fix and a clarification on channel_type, but also
removes ZLIB.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-05-19 09:47:32 +09:30
Rusty Russell
9b944dbed4 common/gossip_store: add flag to *only* fetch "push"-marked messages.
These are the ones which are for our own channels (and our own node_announcement).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-04-21 06:13:55 +09:30
Michael Dance
f067e8c909 Changed external/libwally-core to test_build_fix
Combined with the following commit which is required to
update against changed libsecp256k1 APIs:

Updated deprecated function calls
2022-04-12 15:17:52 +09:30
Rusty Russell
ec72d89975 bolt11: reorder invoice production to match test vectors.
After this, we can exactly reproduce the vectors (in DEVELOPER mode).

1. Move payment_metadata position to match test vector.
2. Create flag to suppress `c` field production.
3. Some vectors put secret before payment_hash, hack that in.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-04-02 09:40:18 +10:30
Rusty Russell
7e789be0ea doc: update BOLTs to latest master.
Just typo fixes and the like.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-04-02 09:40:18 +10:30
Rusty Russell
7f89763f9e sendpay: add payment_metadata argument.
And document the missing arguments.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-04-02 09:40:18 +10:30
Rusty Russell
4718ee076c lightningd: metadata received support (log and decline).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-04-02 09:40:18 +10:30
Rusty Russell
e01abf0b34 bolt11: support payment_metadata.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-04-02 09:40:18 +10:30
Rusty Russell
2526e804f7 doc: big BOLT update to incorporate warnings language.
We do this (send warnings) in almost all cases anyway, so mainly this
is a textual update, but there are some changes:

1. Send ERROR not WARNING if they send a malformed commitment secret.
2. Send WARNING not ERROR if they get the shutdown_scriptpubkey wrong (vs upfront)
3. Send WARNING not ERROR if they send a bad shutdown_scriptpubkey (e.g. p2pkh in future)
4. Rename some vars 'err' to 'warn' to make it clear we send a warning.

This means test_option_upfront_shutdown_script can be made reliable, too,
and it now warns and doesn't automatically close channel.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-04-02 09:40:18 +10:30
Rusty Russell
9f06a59e3c shutdown: don't allow shutdown to p2pkh or p2sh addresses for anchor outputs.
This doesn't have an effect now (except in experimental mode), but it
will when we support anchors.  So we deprecate the use of those in the
close command too.

For experimental mode we have to avoid using p2pkh; adapt that test.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: JSON-RPC: `shutdown` no longer allows p2pkh or p2sh addresses.
2022-04-02 09:40:18 +10:30
Rusty Russell
7491af5495 doc: update BOLT references part1, including bolt11 test vectors with payment_secret.
The signatures on the new examples are sometimes different from what we produce though?
They're valid, however.

And one example has an unneeded feature 5-bit; it's not *wrong*, but
it's not optimal.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-04-02 09:40:18 +10:30
Rusty Russell
116a77f1be lightningd: Restore forwarding of legacy onions.
Partial revert of 43a833e405
"lightningd: remove support for legacy onion format."; we restore the
ability to decode legacy onions for forwarding, but not to generate them.
(We don't accept them properly since making payment_secret compulsory
anyway, so no real change there!)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: Protocol: ... but we still forward legacy HTLC onions for now.
2022-03-31 19:38:05 +10:30
Rusty Russell
20392ae526 connectd: restore obs2 onion support.
I removed these prematurely: we *haven't* had a release since
introducing them!

This consists of reverting d15d629b8b
"plugins/fetchinvoice: remove obsolete string-based API." and
plugins/fetchinvoice: remove obsolete string-based
API. "onion_messages: remove obs2 support."

Some minor changes due to updated fromwire_tlv API since they
were removed, but not much.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: REVERT: Removed backwards compat with onion messages from v0.10.1.
2022-03-29 10:55:12 +10:30
Rusty Russell
9e11ae1a0b plugins/topology: don't get upset if we hit deleted channel.
There's a race under CI, where a channel is deleted then we see the
channel_update in the gossip store.  We assumed this wouldn't happen,
but it can!

```
[gw1] [ 95%] FAILED tests/test_connection.py::test_multichan
[gw1] [ 95%] ERROR tests/test_connection.py::test_multichan
...
>           raise ValueError(str(errors))
E           ValueError:
E           Node errors:
E            - lightningd-3: had BROKEN messages
E            - lightningd-3: Node exited with return code 1
E           Global errors:
...
lightningd-3: 2022-03-28T00:11:42.160Z DEBUG   wallet: Owning output 0 100000sat (SEGWIT) txid 30616903feba1839a3834e2b3b6123759ce1fe0d76414ca77e2dbc17414772e0 CONFIRMED
lightningd-3: 2022-03-28T00:11:42.392Z DEBUG   hsmd: Client: Received message 5 from client
lightningd-3: 2022-03-28T00:11:42.393Z DEBUG   hsmd: new_client: 2
lightningd-3: 2022-03-28T00:11:42.398Z INFO    plugin-topology: Killing plugin: exited during normal operation
lightningd-3: 2022-03-28T00:11:42.400Z **BROKEN** plugin-topology: Plugin marked as important, shutting down lightningd!
...
----------------------------- Captured stderr call -----------------------------
topology: update for channel 105x1x1 not found!
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-29 10:50:01 +10:30
Rusty Russell
aad4495f56 delinvoice: allow desconly arg to only remove the description.
Means that field is now optional in JSON output.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `delinvoice` has a new parameter `desconly` to remove description.
2022-03-29 10:04:16 +10:30
Rusty Russell
ccaf04d268 invoice: add deschashonly parameter.
LNURL wants this so they can include images etc in descriptions.

Replaces: #4892
Changelog-Added: JSON-RPC: `invoice` has a new parameter `deschashonly` to put hash of description in bolt11.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-29 10:04:16 +10:30
Rusty Russell
7147fea2ff common: add const to accepted_extra_tlvs arg.
It was tlv_fields_valid that wanted a non-const: now that's gone, we
can make this correctly const.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-25 13:55:44 +10:30
Rusty Russell
83ee68ab06 common/tlvstream: put TLV checking back in the generic function.
Callers were supposed to call "tlv_fields_valid" after fromwire_tlv,
but few did.  Make this the default, and call the underlying function
directly where we want to be more flexible (one place).

This loses the ability to allow misordered fields, or to pass through
*any* even fields.  We restore that for special cases in the next
patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-25 13:55:44 +10:30
Rusty Russell
a770f51d0e tools/generate_wire.py: make functions allocate the TLV.
Requiring the caller to allocate them is ugly, and differs from
other types.

This means we need a context arg if we don't have one already.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-25 13:55:44 +10:30
Rusty Russell
fa0c29f959 tools/generate_wire.py: tlvs should start with tlv_
No more "towire_offer", but "towire_tlv_offer".

This means we double-up on the unfortunately-named `tlv_payload` inside
the onion, but we should rename that in the spec when we remove
old payloads.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-25 13:55:44 +10:30
Rusty Russell
7829f2eb06 onion_messages: remove obs2 support.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: Removed backwards compat with onion messages from v0.10.1.
2022-03-25 13:55:44 +10:30
Rusty Russell
726b6878d1 offers: import latest variant from draft.
In particular, this changes the name of a field in invoice_request:
`payer_signature` becomes simply `signature`.  So we allow both for
now, and send the old one unless deprecated_apis is disabled.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-25 13:55:44 +10:30
Rusty Russell
e36d4d1143 devtools/decodemsg: fix printing of wireaddr.
printwire_ routines are supposed to print!  And they're only needed
inside devtools/.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-25 13:55:44 +10:30
Rusty Russell
6f7d51ee68 common/onion: always fill in failtlvtype and failtlvpos parameters.
When we did fill them in, we filled them in wrong: the offset should be
the offset in the message, not the field number!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-25 13:55:44 +10:30
Vincenzo Palazzo
53806d1abd cli: make the command line more user friendly.
Also has to fix up tests.

Changelog-Fixed: cli doesn't required anymore to confirm the password if the `hsm_secret` is already encrypted.

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2022-03-24 09:53:38 +10:30
Rusty Russell
cf55d40eca psbt: fix dual-funding memleak.
This happened occasionally in tests/test_opening.py::test_rbf_fails_to_broadcast:
we overwrote the witness stack without freeing the old one.

```
lightningd-2: 2022-03-18T02:23:32.113Z **BROKEN** 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-dualopend-chan#1: MEMLEAK: 0x559c375d9df8
lightningd-2: 2022-03-18T02:23:32.113Z **BROKEN** 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-dualopend-chan#1:   label=wally_tal
lightningd-2: 2022-03-18T02:23:32.114Z **BROKEN** 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-dualopend-chan#1:   backtrace:
lightningd-2: 2022-03-18T02:23:32.114Z **BROKEN** 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-dualopend-chan#1:     ccan/ccan/tal/tal.c:442 (tal_alloc_)
lightningd-2: 2022-03-18T02:23:32.114Z **BROKEN** 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-dualopend-chan#1:     ccan/ccan/tal/tal.c:471 (tal_alloc_arr_)
lightningd-2: 2022-03-18T02:23:32.114Z **BROKEN** 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-dualopend-chan#1:     common/setup.c:13 (wally_tal)
lightningd-2: 2022-03-18T02:23:32.114Z **BROKEN** 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-dualopend-chan#1:     ../../../libwally-core/src/internal.c:285 (wally_calloc)
lightningd-2: 2022-03-18T02:23:32.114Z **BROKEN** 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-dualopend-chan#1:     ../../../libwally-core/src/transaction.c:234 (wally_tx_witness_stack_init_alloc)
lightningd-2: 2022-03-18T02:23:32.114Z **BROKEN** 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-dualopend-chan#1:     ../../../libwally-core/src/psbt.c:1119 (pull_psbt_input)
lightningd-2: 2022-03-18T02:23:32.114Z **BROKEN** 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-dualopend-chan#1:     ../../../libwally-core/src/psbt.c:1411 (wally_psbt_from_bytes)
lightningd-2: 2022-03-18T02:23:32.114Z **BROKEN** 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-dualopend-chan#1:     bitcoin/psbt.c:722 (psbt_from_bytes)
lightningd-2: 2022-03-18T02:23:32.115Z **BROKEN** 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-dualopend-chan#1:     bitcoin/psbt.c:753 (fromwire_wally_psbt)
lightningd-2: 2022-03-18T02:23:32.115Z **BROKEN** 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-dualopend-chan#1:     openingd/dualopend_wiregen.c:246 (fromwire_dualopend_reinit)
lightningd-2: 2022-03-18T02:23:32.115Z **BROKEN** 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-dualopend-chan#1:     openingd/dualopend.c:3855 (main)
lightningd-2: 2022-03-18T02:23:32.115Z **BROKEN** 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-dualopend-chan#1:   parents:
lightningd-2: 2022-03-18T02:23:32.115Z **BROKEN** 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-dualopend-chan#1:     struct wally_psbt
lightningd-2: 2022-03-18T02:23:32.115Z **BROKEN** 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-dualopend-chan#1:     openingd/dualopend.c:3804:struct state
lightningd-2: 2022-03-18T02:23:32.115Z **BROKEN** 0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba932
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-21 10:19:50 +10:30
Rusty Russell
65be18d355 memleak: handle libwally allocations better.
Things allocated by libwally all get the tal_name "wally_tal",
which cost me a few hours trying to find a leak.

In the case where we're making one of the allocations the parent
of the others (e.g. a wally_psbt), we can do better: supply a name
for the tal_wally_end().

So I add a new tal_wally_end_onto() which does the standard
tal_steal() trick, and also changes the (typechecked!) name.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-21 10:19:50 +10:30
Rusty Russell
278ea6c8ac memleak: make sure we catch children which are also "notleak".
By not recursing into "notleak" children, we could miss other "notleak"
pointers:

```
2022-03-15T05:25:33.9759500Z lightningd-1: 2022-03-15T05:00:15.998Z **BROKEN** connectd: MEMLEAK: 0x55b29c3d0cc8
2022-03-15T05:25:33.9759901Z lightningd-1: 2022-03-15T05:00:16.003Z **BROKEN** connectd:   label=common/timeout.c:22:struct oneshot **NOTLEAK**
2022-03-15T05:25:33.9760195Z lightningd-1: 2022-03-15T05:00:16.003Z **BROKEN** connectd:   backtrace:
2022-03-15T05:25:33.9760555Z lightningd-1: 2022-03-15T05:00:16.003Z **BROKEN** connectd:     ccan/ccan/tal/tal.c:442 (tal_alloc_)
2022-03-15T05:25:33.9760905Z lightningd-1: 2022-03-15T05:00:16.003Z **BROKEN** connectd:     common/timeout.c:22 (new_reltimer_)
2022-03-15T05:25:33.9761272Z lightningd-1: 2022-03-15T05:00:16.003Z **BROKEN** connectd:     connectd/multiplex.c:558 (set_closing_timer)
2022-03-15T05:25:33.9761647Z lightningd-1: 2022-03-15T05:00:16.003Z **BROKEN** connectd:     connectd/multiplex.c:586 (write_to_peer)
2022-03-15T05:25:33.9761986Z lightningd-1: 2022-03-15T05:00:16.003Z **BROKEN** connectd:     ccan/ccan/io/io.c:59 (next_plan)
2022-03-15T05:25:33.9762335Z lightningd-1: 2022-03-15T05:00:16.004Z **BROKEN** connectd:     ccan/ccan/io/io.c:435 (io_do_always)
2022-03-15T05:25:33.9762692Z lightningd-1: 2022-03-15T05:00:16.004Z **BROKEN** connectd:     ccan/ccan/io/poll.c:304 (handle_always)
2022-03-15T05:25:33.9763035Z lightningd-1: 2022-03-15T05:00:16.004Z **BROKEN** connectd:     ccan/ccan/io/poll.c:385 (io_loop)
2022-03-15T05:25:33.9763376Z lightningd-1: 2022-03-15T05:00:16.004Z **BROKEN** connectd:     connectd/connectd.c:2131 (main)
2022-03-15T05:25:33.9763645Z lightningd-1: 2022-03-15T05:00:16.004Z **BROKEN** connectd:   parents:
2022-03-15T05:25:33.9764020Z lightningd-1: 2022-03-15T05:00:16.004Z **BROKEN** connectd:     ccan/ccan/io/io.c:91:struct io_conn **NOTLEAK**
2022-03-15T05:25:33.9764471Z lightningd-1: 2022-03-15T05:00:16.004Z **BROKEN** connectd:     connectd/connectd.c:2104:struct daemon
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-03-20 08:42:57 +10:30
Rusty Russell
43a833e405 lightningd: remove support for legacy onion format.
As per proposal in https://github.com/lightning/bolts/pull/962

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: protocol: support for legacy onion format removed, since everyone supports the new one.
2022-03-18 09:20:11 +10:30
Ken Sedgwick
36466af3eb hsmd: Add fields to hsmd_sign_{,remote_}commitment_tx for validating signers 2022-03-16 12:06:21 +10:30
Michael Schmoock
b930b8c548 wireaddr: adds wireaddr_eq_without_port and wireaddr_cmp_type
Adds wireaddr_eq_without_port so it can be used later.
Moves wireaddr_cmp_type from connectd.c to this file, so it can be reused later.
2022-03-11 16:42:45 +10:30
Ken Sedgwick
8f56f96801 hsmd: Add wallet index metadata to existing messages 2022-03-10 10:22:29 +10:30
Ken Sedgwick
8876c03791 hsmd: Add PSBT keypath utility functions 2022-03-10 10:22:29 +10:30
Ken Sedgwick
1e1948bfd2 hsmd: Add redeemscripts to anchor outputs 2022-03-10 10:22:29 +10:30
niftynei
b33868f7c2 coin_mvt: only chain moves have a blockheight
channel moves don't have blockheights
2022-03-05 15:12:12 +10:30
niftynei
7add7ca199 json: reverse parse a coin_mvt tag back into an enum 2022-03-05 15:12:12 +10:30
niftynei
24a1c91045 coin_mvt: report the number of outputs on a channel close tx
The bookkeeper needs to know how many outputs to expect before we can
consider a channel resolved onchain.
2022-03-05 15:12:12 +10:30
Rusty Russell
dd495b2b21 memleak: fix handling of excluded pointers.
We often hand an exclude pointer (usually the current command) to
memleak.  But when we encountered this we would stop iterating, rather
than just ignore it: this means we would often ignore significant siblings.

In particular, fixing this (which has always been there) reveals many
previously-undetected leaks.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-27 07:44:26 +10:30
Rusty Russell
ea71e9fe5b memleak: fix handling of notleak pointers.
We were ignoring the *parent* of the notleak pointers, not the notleak
pointer itself!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-27 07:44:26 +10:30
Rusty Russell
fe86395e7c memleak: restore functionality accidentally removed.
Accidentally removed in 6c9b752751.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-27 07:44:26 +10:30
Rusty Russell
9bf2aa65cd common/param: don't keep around the temporary array off cmd.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-27 07:44:26 +10:30
Rusty Russell
3218e5332f common/param: fix leak with opt_param_def.
We allocate the default, then callback allocates over the top.  Mark
params with a default, so we can free that when it's called.

(We can't do this generally, since not all param args are actually
pointers to pointers, though opt_param_def has to be).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-27 07:44:26 +10:30
Rusty Russell
f0ea4d60b9 onchaind: fix minor leaks.
Not actually leaks, but they do live longer than they need.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-27 07:44:26 +10:30
Rusty Russell
4b7d2dc5b8 common/bolt11: make decoded routes hang off routes arr, not b11.
This causes weirdness in pay, which tal_steal's b11->routes and expects
to get it all.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-27 07:44:26 +10:30
William Casarin
2b92ac4236 bolt11: mark when expiry is decoded
It looks like the x tag isn't marked when parsed either?
2022-02-22 09:58:48 +10:30
William Casarin
af16b9b9f4 bolt11: mark when we decode min_final_cltv_expiry
It looks like decode_c doesn't set have_c unlike the other decode_
methods. At the start of the function, decode_c checks have_c to see if
it's set, but it is never set. It seems like this could allow for
duplicate c tags, which is probably not intended.

Signed-off-by: William Casarin <jb55@jb55.com>
2022-02-22 09:58:48 +10:30
Michael Schmoock
38e2abf68a peer_exchange: set, read and log remote_addr
Changelog-Added: Protocol: set remote_addr on init tlvs
2022-02-22 05:45:47 +10:30
Rusty Russell
b47930f395 gossip_store: handle compacted gossip_store correctly.
Don't send EOF marker to peer, e.g. in tests/test_gossip.py::test_gossip_store_compact:

```
lightningd-2: 2022-01-24T03:34:22.925Z DEBUG   connectd: gossip_store at end, new fd moved to 1875
lightningd-2: 2022-01-24T03:34:22.933Z DEBUG   035d2b1192dfba134e10e540875d366ebc8bc353d5aa766b80c090b39c3a5d885d-connectd: Sending gossip INVALID 4105
lightningd-2: 2022-01-24T03:34:22.933Z DEBUG   035d2b1192dfba134e10e540875d366ebc8bc353d5aa766b80c090b39c3a5d885d-channeld-chan#2: peer_in WIRE_WARNING
lightningd-2: 2022-01-24T03:34:22.941Z DEBUG   035d2b1192dfba134e10e540875d366ebc8bc353d5aa766b80c090b39c3a5d885d-connectd: peer_out INVALID 4105
lightningd-2: 2022-01-24T03:34:22.949Z DEBUG   035d2b1192dfba134e10e540875d366ebc8bc353d5aa766b80c090b39c3a5d885d-channeld-chan#2: billboard perm: Received warning channel 2c7cf1dc9dada7ed14f10c78ade8f0de907c1b70e736c12ff6f7472dc69c3db3: Peer sent unknown message 4105 (INVALID 4105)
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-08 11:15:52 +10:30
Rusty Russell
3c5d27e3e9 subdaemons: remove gossipd fd from per-peer daemons.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-08 11:15:52 +10:30
Rusty Russell
f14c0ef76a common: make sure we hand through peer for io logging.
This is mainly useful for connectd.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-08 11:15:52 +10:30
niftynei
43f1fb4561 coin_moves: an htlc_fulfill claimed by them is a credit to "external"
The money moved into an external account here.
2022-02-07 13:02:09 +10:30
niftynei
39572c1aaf coin_moves: remove unused method
This wasn't used anywhere, so we remove it.
2022-02-07 13:02:09 +10:30
Rusty Russell
8a8d7c4243 elements: unify overhead calculation.
And in particular, fix onchaind grinding code which used the
actual number of inputs and outputs (which already includes the
fee output); that breaks with the next patch which fixes other
calculations.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-01-27 12:22:36 +01:00
niftynei
4dafeede5c coin moves: notify when we make deposits to external accounts
The blockheight is zero though, since these aren't included in a block
yet.

We also don't issue an 'external' deposit event if we can tell that the
address you're sending to actually belongs to our wallet (we'll issue a
deposit event when it gets included in a block)
2022-01-26 13:34:45 +10:30
niftynei
7f511b9ba9 coin moves: external deposits should be *deposits*
Report how much the deposit was for.
2022-01-26 13:34:45 +10:30
niftynei
ff4ae8b5f4 coin_moves: add an 'originating_account' field
If a coin move concerns an external account, it's really useful to know
which 'internal' account initiated the transfer.

We're about to add a notification for withdrawals, so we can use this to
track wallet pushes to outside addresses

Changelog-Added: JSONRPC: `coin_movement` to 'external' accounts now include an 'originating_account' field
2022-01-26 13:34:45 +10:30
Rusty Russell
2b5d25c851 common: remove stderr debug in is_valid_witnessprog.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-01-25 06:26:52 +10:30
Rusty Russell
be299c0d59 common/socket_close: remove now only connectd talks to peer.
connectd does this internally now using ccan/io, with appropriate
credit for ZmnSCPxj who wrote this code in the first place.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-01-25 06:26:52 +10:30
Rusty Russell
f6847f44f6 subds: remove "ignore error" from old LND nodes.
This was put in late 2019, and @t-bast says Eclair doesn't ignore their
errors and has had no issues.

It also conflicts with https://github.com/lightning/bolts/pull/932
which suggests you *should* fail when you receive an error.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-01-25 06:26:52 +10:30
Rusty Russell
2ab5603624 peer subds: ignore failed writes.
In the case where the peer sends an error (and hangs up) immediately
after init, connectd *doesn't actually read the error* (even after all the
previous fixes so it actually receives the error!).

This is because to tried to first write WIRE_CHANNEL_REESTABLISH, and
that fails, so it never tries to read.  Generally, we should ignore
write failures; we'll find out if the socket is closed when we read
nothing.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-01-20 15:24:06 +10:30
Rusty Russell
d51fb5207a msg_queue: don't allow magic MSG_PASS_FD message for peers.
msg_queue was originally designed for inter-daemon comms, and so it has
a special mechanism to mark that we're trying to send an fd.  Unfortunately,
a peer could also send such a message, confusing us!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-01-20 15:24:06 +10:30
Rusty Russell
a93c49ca65 connectd: implement @ correctly.
dev_blackhole_fd was a hack, and doesn't work well now we are async
(it worked for sync comms in per-peer daemons, but now we could sneak
through a read before we get to the next write).

So, make explicit flags and use them.  This is much easier now we
have all peer comms in one place.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-01-20 15:24:06 +10:30
Rusty Russell
26b9384fd0 various: minor cleanups from Christian's review.
More significant things have been folded.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-01-20 15:24:06 +10:30
Rusty Russell
6d4c56e8b6 connectd: put more stuff into struct gossip_state.
We're the only ones who use it now, so put our fields inside it and
make it local.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-01-20 15:24:06 +10:30
Rusty Russell
6115ed02e8 subdaemons: don't stream gossip_store at all.
We now let gossipd do it.

This also means there's nothing left in 'struct per_peer_state' to
send across the wire (the fds are sent separately), so that gets
removed from wire messages too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-01-20 15:24:06 +10:30
Rusty Russell
029d65cf2e connectd: serve gossip_store file for the peer.
We actually intercept the gossip_timestamp_filter, so the gossip_store
mechanism inside the per-peer daemon never kicks off for normal connections.

The gossipwith tool doesn't set OPT_GOSSIP_QUERIES, so it gets both, but
that only effects one place.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-01-20 15:24:06 +10:30
Rusty Russell
1f608acd4e common: add routine for absolute timeouts (vs. relative).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-01-20 15:24:06 +10:30
Rusty Russell
e37a638c0c connectd: do nagle by packet type.
channeld can't do it any more: it's using local sockets.  Connectd
can do it, and simply does it by type.

Amazingly, on my machine the timing change *always* caused
test_channel_receivable() to fail, due to a latent race.

Includes feedback from @cdecker.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-01-20 15:24:06 +10:30
Rusty Russell
7a514112ec connectd: do dev_disconnect logic.
As connectd handles more packets itself, or diverts them to/from gossipd,
it's the only place we can implement the dev_disconnect logic.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-01-20 15:24:06 +10:30
Rusty Russell
9c0bb444b7 per_peer_state: remove struct crypto_state
Now that connectd does the crypto, no need to hand around crypto_state.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-01-20 15:24:06 +10:30
Rusty Russell
ce8b69401c peer_io: replace crypto_sync in daemons, use normal wire messages.
Now connectd is doing the crypto, we can use normal wire io.  We
create helper functions to clearly differentiate between "peer" comms
and intra-daemon comms though.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-01-20 15:24:06 +10:30
Rusty Russell
a2b3d335bb connectd: do decryption for peers.
We temporarily hack to sync_crypto_write/sync_crypto_read functions to
not do any crypto, and do it all in connectd.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-01-20 15:24:06 +10:30
Rusty Russell
ae4669f77f common/coin_mvt: make it clear we're using Lightning HRP not BIP 173.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-30 14:36:55 +10:30
Rusty Russell
425a7af512 common/coin_mvt: clean up API a little.
1. tal_strndup(.., str, strlen(str)) == tal_strdup()
2. tal_strdup also takes(), so document that.
3. Avoid passing 'struct sha256' on the stack: use ptr.
4. Generally, structures shouldn't keep pointers to things they don't own.
   In this case, mvt->node_id.
5. Make payment_hash a pointer, since NULL is more natural than an all-zero
   hash.

And add NON_NULL_ARGS() to the functions; it's cumbersome, but make it
fairly clear what params are optional.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-30 14:36:55 +10:30
Rusty Russell
967ffbfbcb global: use tal_dup_or_null().
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-30 14:36:55 +10:30
Rusty Russell
8759641b06 ccan: update to get new helpers in ccan/tal
tal_dup_talarr() is simply stolen from common/utils, but tal_dup_or_null is new.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-30 14:36:55 +10:30
Rusty Russell
5aa8663c98 common/socket_close.c: remove obsolete comment.
Put more inline comments, and also preserve errno properly from read.

Suggested-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-30 09:50:40 +10:30
Rusty Russell
70ed47d77a channeld: add dev-disable-commit-after instead of dev-disconnect -nocommit
It was always a hack, but an impossible one once connectd will be
interpreting dev-disconnect!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-30 09:50:40 +10:30
Rusty Russell
888745be16 dev_disconnect: remove @ marker.
Once connectd is doing this, we can't close as soon as we send,
and in fact we can't do 'fail write' either.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-30 09:50:40 +10:30
Rusty Russell
2fb58772a4 common/socket_close: don't wait forever.
The "read until closed" trick doesn't work if the other end doesn't
close (as found in the next patch, where we use DEV_DISCONNECT_DISABLE_AFTER).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-30 09:50:40 +10:30
Rusty Russell
35e3c1866e common: generalize extract_channel_id().
connectd is going to end up using this do demux; make it fast and complete.

Fixing this reveals a problem in openingd: it now extracts the channel_id
from funding_signed (which is where we transition off the temporary), and
gets upset.  So fix that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-30 09:50:40 +10:30
Rusty Russell
6ce7e7ea60 common: is_msg_gossip_broadcast() for putting things in filter.
We don't need the other "gossip" messages in our echo-suppression filter.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-30 09:50:40 +10:30
niftynei
ec991e3af7 balance snap: first pass 2021-12-28 04:42:42 +10:30
niftynei
5570c6da08 coin_moves: cleanup tags list, update docs, remove unused method
We don't use journal_entry any more; remove it and update all of the
remaining tags (plus docs)
2021-12-28 04:42:42 +10:30
niftynei
8225a9decf coin_mvt: log events for pushes/lease_fees for leased channels
We need to stash/save the amount of the lease fees on a leased channel,
we do this by re-using the 'push' amount field on channel (which is
technically correct, since we're essentially pushing the fee amount to
the peer).

Also updates a bit of how the pushes are accounted for (pushed to now
has an event; their channel will open at zero but then they'll
immediately register a push event).

Leases fees are treated exactly the same as pushes, except labeled
differently.

Required adding a 'lease_fee' field to the inflights so we keep track of
the fee for the lease until the open happens.
2021-12-28 04:42:42 +10:30
niftynei
29c6718297 coin_mvt: record new 'fees' field on htlc channel moves
We record the amount of fees collected for a routed payment. For
simplicity's sake on the data agg side, we record the fee payment on
*BOTH* the incoming htlc and the outgoing htlc. Note that this results
in double counting if you add up the fees from both an in-routed and
out-routed payment.
2021-12-28 04:42:42 +10:30
niftynei
b6463174d6 coin moves: turn 'tag' into 'tags' array, add OPENER tag
Channels that the node has hopened will now be tagged with 'opener' in a
list of tags.
2021-12-28 04:42:42 +10:30
niftynei
4506f639fa coin_moves: remove 'index' for moves; bump version
Get rid of the 'movement_idx', since we replay events now.

Since we're removing a field from the 'coin_movement' event emission, we
bump the version type.

Changelog-Updated: `coin_movements` events have been revamped and are now on version 2.
2021-12-28 04:42:42 +10:30
niftynei
d2c4d4aec2 coin_mvts: rewrite how onchain events are recorded, update tests
The old model of coin movements attempted to compute fees etc and log
amounts, not utxos. This is not as robust, as multi-party opens and dual
funded channels make it hard to account for fees etc correctly.

Instead, we move towards a 'utxo' view of the onchain events. Every
event is either the creation or 'destruction' of a utxo. For cases where
the value of the utxo is not (fully) debited/credited to our account, we
also record the output_value. E.g. channel closings spend a utxo who's
entire value we may not own.

Since we're now tracking UTXOs onchain, we can now do more complex
assertions about the onchain footprint of them. The integration tests
have been updated to now use more 'chain aware' assertions about the
ending state.
2021-12-28 04:42:42 +10:30
niftynei
3b3b8fb3ec coin_mvt: add "output value" to coin movement
When we log "external" wallet events, it's helpful to know what the
value of that output is, so let's log it
2021-12-28 04:42:42 +10:30
niftynei
b7ca45514c coin_moves: fix account name serialization
Switch over to a more robust method of serializing strings
2021-12-28 04:42:42 +10:30
niftynei
5e1d8d6ad6 amount: helper for msat == 0
Shorthand for checking if value is zero
2021-12-28 04:42:42 +10:30
niftynei
43ade318d9 coin-move: the txid is now optional but outpoints are not
we're pivoting from a txid based world to a outpoint based world. every
coin movement (onchain) will correspond with a outpoint; only the spend
of an outpoint will have a tx_txid
2021-12-28 04:42:42 +10:30
niftynei
737772f1ca coin-mvt: remove all the chain fees tracking
this will be impllicit going forward.
2021-12-28 04:42:42 +10:30
niftynei
e791efe23b coin-mvt: remove all the spend-tracking
We're not going to do 'spend tracks' any more; instead we'll emit an
event whenever an output is included in a broadcast tx

(even if the broadcast fails!!)
2021-12-28 04:42:42 +10:30
niftynei
9b1a0f9b7b coin-moves: remove penalty + chain fee types
FIXME: still has coin tracking stuff!
2021-12-28 04:42:42 +10:30
Vincenzo Palazzo
43ff949ea7 lightningd: support hsm error code
Suggested-by: Rusty Russell
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
Changelog-Changed: Support hsm specific error error code in lightning-cli
2021-12-15 12:24:54 +10:30
lightning-developer
bf71d250a7 Removed FIXME to match bolts PR 918
If PR 918 is merged to the bolts this requirement will be dropped from the spec and we do not need to have a FIXME in the code base.
2021-12-14 11:26:47 +10:30
ZmnSCPxj jxPCSnmZ
d088288daa bitcoin/chainparams.h: Split BIP173 name into onchain and Lightning HRPs.
Fixes: #4937
2021-12-14 11:21:35 +10:30
Simon Vrouwe
605fda7214 common: cleanup unsused parameter in timer_expired() 2021-12-14 09:33:10 +10:30
Rusty Russell
786732601c common: remove unused functions or make static.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-06 10:05:39 +10:30
Rusty Russell
d9968bbc0c bitcoin: remove unused functions, or make static.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-06 10:05:39 +10:30
Rusty Russell
c3899806ee common: remove unused io_lock.c
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-06 10:05:39 +10:30
Rusty Russell
4ffda340d3 check: make sure all files outside contrib/ include "config.h" first.
And turn "" includes into full-path (which makes it easier to put
config.h first, and finds some cases check-includes.sh missed
previously).

config.h sets _GNU_SOURCE which really needs to be done before any
'#includes': we mainly got away with it with glibc, but other platforms
like Alpine may have stricter requirements.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-06 10:05:39 +10:30
Matt Whitlock
1f79aad830 common/utils: introduce tmpdir_mkstemp; use it
Various unit tests were creating temporary files unconditionally in /tmp
and were not cleaning up after themselves. Introduce a new variant of
mkstemp(3p) that respects the TMPDIR environment variable, and use it in
the offending unit tests. This allows each test run to use a dedicated
TMPDIR that can be cleaned up after the run.

Changelog-None

Signed-off-by: Matt Whitlock <c-lightning@mattwhitlock.name>
2021-12-06 09:20:27 +10:30
Rusty Russell
b45544c659 options: fix handling of wildcard (allproto) address.
We treated ':' as an empty DNS name in EXPERIMENTAL, which is wrong.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-05 16:43:09 +01:00
Michael Schmoock
94cb42bee6 status: do not log ping and pong packets
Debug logging of ping and pong packets turned out to be
too excessive since this is now done on all channels
and without config option.

Changelog-None
2021-12-05 14:57:53 +10:30
Andrew Toth
384c359c79 refactor: move exclude parsing to json_tok 2021-12-04 21:37:42 +10:30
Rusty Russell
132ac0b1b4 blinding: check against two of the test vectors in the PR.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-01 05:44:28 +10:30
Rusty Russell
296437c655 bolt12: remove deprecated chains fields
We also move recurrence fields into a separate spec patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-01 05:44:28 +10:30
Rusty Russell
163d3a16f4 doc: clean up offers bolt quotes
As of 2b923a0367c5f9154fcec706e3302cc4658dd889.

Recurrence quotes need to be marked separately, since they're no longer
in offers main bolt.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-01 05:44:28 +10:30
Rusty Russell
894f1841b4 features: don't announce onion_messages in bolt11 invoices.
As noted by Joost.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-01 05:44:28 +10:30
Rusty Russell
4c63fedce9 test: make an onionmessage test vector.
This builds on the enctlv vectors, but actually goes all the way
to creating a modern onionmessage.

Thanks to Thomas H for corrections!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-01 05:44:28 +10:30
Rusty Russell
e02e0a197d common/test/run-blindedpath_enctlv: use modern functions to create test vector.
And include the final destination enctlv.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-01 05:44:28 +10:30
Rusty Russell
b3af5f5a2c spec: import latest onionmessage spec, based on routeblinding.
This is from 6e99c5feaf60cb797507d181fe583224309318e9

We renamed the enctlv field to encrypted_recipient_data in the spec, and the
new onion_message is message 513.  We don't handle it until the next patch.

Two renames:
1. blinding_seed -> blinding_point.
2. enctlv -> encrypted_recipient_data.

We don't do a compat cycle for our JSON APIs for these experimental
features only used by our own plugins, we just rename.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-01 05:44:28 +10:30
Rusty Russell
1ec6346f3d common: rename current onion message structures to obs2_.
Yes, we changed the spec again.  Hopefully for the last time!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-01 05:44:28 +10:30
Rusty Russell
b74848f6f6 common: remove support for pre v0.10.2 onionmessages.
Temporarily disable sendpay_blinding test which uses obsolete onionmsg;
there's still some debate on the PR about how blinded HTLCs will work.

Changelog-EXPERIMENTAL: onionmessage: removed support for v0.10.1 onion messages.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-01 05:44:28 +10:30
Simon Vrouwe
4dded23cd3 hsmtool: use flag TCSANOW when disabling tty ECHO, fixes rare hang in test_hsm*
No idea why TCSAFLUSH was used, could not find anything in PR comments.
Also cannot explain exactly what causes the problem, but the hang can be reproduced
*with* TCSAFLUSH and not with TCSANOW.

According to termios doc:
TCSANOW
    the change occurs immediately.
TCSAFLUSH
    the change occurs after all output written to the object referred by fd has been
    transmitted, and all input that has been received but not read will be discarded
    before the change is made.
2021-11-30 13:34:44 +10:30
Simon Vrouwe
0388314ef8 JSON RPC: Calls made in shutdown loop receive error code -5: LIGHTNINGD_SHUTDOWN 2021-11-30 13:34:44 +10:30
Michael Schmoock
01e8a523e9 bolt7: allow announcement of ADDR_TYPE_DNS 2021-11-30 09:38:17 +10:30
Michael Schmoock
25bd09716f wireaddr: adds helper is_ipaddr, is_toraddr and is_dnsaddr 2021-11-30 09:38:17 +10:30
Rusty Russell
c1f534a159 backtrace: avoid duplicate backtrace objects.
This happened in my tal_dump(), and I couldn't see how we ended up
with object having more than one "backtrace".  Adding asserts that we
never added a second backtrace didn't trigger.

Finally I wondered if we were tal_steal() backtraces, and sure enough
we do that blinding in one place: libwally wrapping.  So fix that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-11-27 10:38:13 +10:30
Rusty Russell
6c9b752751 memleak: make notleak use the tal name instead of a boutique struct.
This lets us mark it directly.

Get rid of long-unused "notleaks" member of struct lightningd too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-11-27 10:38:13 +10:30
Rusty Russell
57328fe59e tests: use common_setup/common_shutdown to avoid leaks.
It also does more checks (like taken() checks).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-11-17 10:51:48 +10:30
Rusty Russell
68043c2e8c common: clean up autodata in common_shutdown().
valgrind locally complains about the allocations in autodata leaking:

```
==138200== 16 bytes in 1 blocks are still reachable in loss record 1 of 2
==138200==    at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==138200==    by 0x10D41A: autodata_register_ (autodata.c:20)
==138200==    by 0x10E7B8: register_autotype_type_to_string (type_to_string.h:79)
==138200==    by 0x10F5CA: register_one_type_to_string0 (block.c:259)
==138200==    by 0x19734C: __libc_csu_init (in /home/rusty/devel/cvs/lightning/common/test/run-route-specific)
==138200==    by 0x4A3D03F: (below main) (libc-start.c:264)
==138200== 
==138200== 176 bytes in 1 blocks are still reachable in loss record 2 of 2
==138200==    at 0x483DFAF: realloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==138200==    by 0x10D472: autodata_register_ (autodata.c:26)
==138200==    by 0x122D37: register_autotype_type_to_string (type_to_string.h:79)
==138200==    by 0x122F1F: register_one_type_to_string0 (node_id.c:50)
==138200==    by 0x19734C: __libc_csu_init (in /home/rusty/devel/cvs/lightning/common/test/run-route-specific)
==138200==    by 0x4A3D03F: (below main) (libc-start.c:264)
==138200== 
make: *** [Makefile:638: unittest/common/test/run-route-specific] Error 7
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-11-17 10:51:48 +10:30
Rusty Russell
35c7f56c73 common: use ccan/base64 instead of libsodium.
For consistency.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-11-14 18:49:46 +01:00
Rusty Russell
ff276e958b common: test that our sodium-based base64 encoding is identitcal to ccan's.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-11-14 18:49:46 +01:00
Rusty Russell
b2c762969c wireaddr: clean up tor parsing.
blob[] is really a string from the commandline; leave it as a char.

And parsing is much simpler than this code makes it seem!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-11-14 18:49:46 +01:00
Rusty Russell
9d18180172 lightningd: really do allow two Torv3 addresses.
This surprised me, since the CHANGELOG for [0.8.2] said:

	We now announce multiple addresses of the same type, if given. ([3609](https://github.com/ElementsProject/lightning/pull/3609))

But it lied!

Changelog-Fixed: We really do allow providing multiple addresses of the same type.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-11-14 18:49:46 +01:00
Rusty Russell
2f247c7bfb torv2: remove support for advertizing and connecting.
October was the date Torv2 is no longer supported by the Tor Project;
it will probably not work at all by next release, so we should remove
it now even though it's not quite the 6 months we prefer for
deprecation cycles.

I still see 110 nodes advertizing Torv2 (vs 10,292 Torv3); we still
parse and display it, we just don't advertize or connect to it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-11-14 18:49:46 +01:00
Rusty Russell
ba77fce453 websocket: switch to address type 6, since DNS will take 5.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-28 15:51:11 +02:00
niftynei
cad082a763 channeld: add in RFC notes for max_htlc_dust_exposure_msat
And update some behavior to check both sides on receipt of a
update_fee, as per the proposed spec.

https://github.com/lightningnetwork/lightning-rfc/pull/919
2021-10-23 12:59:13 +02:00
niftynei
b193eb06d3 dusty-htlcs: enforce limit on dusty htlcs
for every new added htlc, check that adding it won't go over our 'dust
budget' (which assumes a slightly higher than current feerate, as this
prevents sudden feerate changes from overshooting our dust budget)

note that if the feerate changes surpass the limits we've set, we
immediately fail the channel.
2021-10-23 12:59:13 +02:00
niftynei
42e40c1ced htlcs: add flag to 'fail immediately'
If we're over the dust limit, we fail it immediatey *after* commiting
it, but we need a way to signal this throughout the lifecycle, so we add
it to htlc_in struct and persist it through to the database.

If it's supposed to be failed, we fail after the commit cycle is
completed.
2021-10-23 12:59:13 +02:00
niftynei
1fe829c546 lightningd: new option for htlc dust limit
To reduce the surface area of amount of a channel balance that can be
eaten up as htlc dust, we introduce a new config
'--max-dust-htlc-exposure-msat', which sets the max amount that any
channel's balance can be added as dust

Changelog-Added: config: new option --max-dust-htlc-exposure-msat, which limits the total amount of sats to be allowed as dust on a channel
2021-10-23 12:59:13 +02:00
Rusty Russell
0b3c79b3c2 common/dijkstra: hand channel direction to path_score callback.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-22 16:04:37 +02:00
ZmnSCPxj jxPCSnmZ
5356267f15 *: Use new closefrom module from ccan.
This also inadvertently fixes a latent bug: before this patch, in the
`subd` function in `lightningd/subd.c`, we would close `execfail[1]`
*before* doing an `exec`.
We use an EOF on `execfail[1]` as a signal that `exec` succeeded (the
fd is marked CLOEXEC), and otherwise use it to pump `errno` to the
parent.
The intent is that this fd should be kept open until `exec`, at which
point CLOEXEC triggers and close that fd and sends the EOF, *or* if
`exec` fails we can send the `errno` to the parent process vua that
pipe-end.

However, in the previous version, we end up closing that fd *before*
reaching `exec`, either in the loop which `dup2`s passed-in fds (by
overwriting `execfail[1]` with a `dup2`) or in the "close everything"
loop, which does not guard against `execfail[1]`, only
`dev_disconnect_fd`.
2021-10-22 13:17:37 +02:00
Rusty Russell
b013b3ab0c patch websocket-address-support.patch 2021-10-22 11:56:30 +02:00
Rusty Russell
3018113012 common: reltimer_arg to access a timer argument.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-21 20:05:54 +02:00
Rusty Russell
c503232cde common: use bitcoin_outpoint.
I started pulling this thread, and the entire codebase got unravelled.

Oh well, it's done now!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-15 12:09:36 +02:00
Rusty Russell
e7a8a0d291 common/type_to_string: formatting for bitcoin_outpoint.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-15 12:09:36 +02:00
Rusty Russell
2ab4e5b42b utils: add max_unsigned/min_unsigned helpers.
We are usually dealing with unsigned values, so use this.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-15 12:09:36 +02:00
Rusty Russell
c9b82bf1d2 channeld: restore ping command, but only for channeld.
It's probably not worth fixing for the other daemons.

Changelog-Changed: JSON-RPC: `ping` now only works if we have a channel with the peer.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-10 15:32:57 +02:00
Rusty Russell
c394fd5db2 common/read_peer_msg: handle pings in handle_peer_gossip_or_error().
This is a noop for now, since gossipd handles them.  But that
will change in the next patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-10 15:32:57 +02:00
Dustin Dettmer
6dc7fc6aee Fix memory leak abort in debug mode 2021-10-10 13:22:29 +02:00
Rusty Russell
7157a92ea6 channeld: import updated channel_upgrade spec.
It now uses raw bitfields instead of a subtype, and only allows a single
option for any upgrade.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-08 16:07:21 +02:00
Rusty Russell
55dbe82162 features: EXPERIMENTAL_FEATURES: advertize option_quiesce
The latest draft has a feature bit here.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-08 16:07:21 +02:00
Rusty Russell
c92ce59892 BOLT 12: switch invoice_request/invoice to singular chain field.
We keep the now-removed chains field, and in deprecated mode, we set it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: bolt12: `chains` in invoice_request and invoice is deprecated, `chain` is used instead.
2021-10-08 13:47:30 +02:00
Rusty Russell
45bf7a3974 bolt12: update to latest spec.
Main changes are:
1. Uses point32 instead of pubkey32.
2. Uses issuer instead of vendor.
3. Uses byte instead of u8.
4. blinded_path num_hops is now a byte, not u16 (we don't use that yet!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: bolt12: `vendor` is deprecated: the field is now called `issuer`.
2021-10-08 13:47:30 +02:00
Rusty Russell
8f582e770c BOLT12: use point32 instead of pubkey32.
That's the modern BOLT12 term.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-08 13:47:30 +02:00
Rusty Russell
51ed7557a1 bolt12: upgrade to handle latest test vectors.
The latest ones use lno, not lni (this unit tests loads from
../lightning-rfc, silently exiting if it doesn't have the test
vector).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-08 13:47:30 +02:00
ZmnSCPxj jxPCSnmZ
20fe5c6ab1 common/json.c: Have json_add_literal *use* len argument.
The current `json_add_literal` does not use its `len` argument, at all:

09c2fef4a4/common/json.c (L1076-L1083)

Which is something of a WTF, why even require the `len` argument if it
is not even used in the function?
This would have been acceptable if it were a callback and the
callback-requiring function passed it in always, but this function is
not intended to be a callback but instead to be called directly.

Existing callers always pass in `strlen(literal)`, so this bug was not
noticed before:

09c2fef4a4/lightningd/jsonrpc.c (L511)
09c2fef4a4/lightningd/jsonrpc.c (L583)
09c2fef4a4/lightningd/options.c (L1408-L1409)
2021-10-06 16:09:20 +02:00
W. J. van der Laan
dd93f5dd29 Add missing headers for compiling on FreeBSD 13
- `netinet/in.h`: struct sockaddr_in, struct sockaddr_in6
- `unistd.h`: close()
2021-09-24 12:40:09 +09:30
Rusty Russell
09e0794b06 common/features: try to add all proposed feature names.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-23 15:02:05 +02:00
Rusty Russell
7ef2f4d7fb devtools/features: tool to convert feature bitmap to names.
For example:

```
$ ./devtools/features 80008008226aa2
option_data_loss_protect/odd (optional)
option_upfront_shutdown_script/odd (optional)
option_gossip_queries/odd (optional)
option_var_onion_optin/odd (optional)
option_gossip_queries_ex/odd (optional)
option_static_remotekey/odd (optional)
option_payment_secret/even (compulsory)
option_basic_mpp/odd (optional)
option_anchor_outputs/odd (optional)
option_shutdown_anysegwit/odd (optional)
option_onion_messages/odd (optional)
option_unknown_54/odd (optional)
```


Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-23 15:02:05 +02:00
Rusty Russell
79e09b92ef Makefile: remove generated files.
By popular merge-hell demand.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Build: Python is now required to build, as generated files are no longer checked into the repository.
2021-09-22 15:25:14 +02:00
Rusty Russell
eaec226f99 common: json_to_reply_path to helper to handle blindedpath/onion_message hook
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-22 09:10:34 +09:30
Rusty Russell
3a966191b8 common/blindedpath: enctlv unwrapping primitives.
And we check them on our test vectors.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-22 09:10:34 +09:30
Rusty Russell
b98e8a099d common/blindedpath: enctlv creation primitives.
And test vector generation!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-22 09:10:34 +09:30
Rusty Russell
f31f7b1eec common/sphinx: add helper to prepend length to payload.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-22 06:44:26 +09:30
Rusty Russell
1f8564ffae type_to_string: detect type failure in DEVELOPER mode.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-21 18:04:43 +02:00
Rusty Russell
24536c5561 common/autodata: use instead of ccan/autodata
This means it needs to be linked ~everywhere.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-21 18:04:43 +02:00
Rusty Russell
b594c53771 common/autodata: autodata replacement which uses __attribute__((constructor))
This is a GCC extension, but avoids much other messiness.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-21 18:04:43 +02:00
Jan Sarenik
284ad2bade Fix for Alpine Linux ond OpenBSD
After recent header files clean-up it was not possible to
build c-lightning 7401b2682. This patch fixes it both for
Alpine Linux and OpenBSD.

Proposed-by: nathanael <nathanael@dalliard.ch>

Changelog-None
2021-09-20 14:44:27 +02:00
Rusty Russell
7401b26824 cleanup: remove unneeded includes in C files.
Before:
 Ten builds, laptop -j5, no ccache:

```
real	0m36.686000-38.956000(38.608+/-0.65)s
user	2m32.864000-42.253000(40.7545+/-2.7)s
sys	0m16.618000-18.316000(17.8531+/-0.48)s
```

 Ten builds, laptop -j5, ccache (warm):

```
real	0m8.212000-8.577000(8.39989+/-0.13)s
user	0m12.731000-13.212000(12.9751+/-0.17)s
sys	0m3.697000-3.902000(3.83722+/-0.064)s
```

After:
 Ten builds, laptop -j5, no ccache: 8% faster

```
real	0m33.802000-35.773000(35.468+/-0.54)s
user	2m19.073000-27.754000(26.2542+/-2.3)s
sys	0m15.784000-17.173000(16.7165+/-0.37)s
```

 Ten builds, laptop -j5, ccache (warm): 1% faster

```
real	0m8.200000-8.485000(8.30138+/-0.097)s
user	0m12.485000-13.100000(12.7344+/-0.19)s
sys	0m3.702000-3.889000(3.78787+/-0.056)s
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-17 09:43:22 +09:30
Rusty Russell
ea30c34d82 cleanup: remove unneeded includes in header files.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-17 09:43:22 +09:30
Rusty Russell
00a0d09340 tools/check-includes.sh: test that c files include their .h files.
This is best-practice (to ensure prototypes match up), but there were a
few places we didn't (at least, directly).  Make it a requirement,
either of form "foo.h" or <dir/foo.h>.

The noise is the change to our print templates.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-17 09:43:22 +09:30
Rusty Russell
acef45d02e common/channel_type: fix up bolt quotes now channel-types is merged.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-13 15:53:23 +02:00
Rusty Russell
183fe107e8 lightningd: use channel_type, pass to-and-from channeld.
Instead of explicit option_static_remotekey and option_anchor_outputs flags.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-13 15:53:23 +02:00