By popular merge-hell demand.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Build: Python is now required to build, as generated files are no longer checked into the repository.
One change from the obsolete version handling, gossipd will no longer send
forwarding onion msgs to lightningd, but will forward it directly.
That was the effect before, anyway.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Before:
Ten builds, laptop -j5, no ccache:
```
real 0m36.686000-38.956000(38.608+/-0.65)s
user 2m32.864000-42.253000(40.7545+/-2.7)s
sys 0m16.618000-18.316000(17.8531+/-0.48)s
```
Ten builds, laptop -j5, ccache (warm):
```
real 0m8.212000-8.577000(8.39989+/-0.13)s
user 0m12.731000-13.212000(12.9751+/-0.17)s
sys 0m3.697000-3.902000(3.83722+/-0.064)s
```
After:
Ten builds, laptop -j5, no ccache: 8% faster
```
real 0m33.802000-35.773000(35.468+/-0.54)s
user 2m19.073000-27.754000(26.2542+/-2.3)s
sys 0m15.784000-17.173000(16.7165+/-0.37)s
```
Ten builds, laptop -j5, ccache (warm): 1% faster
```
real 0m8.200000-8.485000(8.30138+/-0.097)s
user 0m12.485000-13.100000(12.7344+/-0.19)s
sys 0m3.702000-3.889000(3.78787+/-0.056)s
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is best-practice (to ensure prototypes match up), but there were a
few places we didn't (at least, directly). Make it a requirement,
either of form "foo.h" or <dir/foo.h>.
The noise is the change to our print templates.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This was just a minor leak, found by CI for
test_openchannel_hook_chaining. We didn't call negotiation_aborted
which frees various fields: negotiation_failed() does that for us.
```
MEMLEAK: 0x55b0f2d5f3c8
label=common/channel_type.c:19:struct channel_type
backtrace:
ccan/ccan/tal/tal.c:442 (tal_alloc_)
common/channel_type.c:19 (channel_type_none)
common/channel_type.c:27 (channel_type_static_remotekey)
common/channel_type.c:136 (channel_type_accept)
openingd/openingd.c:844 (fundee_channel)
openingd/openingd.c:1240 (handle_peer_in)
openingd/openingd.c:1510 (main)
parents:
openingd/openingd.c:1414:struct state
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
e.g. you can negotiate anchor_outputs, but still ask for a
non-anchor-output channel.
If/when we make those features compulsory, downgrade will
not be allowed.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It was merged (but this doesn't update the BOLT quotes, that's in another patch).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Protocol: We now send and support channel_type in channel open (not dual-funding though).
Currently we actually insist it's the default, but in future it could be
different.
We also need to tell openingd what the channel_type was, if we resume
via openingd_funder_complete().
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We make it a first-class citizen internally, even though we won't use
it over the wire (at least, non-experimental builds). This scheme
follows the latest draft, in which features are flagged compulsory.
We also add several helper functions.
Since uses the *even* bits (as per latest spec), not the *odd* bits,
we have some other fixups.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We want to use this to handle the simple description for channel_type.
It also needs to handle variable-size types (just like subtypes).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Openingd can query them itself (as dualopend already does). And move
the two feature args next to each other on the wire.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This touches a lot of text, mainly to change "if `option_anchor_outputs`"
to "if `option_anchors`"
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This check is going away anyway (only Electrum enforced it), but we
know that all wumbo peers expect large HTLCs to work today.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Protocol: Allow sending large HTLCs if peer offers option_support_large_channel (> 4294967295msat)
dualopend doesn't always listen to lightningd messages, so it would
sometimes hang at the end of tests.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
1. We assumed an empty upfront_shutdown_script TLV would become NULL:
RPC call failed: method: fundchannel, payload: {'id': '022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59', 'amount': 1000000, 'announce': True}, error: {'code': -1, 'message': 'They sent error channel e7c2d5d14462fe269631418fbfc3db327843382e6a2a5a9c2991d2d6ba31d9f5: Unacceptable upfront_shutdown_script ', 'data': {'id': '022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59', 'method': 'fundchannel_start'}}"
2. We were assuming an empty enctlv would become NULL, too.
We should not have done this (there's a semantic difference between
"empty" and not-present for TLVs), so prepare for the change.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
with channel leases, we use the open_tlv data after a round of talking
to the peer (which clears out the tmpctx). It'll get cleaned up
when this peer opens/fails.
We need the 'actual' accepter's funding for the reserve calculations,
which includes the lease fee that the opener is paying them, so we
calculate it before doing all that jazz.
However, we MUST send the actual "on paper" (e.g. without lease fee)
amount to the peer in accept_channel2, so we stash the original amount
and send it.
If there's no plugin currently in place, we simply won't return any
funding at all, in which case we'd expect them to handle however
they want. (our implementation would fail the open, as we only accept
opens that have at least as much as we've requested provided)
We need to know what the lease we're expecting is. To do this
we pass around the hex encoded portion of the wire format.
We can use this passed in expected lease rates to confirm that the peer
is, in fact, using the same rates as what we have currently.
Changelog-Added: JSON-RPC: fundchannel, multifundchannel, and openchannel_init now accept a 'compact_lease' for any requested funds
Using a 'feestep' is more restrictive than you'd want, instead we
enforce that the next feerate must be at least 1/64th more than the
last, but put no upper limit on it
Includes update to lnprototest changes
Contributed-By: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
Changelog-EXPERIMENTAL: Protocol: Replaces init_rbf's `fee_step` for RBF of v2 opens with `funding_feerate_perkw`, breaking change
Enable non-dev builds to send custom messages.
Preserves 'dev-' for compat-enabled builds.
Changelog-Changed: JSON-RPC: moved dev-sendcustommsg to sendcustommsg
Let the callers do that (only channeld needs to do this).
We temporarily send an error on unknown reestablish in openingd, as
this mimic previous behavior and avoids breaking tests (it does leave
a BROKEN message in the logs though, so
test_funding_external_wallet_corners needs to ignore that for now.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Peer sends funding locked, we tell lightningd who saves it to disk.
Then we restart/reconnect and they retransmit funding_locked. We were
re-notifying lightningd about their lock-in, which was crashing/breaking
things. Instead, we ignore duplicate lock-in messages from the peer.
lightningd-1: 2021-05-11T18:00:12.844Z **BROKEN** 022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59-chan#1: Internal error DUALOPEND_AWAITING_LOCKIN: channel_got_funding_locked twice
This is set by the peer and is non-negotiable. We're not even going to
check if you got it right. You were told about it via `openchannel2`.
It is what it is.
The other side doesn't know it until *after* it parses this msg. We
add a quick hack to still allow old nodes to work (for now!).
This also fixes a bug (spotted by @niftynei) where any errors we sent
before accepter_start2 would have the new (unknowable!) channel_id
rather than the temp one.
Authored-by: Rusty Russell <rusty@rustcorp.com.au>
You can now activate dual-funded channels using the
`--experimental-dual-fund` flag
Changelog-Changed: Config: `--experimental-dual-fund` runtime flag will enable dual-funded protocol on this node
In the case of failures/disconnects between dualopend receiving the sigs
and the main man disconnecting (as can happen using the testing
disconnects), failing to forward their sigs over to lightningd can
result in the channel never progressing from DUALOPEND_OPEN_INIT
to DUALOPEND_AWAITING_LOCKIN.
Always re-relay.
> If the peer's revocation basepoint is unknown (e.g. `open_channel2`),
> a temporary `channel_id` should be found by using a zeroed out basepoint
> for the unknown peer.
We consolidate to the latest/singular RFC patch for dual-funding, so
there's just a single patchfile for the change. Plus we move back to the
opener setting the desired feerate, the accepter merely declines to
participate if they disagree with the set rate.
Looks like #4394 treated a symptom but not the root cause. We were
actually sending the message framed with the WIRE_CUSTOMMSG_OUT and
the length prefix over the encrypted connection to the peer. It just
happened to be a valid custommsg...
This fixes the issue, and this time I made sure we actually send the
raw message over the wire. However for backward compatibility we
needed to imitate the faulty behavior which is 90% of this patch :-)
Changelog-Fixed: plugin: `dev-sendcustommsg` included the type and length prefix when sending a message.
If we're doing an RBF, it's possible that the peer will send us a
funding_locked, shutdown, or tx_signatures message. (We get tx_sigs out
of order on a reconnect)
This lets us gracefully handle a shutdown or funding_locked
sent at any time (after first funding tx) as well.
We move over to the new "warning" paradigm, instead of using
an "rbf_fail" message.
Every failure is either a warning or an error; on warnings we
hang up and reconnect later, effectively resetting the state.
We're *mostly* set up for both sides doing RBF, except that it reverses
the callback flow (using the plugin vs RPC calls) and we're not
currently smart enough to flip between them gracefully
We need to make sure that there's at least one input that's represented
in every single RBF-attempt for this channel, to prevent "parallel"
subsequent RBFs from succeeding/opening (the multi-channel backdoor?!)
The spec doesn't say to do this, but it makes sense, otherwise
they'll never be able to mutually close the channel.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
No more sending "all-channel" errors; in particular, gossipd now only
sends warnings (which make us hang up), not errors, and peer_connected
rejections are warnings (and disconnect), not errors.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Plugins: `peer_connected` rejections now send a warning, not an error, to the peer.
And make all the callers choose which one. In general, I prefer warn,
which lets them reconnect and try again, however some places are either
stated that they must be errors in the spec itself, or in openingd
where we abandon the channel when we close the connection anyway.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: we now send warning messages and close the connection, except on unrecoverable errors.
This is in line with the warnings draft, where all-zeroes in a
channel_id is no longer special (i.e. it will be ignored).
But gossipd would send these if it got upset with us, so it's best
practice to ignore them for now anyway.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Protocol: we treat error messages from peer which refer to "all channels" as warnings, not errors.
This takes from the draft spec at https://github.com/lightningnetwork/lightning-rfc/pull/834
Note that if this draft does not get included, the peer will simply
ignore the warning message (we always close the connection afterwards
anyway).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Protocol: we now report the new (draft) warning message.
Let plugins know when a channel open has failed.
We need to notify accepters now too, so we remove the check on who's
funding the channel before sending the 'failed' message from
dualopend->master.
We already know what the channel id is, we should go ahead and pass it
on to any listening plugins -- this makes clean up easier/possible
if a open fails early on and we've got reserved utxos.
There's a case where a dropped funding_locked will result in the peer
moving onto channeld, while we stay in dualopend. As we haven't
received their funding_locked, we retransmit tx_sigs, which channeld
will need to handle.
With the patch the peer drops it on the floor; the peer will resend
funding_locked on reconnect, which will correctly advance us to
channeld and CHANNELD_NORMAL
Prior to this, all reconnect logic lived in channeld. If you
disconnected before we finished building a funding transaction, that was
no big deal. Now, however, we're waiting for the funding to lock in in
dualopend, instead of handing straight to channeld to wait.
So we need a way to restart dualopend.
We were incorrectly saving the *remote's* commitment transaction as the
'last_tx' for a peer, not our own local one.
When we applied the 'remote_commit_sig' to it, it would fail since the
remote's signature doesn't validate for their commitment transaction.
This will make it possible to do RBF, since we can re-start the opening
process in dualopend while waiting for lock-in.
Note the new channel states are being used, DUALOPEND_INIT and
DUALOPEND_AWAITING_LOCKIN, to differentiate from openingd/channeld opens
This is vital for calculating merkle trees; I previously used
towire+fromwire to get this!
Requires generation change so we can magic the ARRAY_SIZE var (the C
pre-processor can't uppercase things).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We let the plugin decide what feerate to accept/whether or not to add
funds to the open. To aid this decision, we also send the plugin what we
(c-lightning) currently have as our max and min acceptable feerates.
We also now use these as our default for max/min acceptable feerate
range when sending an openchannel offer to a peer.
In the future, it might be a good idea to make these more easily
changeable, either via a config setting (?) or a command param.
Note that check-whitespace and check-bolt already do this, so we
can eliminate redundant lines in common/Makefile and bitcoin/Makefile.
We also include the plugin headers in ALL_C_HEADERS so they get
checked.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes#4140
Reported-By: @PsySc0rpi0n
Changelog-Fixed: openingd now uses the correct dust limit for determining the allowable floor for a channel open (affects fundee only)
This is handy/necessary for getting multifundchannel to work, as we need
to know what output to tell all the other peers about.
Changelog-Added: Experimental!! JSON-RPC: openchannel_init returns a field `funding_serial` that indicates the serial_id of the funding output in the provided PSBT
`check_balances` had a weird interface because it was meant to be able
to be used at any 'intermediate' point to verify that a single side had
a valid inputs/output balance.
This was worse than useless. Now it just straight checks for both sides'
balances are correct and that both sides pay their fees. Called after
transaction is constructed.
There's a few structs/wire calls that only exist under experimental features.
These were in a common file that was shared/used a bunch of places but
this causes problems. Here we move one of the problematic methods back
into `openingd`, as it's only used locally and then isolate the
references to the `witness_stack` in a new `common/psbt_internal` file.
This lets us remove the iff EXP_FEATURES inclusion switches in most of
the Makefiles.
Rusty pointed out that having an empty channel_id is suboptimal; adding
another call is probably the right idea rather than re-using an existing
one.
Suggested-By: @rustyrussell
There are 3 commands for opening a channel with dualfunding.
`openchannel_init` is the first of these.
It initializes the open-channel dialog, and stops once we've run out of
updates (input/outputs) to send to the peer.
