This takes an extra 8 bytes per channel, but means we can go back and
get more information about them; this is implemented in
gossmap_chan_get_update_details() which is what listchannels will need.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Equivalent to gossipd/test/run-find_route.c and gossipd/test/run-find_route-specific.c
except they use gossmap.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
There are several reports of desynchronization with LND here; a simple
approach is to only have one feerate change in flight at any time.
Even if this turns out to be our fault, it's been a historic area of
confusion, so this restriction seems reasonable.
Changelog-Fixed: Protocol: Don't create more than one feerate change at a time, as this seems to desync with LND.
Fixes: #4152
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
The main change which affects us is that 2016 blocks to forget a channel
is a fixed number in the spec; we make this clear by renaming the
(developer-only) max_funding_unconfirmed to dev_max_funding_unconfirmed
and making it compile DEVELOPER only.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
They're not defined to be, though we've not seen this on Linux (testing
showed that it is page-level atomic, which means it can still happen across
page boundaries though!). This was pointed out by whitslack in
https://github.com/ElementsProject/lightning/issues/4288
In practice, this just means not complaining when it happens, and also
not trying to get tricky to use it on MacOS (we can safely seek & write,
since we're single-threaded).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: Removed bogus UNUSUAL log about gossip_store 'short test'.
You can now activate dual-funded channels using the
`--experimental-dual-fund` flag
Changelog-Changed: Config: `--experimental-dual-fund` runtime flag will enable dual-funded protocol on this node
clang 10.0.0 (erroneously?) claims an enum_side cannot be >= NUM_SIDES.
Make it clear that we're testing the raw u8 for validity.
Fixes: #4409
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It's not unheard of for people to give the wrong funding tx to us,
getting their funds stuck. Interestingly, we can allow mutual close
using a different txid and output number as long as they (solely)
funded the channel, and the channel hasn't been used.
This defines a "play area" feature to do just that.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
> If the peer's revocation basepoint is unknown (e.g. `open_channel2`),
> a temporary `channel_id` should be found by using a zeroed out basepoint
> for the unknown peer.
We consolidate to the latest/singular RFC patch for dual-funding, so
there's just a single patchfile for the change. Plus we move back to the
opener setting the desired feerate, the accepter merely declines to
participate if they disagree with the set rate.
Caused by missing common/iso4217.c from common/Makefile:
```
In file included from ./common/iso4217.h:4,
from common/iso4217.c:3:
./wire/wire.h:7:10: fatal error: secp256k1_recovery.h: No such file or directory
7 | #include <secp256k1_recovery.h>
| ^~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make: *** [Makefile:265: common/iso4217.o] Error 1
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This was likely missed because we don't run the tests under valgrind anymore
due to time constraints. I do run them on a semi-regular basis, which is why
I found this.
No more sending "all-channel" errors; in particular, gossipd now only
sends warnings (which make us hang up), not errors, and peer_connected
rejections are warnings (and disconnect), not errors.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Plugins: `peer_connected` rejections now send a warning, not an error, to the peer.
And make all the callers choose which one. In general, I prefer warn,
which lets them reconnect and try again, however some places are either
stated that they must be errors in the spec itself, or in openingd
where we abandon the channel when we close the connection anyway.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: we now send warning messages and close the connection, except on unrecoverable errors.
This is in line with the warnings draft, where all-zeroes in a
channel_id is no longer special (i.e. it will be ignored).
But gossipd would send these if it got upset with us, so it's best
practice to ignore them for now anyway.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Protocol: we treat error messages from peer which refer to "all channels" as warnings, not errors.
This takes from the draft spec at https://github.com/lightningnetwork/lightning-rfc/pull/834
Note that if this draft does not get included, the peer will simply
ignore the warning message (we always close the connection afterwards
anyway).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Protocol: we now report the new (draft) warning message.
Now we create a separate set of local mods, and apply and unapply it.
This is more efficient than the previous approach, since we can do
some work up-front. It's also more graceful (and well-defined) when a
local modification overlaps an existing one.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We should actually be including this (as it may define _GNU_SOURCE
etc) before any system headers. But where we include <assert.h> we
often didn't, because check-includes would complain that the headers
included it too.
Weaken that check, and include config.h in C files before assert.h.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
The fetchinvoice and offers plugins disable themselves if the option
isn't enabled (it's enabled by default on EXPERIMENTAL_FEATURES).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: `experimental-offers` enables fetch, payment and creation of (early draft) offers.
Don't include exp directly, use an ifdef in common/bolt12
(like we do for peer and onion wiregen files).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Note that this also changes so the feature is not represented in channels,
reflecting the recent drafts.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: `experimental-onion-messages` enables send, receive and relay of onion messages.
Allow a user to switch on dual-funding without needing to compile
as EXPERIMENTAL_FEATURES.
Doesn't work yet, since everything is still behind
'EXPERIMENTAL_FEATURES' compile time flags... but useful for testing
This is experimental for now, but can eventually deprecated
'decodepay' and even decode other kinds of messages.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Our new "decode" command will also handle bolt11. We make a few cleanups:
1. Avoid type_to_string() in JSON, instead use format functions directly.
2. Don't need to escape description now that JSON core does that for us.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This makes for more useful errors. It prints where it was up to in
the guide, but doesn't print the entire JSON it's scanning.
Suggested-by: Christian Decker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
In several places we want to access the first element of an array.
This uses a '[indexnum:xxx]' form which is a bit weird, but works similarly
to the way we specify member matches.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This takes a JSON-style format string, and does intelligent parsing,
removing a lot of boilerplate from code which needs to deal with JSON.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This avoids duplication of both logic and error-prone values, such as
the salt. Grouping all hsm encryption logic into a public API will also
allow us to fuzz it.
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
Using onionmessage hook, we get the response and either present it
to the user (invoice) or return the error to the user.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Invoices are signed with our own key, but we use a transient payer_key with a
tweak for invoice_requests (and refunds).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-changed: Default network on both new and old
installs is now bitcoin. The warning "default network
changing in 2020" was removed.
Reverts 36c517bFixes#4159
1. Hoist 7200 constant into the bolt12 heade2.
2. Make preimage the last createinvoice arg, so we could make it optional.
3. Check the validity of the preimage in createinvoice.
4. Always output used flag in listoffers.
5. Rename wallet offer iterators to offer_id iterators.
6. Fix paramter typos.
7. Rename `local_offer_id` parameter to `localofferid`.
8. Add reference constraints on local_offer_id db fields.
9. Remove cut/paste comment.
10. Clarify source of fatal() messages in wallet.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This takes an unsigned bolt11 (or bolt12 if EXPERIMENTAL_FEATURES) string
and signs it and puts it in the database.
The invoice command could now be moved out to a plugin, in fact.
Changelog-Added: JSON-RPC: `createinvoice` new low-level invoice creation API.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is for offers which have `send_invoice`: we need to associate the
payment with the original offer, in (the usual) case where it is a single
use offer. We mark it used when it's paid, to avoid a race.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Still asserts that it's the standard size, but makes it a dynamic
member. For simpliciy, changes the parse_onionpacket API (it must be
a tal object now, so we might as well allocate it here to catch all
the callers).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
In a couple of places we accept arrays of strings and don't validate
them. If we forward them, e.g., call a JSON-RPC method from the
plugin, we end up embedding the unverified string in the JSON-RPC
call without escaping, which then leads to invalid JSON being passed
on.
This at least partially causes #4238
This is vital for calculating merkle trees; I previously used
towire+fromwire to get this!
Requires generation change so we can magic the ARRAY_SIZE var (the C
pre-processor can't uppercase things).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
When we support bolt12, this won't exist. We only need min_final_cltv_expiry,
routes and features, so put them into struct payment explicitly.
We move the default final ctlv out to the caller, too, which is clearer.
e.g. keysend was using this value, but it was hard to tell.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Needed for v2 of channel opens, where the minimum weight is 110; a
'simple utxo' (sig + key) weighs in at 107, so we a need a way to
establish a floor for this case.
Changelog-Added: JSON-RPC: fundpsbt/utxopsbt have new param, `min_witness_utxo`, which sets a floor for the weight calculation of an added input
Avoids much cut & paste. Some tests don't need any of it, but most
want at least some of this infrastructure.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We already do some sanity checks, add this one.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: invalid UTF-8 strings now rejected.
We don't have a problem with them, but callers may; easier to reject bad
UTF8 here than let the caller fail when it tries to parse output.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
we only want to sign the inputs that we've reserved via utxopsbt or
fundpsbt. we mark them with a flag (reusing the now defunct max-len
flag is fine), then look for inputs with that flag to pass to signonly
Just applied the same suppression as rusty in:
6635fe12e4 (Rusty Russell 2020-05-15 15:57:29 +0930 146)
/* cppcheck-suppress uninitvar - false positive on f1->bits */
My cppcheck was complaining about the same issue in the following functions.
I wonder why travis does not care though.
Changelog-None
1. One place returned false instead of -1.
2. The names implied it returned a bool, and it doesn't.
Fix both, and curse C's loose typing a little.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
There's a spec rule about only ever sending a correctly sized
feature-bits, so as a precaution we have `clear_feature_bit` correctly
resize when a bit is cleared.
libwally has a quirk where the finalize method will fail to 'completely'
finalize an input's parts if either the final_scriptsig or
final_redeemscript fields are set
since we manually set the final_witness stack here, we also need to
fully finalize the redeemscript -> final_scriptsig here as well.
And make caller of json_stream_forward_change_id use it, since
we're going to reuse that.
Also call json_out_finished here, so next object doesn't have a ","
prepended.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Note that check-whitespace and check-bolt already do this, so we
can eliminate redundant lines in common/Makefile and bitcoin/Makefile.
We also include the plugin headers in ALL_C_HEADERS so they get
checked.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This:
- Allows `.*btc` amounts (without post-decimal)
- Avoids creating decimals when amount is 0 btc
- Corrects our handling of the suffixes (memeqstr would
sometimes return false because of null-termination)
Changelog-Fixed: We are now able to parse any amount string (XXXmsat, XX.XXXbtc, ..) we create.
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
Instead of a boutique message, use a "real" channel_announcement for
private channels (with fake sigs and pubkeys). This makes it far
easier for gossmap to handle local channels.
Backwards compatible update, since we update old stores.
We also fix devtools/dump-gossipstore to know about the tombstone markers.
Since we increment our channel_announce count for local channels now,
the stats in the tests changed too.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We were always ordering heap by distance, not score (which are different
if we are routing by cheapest, not shortest!).
This simplifies our callbacks, too.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
There's a few structs/wire calls that only exist under experimental features.
These were in a common file that was shared/used a bunch of places but
this causes problems. Here we move one of the problematic methods back
into `openingd`, as it's only used locally and then isolate the
references to the `witness_stack` in a new `common/psbt_internal` file.
This lets us remove the iff EXP_FEATURES inclusion switches in most of
the Makefiles.
There are 3 commands for opening a channel with dualfunding.
`openchannel_init` is the first of these.
It initializes the open-channel dialog, and stops once we've run out of
updates (input/outputs) to send to the peer.
We force use of tal_wally_start/tal_wally_end around every wally
allocation, and with "end" make the caller choose where to reparent
everything.
This is particularly powerful where we allocate a tx or a psbt: we
want that tx or psbt to be the parent of the other allocations, so
this way we can reparent the tx or psbt, then reparent everything
else onto it.
Implementing psbt_finalize (which uses a behavior flag antipattern)
was tricky, so I ended up splitting that into 'psbt_finalize' and
'psbt_final_tx', which I think also makes the callers clearer.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
1. Rename memleak_enter_allocations to memleak_find_allocations.
2. Unify scanning for pointers into memleak_remove_region / memleak_remove_pointer.
3. Document the functions.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
The next patch perturbed things enough that we suddenly started
getting (with --track-origins=yes):
Valgrind error file: valgrind-errors.120470
==120470== Use of uninitialised value of size 8
==120470== at 0x14EBD5: htable_val (htable.c:150)
==120470== by 0x14EC3C: htable_firstval_ (htable.c:165)
==120470== by 0x14F583: htable_del_ (htable.c:349)
==120470== by 0x11825D: pointer_referenced (memleak.c:65)
==120470== by 0x118485: scan_for_pointers (memleak.c:121)
==120470== by 0x118500: memleak_remove_region (memleak.c:130)
==120470== by 0x118A30: call_memleak_helpers (memleak.c:257)
==120470== by 0x118A8B: call_memleak_helpers (memleak.c:262)
==120470== by 0x118A8B: call_memleak_helpers (memleak.c:262)
==120470== by 0x118B25: memleak_find_allocations (memleak.c:278)
==120470== by 0x10EB12: closing_dev_memleak (closingd.c:584)
==120470== by 0x10F3E2: main (closingd.c:783)
==120470== Uninitialised value was created by a heap allocation
==120470== at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==120470== by 0x1604E8: allocate (tal.c:250)
==120470== by 0x160AA9: tal_alloc_ (tal.c:428)
==120470== by 0x119BE0: new_per_peer_state (per_peer_state.c:24)
==120470== by 0x11A101: fromwire_per_peer_state (per_peer_state.c:95)
==120470== by 0x10FB7C: fromwire_closingd_init (closingd_wiregen.c:103)
==120470== by 0x10ED15: main (closingd.c:626)
==120470==
This is because there is uninitialized padding at the end of struct
peer_state.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
They previously prevented any child from being detected as leaks, now
they just mark the tal allocation itself as not being a leak.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
```
cc common/amount.c
common/amount.c:306:15: error: implicit conversion from 'unsigned long long' to
'double' changes value from 18446744073709551615 to 18446744073709551616
[-Werror,-Wimplicit-int-float-conversion]
if (scaled > UINT64_MAX)
~ ^~~~~~~~~~
/usr/include/sys/stdint.h:123:21: note: expanded from macro 'UINT64_MAX'
^~~~~~~~~~~~~~~~~~~~~
1 error generated.
gmake: *** [Makefile:254: common/amount.o] Error 1
bsd$
```
Fixes: #4044
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: delpay a new method to delete the payment completed or failed.
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
We can use a fixed value and close the channel if they don't cover their
amount; this wasn't really helping with anything other than setting a
floor for an expected feerate
Greatly simplify the changeset API. Instead of 'diff' we simply generate
the changes.
Also pulls up the 'next message' method, as at some point the
interactive tx protocol will be used for other things as well
(splices/closes etc)
Suggested-By: @rustyrussell
v2 of channel open uses the channel revocation basepoints to calculate
the channel_id, instead of the funding_txid + outnum
Moving away from the funding_txid opens the way for splicing + rbf
v2 channel open uses a different method to derive the channel_id, so now
we save it to the database so that we dont have to remember how to
derive it for each.
includes a migration for existing channels
There's a lot of it, and it means we can't `make check-source` on
these files.
Also bring bolt quotes up-to-date.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It's now only needed by devtools/mkfunding, so include a reduced one
there, and this also means we remove tx_spending_utxos().
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This removes the reservation cleanup at startup, too, now they're all
using 'reserved_til'.
This changes test_withdraw, since it asserted that outputs were marked
spent as soon as we broadcast a transaction: now they're reserved until
it's mined. Similarly, test_addfunds_from_block assumed we'd see funds
as soon as we broadcast the tx.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: `withdraw` now randomizes input and output order, not BIP69.
This avoids overwriting the ones in git, and generally makes things neater.
We have convenience headers wire/peer_wire.h and wire/onion_wire.h to
avoid most #ifdefs: simply include those.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We're going to make experimental versions of these completely separate files.
Also remove the dependency on the Makefile itself: it simply causes
unnecessary churn. We can always force-rebuild when we change a rule.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
