Commit Graph

390 Commits

Author SHA1 Message Date
Rusty Russell
9450d46db1 bitcoin/short_channel_id: pass by copy everywhere.
It's a u64, we should pass by copy.  This is a big sweeping change,
but mainly mechanical (change one, compile, fix breakage, repeat).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-03-20 13:51:48 +10:30
Rusty Russell
e0e879c003 common: remove type_to_string files altogther.
This means including <common/utils.h> where it was indirectly included.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-03-20 13:51:48 +10:30
Rusty Russell
37d22f9141 global: change all type_to_string to fmt_X.
This has the benefit of being shorter, as well as more reliable (you
will get a link error if we can't print it, not a runtime one!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-03-20 13:51:48 +10:30
Rusty Russell
d8c06dccac global: expose all fmt_X functions for direct use, make uniform.
We have various functions to convert to a string, rename them all so we can
count on fmt_X being the formatter for struct X, and make them all return
`char *`.

Sometimes they existed but were private, sometimes they had a
different name.  Most take a pointer, but simple types pass by copy:
short_channel_id, amount_msat and amount_sat.

The following public functions changed:
1. psbt_to_b64 -> fmt_wally_psbt.
2. pubkey_to_hexstr -> fmt_pubkey.
3. short_channel_id_to_str -> fmt_short_channel_id (scid by copy now!)
4. fmt_signature -> fmt_secp256k1_ecdsa_signature
5. fmt_amount_sat/fmt_amount_msat pass copy not pointer, return non-const char *.
6. node_id_to_hexstr -> fmt_node_id

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-03-20 13:51:48 +10:30
Jon Griffiths
aa23c2a2b2 script: consistently take the script length in identification functions
Standardizes the is_xxx script function all take a script length, and changes
their first-level callers to pass it. This has several knock on benefits:

- We remove the repeated tal_count/tal_bytelen calls on the script, in
  particular the redundant calls that result when we must check for multiple
  types of script - which is almost all cases.
- We remove the dependency on the memory being tal-allocated (It is, in
  all cases, but theres no reason we need to require that).
- We remove all cases where we create a copy of the script just to id it.
- We remove all allocations for non-interesting scripts while iterating block
  txs in process_getfilteredblock_step1().
- We remove all allocations *including for potentially interesting scripts* in
  topo_add_utxos().

Signed-off-by: Jon Griffiths <jon_p_griffiths@yahoo.com>
2024-03-18 12:24:49 +10:30
Rusty Russell
df44431f8c common: add tal_arr_eq helper.
We do `memeq(a, tal_bytelen(a), b, tal_bytelen(b))` remarkably often...

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-02-16 15:02:38 +01:00
Rusty Russell
27ef4b98c1 devtools: remove warning message from dump-gossipstore.
It prints a message to stderr, but actually it's fine with this version:

```
dump-gossipstore: UNKNOWN GOSSIP minor VERSION 14 (expected 12)
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-02-12 11:43:33 +01:00
Rusty Russell
07cd4a809b gossipd: remove spam handling.
We weakened this progressively over time, and gossip v1.5 makes spam
impossible by protocol, so we can wait until then.

Removing this code simplifies things a great deal!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: Protocol: we no longer ratelimit gossip messages by channel, making our code far simpler.
2024-02-04 09:24:44 +10:30
Rusty Russell
e7ceffd565 gossipd: remove zombie handling.
We never enabled it, because we seemed to be eliminating valid
channels.  We discard zombie-marked records on loading.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-02-04 09:24:44 +10:30
Rusty Russell
f2f43eeffa gossipd: strip private updates from gossip_store on startup.
We rename them to _obs, too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2024-01-31 14:47:33 +10:30
Peter Neuroth
3cbe35e7da devtools: Deduplicate links from linkify
The method linkify just added links for an entry to a list and produced
duplicates if we got more than one changelog line in a PR. We now only
produce a link once per PR.

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>
2024-01-15 18:36:01 +01:00
Rusty Russell
e609bc934e channeld: tell lightningd about local anchor for each commitment tx.
It's going to want to remember these, in case it encounters peers'
commitment tx and needs to boost it with CPFP on the anchor.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-27 11:19:56 +10:30
Rusty Russell
9b3965c805 channeld: rename channel_splice_txs to channel_txs.
channel_txs was a thin wrapper around channel_splice_txs, but that's
just confusing.  Rename channel_splice_txs to channel_txs, and just
call it everywhere.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-10-27 11:19:56 +10:30
Rusty Russell
fea4eb8968 common: helper to generate bolt12 test vector bolt12/offers-test.json.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-21 13:31:03 +09:30
Rusty Russell
16e0d38fff bolt12-cli: add encode routine to make bolt12 vectors.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-09-21 13:31:03 +09:30
Alex Myers
bd4a001279 cli: don't crash when there's no argument
This should provide the default help message and exit, but was
resulting in a segmentation fault from freeing pointers passed to
the default config.

Changelog-Fixed: lightning-cli properly returns help without argument
2023-08-16 12:25:25 +09:30
Rusty Russell
5ba4713b21 devtools/credit: ! not ^ to invert the match set in sh.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-07-31 17:17:15 +09:30
grubles
a1d25961de devtools/print_wire: long long instead of longing
adding PRId64 magic instead

Changelog-None
2023-07-24 15:24:05 +09:30
Rusty Russell
8178b7389f gossip_store: add "dying" flag to indicate not to gossip dying channels.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-07-20 11:47:32 +09:30
Dustin Dettmer
aba4d18ed1 signed types: add handlers for signed types
We're adding signed types to the spec! This adds the support mechanisms
for them.
2023-07-13 19:34:47 +09:30
Rusty Russell
af6d7c0779 global: thread zero fee option everywhere.
In most cases, it's the same as option_anchor_outputs, but for
fees it's different.  This transformation is the simplest:
pass it as a pair, and test it explicitly.

In future we could rationalize some paths, but this was nice
and mechanical.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-29 11:28:47 -04:00
Rusty Russell
c1d3da889c tools/gossipwith: shutdown, don't close, TCP socket.
This isn't the cause of the test_gossip_ratelimit flake I saw (since
the final gossip msg clearly was received), but it's still good to fix
since it means we might not send the final messages.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-20 17:24:48 +09:30
Matt Morehouse
e81bf8b89f devtools: call common_setup in main
Fixes nullability errors detected by UBSan:

wire/fromwire.c:173:46: runtime error: null pointer passed as argument 1, which is declared to never be null
external/libwally-core/src/secp256k1/include/secp256k1.h:432:3: note: nonnull attribute specified here
    #0 0x65214a in fromwire_secp256k1_ecdsa_signature wire/fromwire.c:173:6
    #1 0x659500 in printwire_secp256k1_ecdsa_signature devtools/print_wire.c:331:1
    #2 0x646ba2 in printwire_channel_update wire/peer_printgen.c:1900:7
    #3 0x637182 in printpeer_wire_message wire/peer_printgen.c:128:11
    #4 0x65a097 in main devtools/decodemsg.c:85:10
2023-06-05 16:16:21 +02:00
Rusty Russell
edbaa944da lightningd: switch parsing to common/configvar
Now we wire in the code which gathers configvars and parses from there;
lightningd keeps the array of configuration variables for future use.

Note that lightning-cli also needs to read the config, but it has its
own options (including short ones!) and doesn't want to use this
configvar mechanism, so we have a different API for that now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-03 10:50:29 +09:30
Rusty Russell
f45c17450b ccan: update to latest ccan/opt
This adds:
1. ability to search for an option by name.
2. allowance to set our own bits when registering options.
3. show callbacks which can say "don't show", and variable length.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-03 10:50:29 +09:30
Rusty Russell
a6772e9dec common: add new internal type for websockets.
Now it's not a public type, we need a way to refer to it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-01 09:28:39 +09:30
Rusty Russell
3f35d48fe4 common: remove websocket type from wireaddr.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-01 09:28:39 +09:30
Rusty Russell
d40379885d common/wireaddr.h: simplify parse_wireaddr API.
1. Make it the standard "return the error" pattern.
2. Rather than flags to indicate what types are allowed, have the callers
   check the return explicitly.
3. Document the APIs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-06-01 09:28:39 +09:30
Rusty Russell
4deb552fe9 build: don't generate experimental variants of wire files.
We no longer have any experimental-only wire definitions.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-05-23 09:34:08 +09:30
Rusty Russell
295557ac50 connectd: don't try to set TCP_CORK on websocket pipe.
Most of this is piping the flag through so we know it's a websocket!

Reported-by: @ShahanaFarooqui
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-04-10 09:41:56 +09:30
Christian Decker
67a39b59e7 tools: Add yml mode to blockreplace.py 2023-03-25 15:40:35 +10:30
Rusty Russell
73e50b26f9 devtools: fix ZOMBIE detection in devtools/dump-gossipstore.
A victim of simultaneous changes, and I didn't pick it up :(
    
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-02-15 14:19:42 -06:00
Rusty Russell
0274d88bad common/gossip_store: clean up header.
It's actually two separate u16 fields, so actually treat it as
such!

Cleans up zombie handling code a bit too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2023-01-30 15:15:41 -06:00
Alex Myers
6bff10cd40 gossip_store: add a flag for zombie entries
This will allow gossipd to store and persist gossip for channels rather
than deleting them entirely when the channels are pruned from the
network.
2023-01-30 16:33:03 +10:30
Christian Decker
a31575ca0b tools: Add multi-language support to blockreplace.py
Suggested-by: Rusty Russell <@rustyrussell>
2022-12-06 11:39:53 +10:30
Christian Decker
744d111cea doc: Create a blockreplace tool to update generated blocks in docs
We introduced a minor issue in #5757 that was causing the manpages to
be added every time we regenerate instead of replacing them. The
script was a bit unscrutable, and we do this block-replacement in
several places I thought it might be a good idea to have a dedicated
tool to do it.

This allows us to have simpler Makefiles whenever we update a
generated block inside another file. It's python, but does not have
dependencies :-)

Changelog-None
2022-12-06 11:39:53 +10:30
Christian Decker
dfb963e249 db: Backfill missing HTLC IDs in the forwards table
We have a primary key that is spanning the `in_channel_id` and the
`in_htcl_id`. The latter gets set to NULL when the HTLC and channel
gets deleted, so we coalesce with a random large number that is
unlikely to collide for the primary key.
2022-11-28 14:47:43 +01:00
Rusty Russell
2e4a58efac check: fix warnings from shellcheck 0.8.0
And make errors gcc-style, so emacs can jump through the automatically.

```
In devtools/reduce-includes.sh line 21:
	    echo -n "-$LINE"
                 ^-- SC3037 (warning): In POSIX sh, echo flags are undefined.


In devtools/reduce-includes.sh line 25:
	    echo -n "."
                 ^-- SC3037 (warning): In POSIX sh, echo flags are undefined.


In tools/rel.sh line 6:
prefix=$(printf '%s\n' "${from#$common}" | sed 's@[^/][^/]*@..@g')
                               ^-----^ SC2295 (info): Expansions inside ${..} need to be quoted separately, otherwise they match as patterns.

Did you mean: 
prefix=$(printf '%s\n' "${from#"$common"}" | sed 's@[^/][^/]*@..@g')


In tools/rel.sh line 7:
printf '%s\n' "$prefix/${to#$common}"
                            ^-----^ SC2295 (info): Expansions inside ${..} need to be quoted separately, otherwise they match as patterns.

Did you mean: 
printf '%s\n' "$prefix/${to#"$common"}"

For more information:
  https://www.shellcheck.net/wiki/SC3037 -- In POSIX sh, echo flags are undef...
  https://www.shellcheck.net/wiki/SC2295 -- Expansions inside ${..} need to b...
make: *** [Makefile:553: check-shellcheck] Error 123
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell
8a217f13cf bolt12: update comments to match latest spec.
No code changes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell
37bc4603b8 lightningd: re-add 'offerout' functionality, as 'invoicerequest'.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell
fdb3d9186a devtools/bolt12-cli: fix decode to understand modern fields.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell
1e3cb01546 bolt12: import the latest spec, update to fit.
I know this is an unforgivably large diff, but the spec has changed so
much that most of this amounts to a rewrite.

Some points:
* We no longer have "offer_id" fields, we generate that locally, as all
  offer fields are mirrored into invoice_request and then invoice.
* Because of that mirroring, field names all have explicit offer/invreq/invoice
  prefixes.
* The `refund_for` fields have been removed from spec: will re-add locally later.
* quantity_min was removed, max == 0 now mean "must specify a quantity".
* I have put recurrence fields back in locally.

This brings us to 655df03d8729c0918bdacac99eb13fdb0ee93345 ("BOLT 12:
add explicit invoice_node_id.")

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell
8720bbedae common/onion: split into decode and encode routines.
Some places (e.g. the pay plugin) only need to construct onions,
not decode them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
niftynei
7bd0d7641c mkfunding: no scriptPubKey on utxo causing crash, so we add one
Changelog-Fixed: devtools: `mkfunding` command no longer crashes (abort)

Fixes #5363

Assisted-By: @TKChattoraj
2022-11-01 18:23:00 +01:00
niftynei
87616b7ffc mkfunding: add missing common_setup
Was crashing on allocating wally memory, b/c was missing the wally_init
that's wrapped up in common_setup

Assisted-By: @TKChattoraj
2022-11-01 18:23:00 +01:00
Rusty Russell
5cf86a1a2e common: update to latest onion message spec.
Mainly, field name changes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: Protocol: Support for forwarding blinded payments (as per latest draft)
2022-10-26 11:29:06 +10:30
Rusty Russell
511e8e6477 common/blindedpay: routines to construct a blinded payment.
Don't shoehorn it into onion_nonfinal_hop() and onion_final_hop(), but
provide an explicit routine "blinded_onion_hops" and an onion helper
"onion_blinded_hop()" for it to call.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-10-26 11:29:06 +10:30
Rusty Russell
077ec99788 common/onion: blinded payment support.
We make it look like a normal payment for the caller.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-10-26 11:29:06 +10:30
Rusty Russell
662c6931f3 Remove point32.
The x-only dream is dead.  Remove all trace.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-10-26 11:29:06 +10:30
Rusty Russell
e30ea91908 BOLTs: update to more recent bolt12 spec.
It's 2b7ad577d7a790b302bd1aa044b22c809c76e49d, which reverts the
point32 changes.

It also restores send_invoice in `invoice`, which we had removed
from spec and put into the recurrence patch.

I originally had implemented compatibility, but other changes
which followed this are far too widespread.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: offers: complete rework of spec from other teams (yay!) breaks previous compatibility (boo!)
2022-10-26 11:29:06 +10:30