Commit Graph

12167 Commits

Author SHA1 Message Date
Rusty Russell
3afa5077fe offers: make them always unsigned.
This is in preparation for the spec update where the signature field
does not even exist.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell
9a0d2040d3 common/features: add explicit bolt12 feature sets.
The spec only specifies the mpp bit for invoices, but in
general they are separate spaces.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell
6e755d6fe8 common/bolt12: code to initialize invreqs from offers, invs from invreqs.
This is an important part of the coming spec: we mirror all fields,
known and unknown.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell
aa73878831 common/bolt12: add code to generate offer_id, extract parts of streams.
The new spec removes the offer_id, in favor of mirroring all the
fields.  So we need a way of generating a convenient identifier to
identify the offer, and this works.

We also want to extract parts of streams elsewhere, so expose that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell
c2c9f45dac offers: create a real blinded path, if necessary.
It's just to a direct peer, and we only create one, but this is
enough to test, and make payments to non-public nodes work.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell
744605997e offers: monitor blockheight.
We need this to create the payment_constraints for an invoice blinded path.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell
4bc10579e6 listincoming: add htlc_min_msat, public and peer_features fields.
This is needed for offers to generate blinded paths.

No documentation changes since listincoming is an undocumented
internal hack interface which topology presents for production
of routehints.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell
c6f50220e1 common/onion_decode: put the path_id into onion_payload->payment_secret.
And check it in invoice.c, insead of a hack where we compare against invhash.
Restore checking, too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell
595fbd2a19 createinvoice: make a minimal blinded "path" in bolt12 invoice if none presented.
The "path" is just a message to ourselves.  This meets the minimal
requirement for bolt12 invoices: that there be a blinded path (at
least so we can use the path_id inside in place of "payment_secret").

We expose the method to make this path_id to a common routine: offers
will need this for generating more sophisticated paths.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell
a5471a405b lightningd: temporarily ignore missing payment_secret for bolt12.
We're going to mess with it in the next patch...

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell
85cb302b65 invoice: invert check to reduce indentation.
Instead of doing command_fail() in the else, do it immediately then
unindent the normal path.

No code changes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell
5becfa6ee1 onion_message: don't use general secret, use per-message secret.
We had a scheme where lightningd itself would put a per-node secret in
the blinded path, then we'd tell the caller when it was used.  Then it
simply checks the alias to determine if the correct path was used.

But this doesn't work when we start to offer multiple blinded paths.
So go for a far simpler scheme, where the secret is generated (and
stored) by the caller, and hand it back to them.

We keep the split "with secret" or "without secret" API, since I'm
sure callers who don't care about the secret won't check that it
doesn't exist!  And without that, someone can use a blinded path for a
different message and get a response which may reveal the node.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell
4cfd972407 common/blindedpath: expose API at a lower level.
We actually want lightningd to create these, since it wants to put the
path_id secret in the last element.  So best API is actually a generic
one, rather than separate APIs to create first and last ones.

And really, the more explicit initialization makes the users clearer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell
01a47720c3 plugins/libplugin-pay: hack in blinded path support.
We simply take the first one, and route to the start of that.  Then we
append the blinded path to the onion construction.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell
8720bbedae common/onion: split into decode and encode routines.
Some places (e.g. the pay plugin) only need to construct onions,
not decode them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell
c5656ec90a common/onion: handle payment by node_id.
In a blinded path, you can specify node_id instead of scid.  Handle
that case.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell
987df688ed lightningd: don't return normal errors on blinded path entry, either.
This current spec is not strict enough: we might complain that the
next peer is not connected, for example, which leaks information.

So return WIRE_INVALID_ONION_BLINDING even if we're the first hop
on the path, to be safe.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Rusty Russell
2760490d5d common: catch up on latest routeblinding spec.
This makes us match eed2ab0c30ad7f93e3b2641ca9d7ade32f3d121d
("Use `invalid_onion_blinding` everywhere").

1. Numerous typographical changes.
2. Make sure we *always* return WIRE_INVALID_ONION_BLINDING if
   we're in a blinded path.
3. Handle p->total_msat correctly (MPP payments).
4. Reorganize blinding handling just like spec order.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 15:08:03 +01:00
Christian Decker
eb122827f6 wallet: Add utxo_is_immature helper 2022-11-09 11:55:25 +01:00
Christian Decker
adf14151fa wallet: Use boolean to determine whether an output is coinbase 2022-11-09 11:55:25 +01:00
niftynei
26f5dcd2a5 wallet: mark coinbase outputs as 'immature' until spendable
Changelog-Changed: JSON-RPC: `listfunds` now lists coinbase outputs as 'immature' until they're spendable
Changelog-Changed: JSON-RPC: UTXOs aren't spendable while immature
2022-11-09 11:55:25 +01:00
niftynei
d60dbba43b tests: test for coinbase wallet spend.
Attempt to spend a coinbase transaction, expected to fail.
2022-11-09 11:55:25 +01:00
Rusty Russell
ae3550cb00 lightning-cli: support --filter parameter.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: cli: new `--filter` parameter to reduce JSON output.
2022-11-09 20:25:58 +10:30
Rusty Russell
c31fb99d2d doc: add lightingd-rpc documentation.
This documents how to communicate with lightningd over RPC, including
use of the `filter` object.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Documentation: `lightningd-rpc` manual page describes details of our JSON-RPC interface, including compatibility and filtering.
2022-11-09 20:25:58 +10:30
Rusty Russell
cb1156cd32 libplugin: support filters.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 20:25:58 +10:30
Rusty Russell
b6134303d4 pyln: add context manager to simpify filter use.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: pyln: LightningRpc has new `reply_filter` context manager for reducing output of RPC commands.
2022-11-09 20:25:58 +10:30
Rusty Russell
1436ad334d pytest: add filter tests.
We suppress schema reply checking when filter is set: we could just
remove all the `required` fields in the JSON schema.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 20:25:58 +10:30
Rusty Russell
2a14afbf21 lightningd: set filter when we see 'filter' object.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `filter` object allows reduction of JSON response to (most) commands.
2022-11-09 20:25:58 +10:30
Rusty Russell
3b4c1968a3 common/test: add unit tests for JSON filtering.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 20:25:58 +10:30
Rusty Russell
f0731d2ca1 common/json_stream: support filtering don't print fields not allowed.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 20:25:58 +10:30
Rusty Russell
22c42de6f1 tests/fuzz: don't pull in JSON common at all.
No tests currently use it, and if they do we'll want to do some
per-test objects.  Otherwise, we are about it introduce a dependency
on common/json_filter.o, which is a can of worms.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 20:25:58 +10:30
Rusty Russell
508a170598 common/json_filter: routine to turn "filter" JSON into a filter.
Since the "struct command" is different from plugins and lightningd, we
need an accessor for this to work (the plugin one is a dummy for now!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 20:25:58 +10:30
Rusty Russell
3c75770586 common/json_filter: routines for json filtering.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-09 20:25:58 +10:30
Christian Decker
15112ae87b gci: Force MacOS CI Job to use python 3.10
The runner version was recently bumped causing our tests to fail
because it couldn't find the `poetry` version it just installed. We
instead install python3.10, and force its use, since otherwise we end
up compiling grpcio and fail to do so.

Changelog-None
2022-11-08 17:41:21 +01:00
Rusty Russell
a4c482dc07 common/sphinx: don't use fixed lengths anywhere.
1. Remove the very concept of ONION_REPLY_SIZE, instead make it a
   local variable in create_onionreply().

2. Use the proper fromwire_ primitives in unwrap_onionreply() so we
   don't have to do explicit length checks.

3. Make fromwire_tal_arrn() return NULL if it fails to pull, instead of
   a zero-length allocation.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Protocol: we now correctly decrypt non-256-length onion errors (we always forwarded them fine, now we actually can parse them).
2022-11-08 17:40:57 +01:00
Rusty Russell
fe1b285bba pytest: add test for generating non-standard length onion errors.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-08 17:40:57 +01:00
Rusty Russell
75c382fe16 lightningd: --dev-onion-reply-length option.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-11-08 17:40:57 +01:00
Alex Myers
341d73fdc2 reckless: fix git clone issue with removed dir
Reckless was failing to install multiple plugins due to git not
appreciating the cwd being a now removed dir after the first plugin
tmp files were cleaned up.
2022-11-08 13:19:36 +01:00
Alex Myers
a728b04243 Reckless: add man page 2022-11-08 13:19:36 +01:00
Alex Myers
e48fda1ba0 reckless: analyze repositories with urlparse 2022-11-08 13:19:36 +01:00
Alex Myers
2f4e862863 reckless: improve config file handling
While loading the appropriate lightningconfig file, it is now checked
against the active config file in lightningd. Because a deviation from the
default file structure would not be possible, a -conf option is also added
to explicitly pass the lightningd config file into reckless.
2022-11-08 13:19:36 +01:00
Alex Myers
53ad1ee576 reckless: add function for lightning-cli calls
This also simplifies dynamic enable/disable by catching the exception
raised when the cli is unable to connect to RPC (lightningd offline or
misconfigured relative to reckless).
2022-11-08 13:19:36 +01:00
Alex Myers
4a95a4c7da reckless: multiline string style cleanup 2022-11-08 13:19:36 +01:00
Alex Myers
71351ceacf reckless: replace os.path with pathlib operations
This change makes it easier to follow retrieval of parent directories.
Additional os.path operations replaced with their pathlib.Path equivalents
to keep module usage consistent.
2022-11-08 13:19:36 +01:00
Alex Myers
df98c8b927 reckless: refactor argument list handling.
The goal was to support passing a list to install, enable, etc. in order
to improve performance. Passing lists to most of the functions was less
practical than iterating through the items from the top level.
2022-11-08 13:19:36 +01:00
Alex Myers
24422e9f7c reckless: add type hints 2022-11-08 13:19:36 +01:00
Alex Myers
5d23c7ab0b reckless: raise exception or early termination instead of returning None
More pythonic than returning mixed types.
2022-11-08 13:19:36 +01:00
Alex Myers
791e521179 reckless: update help alias
`reckless help <cmd>` previously called the function docstring. This could
be updated to use the subparser help, but would require a strict naming
convention or a dictionary. Providing a hint to use the built-in
contextual help via the option flag is hopefully sufficient.
2022-11-08 13:19:36 +01:00
Alex Myers
f3934cda50 reckless: use argparse subparsers
A more pythonic approach which should also enable additional help context
for subcommands.
2022-11-08 13:19:36 +01:00
Alex Myers
651c5b6de0 reckless: use config that was explicitly passed to lightningd
Regtest environments commonly use explicit definition of the config
file for lightningd.  This can be queried and utilized by default,
saving redundant definitions between lightning and reckless.
2022-11-08 13:19:36 +01:00