Commit graph

677 commits

Author SHA1 Message Date
Rusty Russell
35e3c1866e common: generalize extract_channel_id().
connectd is going to end up using this do demux; make it fast and complete.

Fixing this reveals a problem in openingd: it now extracts the channel_id
from funding_signed (which is where we transition off the temporary), and
gets upset.  So fix that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-30 09:50:40 +10:30
Christian Decker
a98ccac777 pay: Disable channels that are not in normal state
Since we have the exact state of the channels from the `listpeers`
response we can filter out the ones that are not yet normal or not
anymore. This is more of a safety net, given that the `gossip_store`
should contain local disables, but better not be racy :-)

Changelog-None
2021-12-14 15:07:45 +10:30
ZmnSCPxj jxPCSnmZ
d088288daa bitcoin/chainparams.h: Split BIP173 name into onchain and Lightning HRPs.
Fixes: #4937
2021-12-14 11:21:35 +10:30
Denis Ahrens
522ee5c595 fix 'listincoming' field incoming_capacity_msat.
incoming_capacity_msat field showed the value as microsat.

Changelog-Fixed: JSON-RPC: listincoming showed incoming_capacity_msat field 1000 times actual value.
2021-12-14 11:19:51 +10:30
Rusty Russell
484222b0a1 daemons: remove unused functions or make static.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-06 10:05:39 +10:30
Rusty Russell
4ffda340d3 check: make sure all files outside contrib/ include "config.h" first.
And turn "" includes into full-path (which makes it easier to put
config.h first, and finds some cases check-includes.sh missed
previously).

config.h sets _GNU_SOURCE which really needs to be done before any
'#includes': we mainly got away with it with glibc, but other platforms
like Alpine may have stricter requirements.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-06 10:05:39 +10:30
Matt Whitlock
1f79aad830 common/utils: introduce tmpdir_mkstemp; use it
Various unit tests were creating temporary files unconditionally in /tmp
and were not cleaning up after themselves. Introduce a new variant of
mkstemp(3p) that respects the TMPDIR environment variable, and use it in
the offending unit tests. This allows each test run to use a dedicated
TMPDIR that can be cleaned up after the run.

Changelog-None

Signed-off-by: Matt Whitlock <c-lightning@mattwhitlock.name>
2021-12-06 09:20:27 +10:30
Andrew Toth
fa6f01d5b1 Add route exclusion payment modifier and pay rpc exclude arg
Changelog-Added: Add `exclude` option for `pay` command to manually exclude channels or nodes when finding a route.
2021-12-04 21:37:42 +10:30
Andrew Toth
384c359c79 refactor: move exclude parsing to json_tok 2021-12-04 21:37:42 +10:30
Rusty Russell
296437c655 bolt12: remove deprecated chains fields
We also move recurrence fields into a separate spec patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-01 05:44:28 +10:30
Rusty Russell
163d3a16f4 doc: clean up offers bolt quotes
As of 2b923a0367c5f9154fcec706e3302cc4658dd889.

Recurrence quotes need to be marked separately, since they're no longer
in offers main bolt.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-01 05:44:28 +10:30
Rusty Russell
5361107c9c gossipd: handle receipt of modern onion message.
And wire it through to the hook; update the plugins to recognize
modern vs obs2 onions.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-01 05:44:28 +10:30
Rusty Russell
e7b263304e lightningd: Send updated onion spec messages.
It's very similar to the previous, but there are a few changes:

1. The enctlv fields are numbered differently.
2. The message itself is a different number.

The onionmsg_path type is the same, however, so we keep that constant
at least.

The result is a lot of cut & paste, but we will delete the old one
next release.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-01 05:44:28 +10:30
Rusty Russell
5cf2c2fbd7 lightningd: return both obsolete and modern blindedpaths from "blindedpath" RPC.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-01 05:44:28 +10:30
Rusty Russell
b3af5f5a2c spec: import latest onionmessage spec, based on routeblinding.
This is from 6e99c5feaf60cb797507d181fe583224309318e9

We renamed the enctlv field to encrypted_recipient_data in the spec, and the
new onion_message is message 513.  We don't handle it until the next patch.

Two renames:
1. blinding_seed -> blinding_point.
2. enctlv -> encrypted_recipient_data.

We don't do a compat cycle for our JSON APIs for these experimental
features only used by our own plugins, we just rename.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-01 05:44:28 +10:30
Rusty Russell
1ec6346f3d common: rename current onion message structures to obs2_.
Yes, we changed the spec again.  Hopefully for the last time!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-01 05:44:28 +10:30
Rusty Russell
b74848f6f6 common: remove support for pre v0.10.2 onionmessages.
Temporarily disable sendpay_blinding test which uses obsolete onionmsg;
there's still some debate on the PR about how blinded HTLCs will work.

Changelog-EXPERIMENTAL: onionmessage: removed support for v0.10.1 onion messages.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-01 05:44:28 +10:30
Rusty Russell
166ee4bac8 plugins/Makefile: improve header dependencies.
Not all plugins depended on their headers.  Keep it simple: all
plugins depend on all plugin headers.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-01 05:44:28 +10:30
Simon Vrouwe
e08528b7b5 libplugin-pay: fix valgrind error
for the case rpc "listpeers" returns an error, such as in shutdown
2021-11-30 13:34:44 +10:30
Rusty Russell
6c9b752751 memleak: make notleak use the tal name instead of a boutique struct.
This lets us mark it directly.

Get rid of long-unused "notleaks" member of struct lightningd too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-11-27 10:38:13 +10:30
Rusty Russell
57328fe59e tests: use common_setup/common_shutdown to avoid leaks.
It also does more checks (like taken() checks).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-11-17 10:51:48 +10:30
Rusty Russell
cd36436179 paystatus: remove doubled amount_msat.
It's always been there:

```
{
   "pay": [
      {
         "bolt11": "...",
         "amount_msat": "1000msat",
         "amount_msat": "1000msat",
         "destination": "03...",
         "attempts": [
            {
               "strategy": "Initial attempt",
               "start_time": "2021-11-06T04:20:20.135Z",
               "age_in_seconds": 229032,
               "end_time": "2021-11-06T04:20:27.792Z",
               "state": "completed",
               "success": {
                  "id": 31994,
                  "payment_preimage": "..."
               }
            }
         ]
      }
   ]
}
```

Reported-by: denis2342 on IRC
Changelog-Fixed: JSON-RPC: `paystatus` entries no longer have two identical `amount_msat` entries.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-11-17 10:50:33 +10:30
Vincenzo Palazzo
e0745358d1 Fix typo in the error message.
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2021-10-30 11:12:21 +02:00
Rene Pickhardt
0ba1bc30fe pay: Use log probability based bias in channel selection
Changelog-Changed: pay: The route selection will now use the log-propability-based channel selection to increase success rate and reduce time to completion
2021-10-22 16:04:37 +02:00
Rusty Russell
233d339061 libplugin-pay: bias towards larger channels.
We bias by channel linearly by capacity, scaled by median fee.
This means that we effectively double the fee if we would use the
entire capacity, and only increase it by 50% if we would only use
1/2 the capacity.

This should drive us towards larger channels.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Plugins: `pay` now biases towards larger channels, improving success probability.
2021-10-22 16:04:37 +02:00
Rusty Russell
0b3c79b3c2 common/dijkstra: hand channel direction to path_score callback.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-22 16:04:37 +02:00
Rusty Russell
67220ca31d offers: fix overzealous BROKEN log.
sendonionmessage can fail when sending a reply, either because
the reply had a bad first peer, or because it went offline.  The
latter happens in CI, which is how I found this.

Also fixed typo "onio" -> "onion".

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-14 10:09:03 +10:30
Christian Decker
cd7d87f98e pay: listpays groups by payment_hash and groupid
Fixes #4482
Fixes #4481

Changelog-Added: pay: Payment attempts are now grouped by the pay command that initiated them
Changelog-Fixed: pay: `listpays` returns payments orderd by their creation date
Changelog-Fixed: pay: `listpays` no longer groups attempts from multiple attempts to pay an invoice
2021-10-13 13:41:18 +10:30
Christian Decker
49cddaff3d pay: Stash and forward results for duplicate pay calls 2021-10-13 13:41:18 +10:30
Christian Decker
917bea6764 pay: Mark completed payments as such by nullifying cmd
We want to avoid returning duplicate results when cross-completing, so
mark them as completed when we return a result.
2021-10-13 13:41:18 +10:30
Christian Decker
99f6faaabb libplugin: Add callbacks for successful and failed payments
We're about to suspend duplicate calls to `pay` and this will help us
notify them if the original payment completes.
2021-10-13 13:41:18 +10:30
Christian Decker
a4fb5bceb0 pay: Make pay idempotent 2021-10-13 13:41:18 +10:30
Christian Decker
5f260840ab jsonrpc: Add groupid to waitsendpay 2021-10-13 13:41:18 +10:30
Christian Decker
f384385f8e pay: Call listsendpays to find prior attempts and abort if needed
So far we've always been deferring the deletion, retry and early abort
logic to `sendonion` and `sendpay` which do not have the context to
decide if a call is legitimate or not (they were mostly based on
heuristics). By calling `listsendpays` for the invoice's
`payment_hash` we can identify what our `groupid` should be, but more
importantly we can also abort if another payment is pending or a prior
attempt has already succeeded.
2021-10-13 13:41:18 +10:30
Christian Decker
db84b984e5 pay: Add groupid to the payment struct 2021-10-13 13:41:18 +10:30
Rusty Russell
c92ce59892 BOLT 12: switch invoice_request/invoice to singular chain field.
We keep the now-removed chains field, and in deprecated mode, we set it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: bolt12: `chains` in invoice_request and invoice is deprecated, `chain` is used instead.
2021-10-08 13:47:30 +02:00
Rusty Russell
45bf7a3974 bolt12: update to latest spec.
Main changes are:
1. Uses point32 instead of pubkey32.
2. Uses issuer instead of vendor.
3. Uses byte instead of u8.
4. blinded_path num_hops is now a byte, not u16 (we don't use that yet!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: bolt12: `vendor` is deprecated: the field is now called `issuer`.
2021-10-08 13:47:30 +02:00
Rusty Russell
8f582e770c BOLT12: use point32 instead of pubkey32.
That's the modern BOLT12 term.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-08 13:47:30 +02:00
ZmnSCPxj jxPCSnmZ
d4690358d9 plugins/libplugin.c: Allow freeing notification struct command *.
We always allocate a new `struct command` when we get a full JSON
object from stdin:

b2df01dc73/plugins/libplugin.c (L1229-L1233)

If it happens to be a notification, we pass the `struct command` to
the handler, and not free it ourselves:

b2df01dc73/plugins/libplugin.c (L1270-L1275)

There are only nine points in `plugins/libplugin.c` where we `tal_free`
anything, and only one of them frees a `struct command`:

b2df01dc73/plugins/libplugin.c (L224-L234)

The above function `command_complete` is not appropriate for
notification handlers; the above function sends out a response
to our stdout, which a notification handler should not do.

However, as-is, it does mean that notification handling leaks
`struct command` objects, which can be problematic if we ever
have future built-in plugins which are significantly more
dependent on notifications.

This commit changes notification handlers to return
`struct command_result *`, because possibly in the future
notification handlers may want to perform `send_outreq`, so we
might as well use our standard convention for callbacks, and
to encourage future developers to check how to properly
terminate notification handlers (and free up the
`struct command`).

We also now provide a `notification_handled` function which a
notification handler must eventually call, as well as a
`notification_handler_pending` which is just a snowclone of
`command_still_pending`.
2021-10-08 14:40:04 +10:30
Rusty Russell
368fc07d05 offers: send a modern onion reply in response to a modern request.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Experimental: Protocol: Updated onion_message support to match updated draft specification (with backwards compat for old version)
2021-10-04 11:58:31 +02:00
Rusty Russell
af7e0a1183 offers: save replytok properly.
We carefully copied the buffer, but the tok is inside an array.  We get away
with it for now, but with coming changes it gets freed.  We need to copy
the token and all the tokens within it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-04 11:58:31 +02:00
Rusty Russell
a71f2475d5 fetchinvoice: handle modern onion_message reply.
This comes in via the onion_message_ourpath hook, and we identify the
path by checking the node alias it came to (vs the obsolete version
which used the blinding).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-04 11:58:31 +02:00
Rusty Russell
8dd038c106 fetchinvoice: send modern as well as obsolete messages.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-04 11:58:31 +02:00
Vincenzo Palazzo
9d5c992fc8 plugin: Adding status to the pay plugin
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>

Rebase

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2021-09-22 16:23:24 +09:30
Christian Decker
910e79ddb5 bcli: Accept "already in chain" errors as success 2021-09-22 09:08:48 +09:30
Christian Decker
e6d5df7918 jsonrpc: Simplify the state decision of listpays
It was really different from the way we decide the overall state of a
`pay` command's output. Now we use a more similar state decision,
based on collecting all states and checking them at the end to
determine the outcome.
2021-09-22 09:08:48 +09:30
Christian Decker
6ad8a8a3c9 pay: Fix wrong result being returned by pay
We were checking against the wrong enum value, resulting in an
erroneous `pending` state as a final result.

Changelog-Fixed: pay: `pay` would sometimes misreport a final state of `pending` instead of `failed`
2021-09-22 09:08:48 +09:30
Rusty Russell
2112f4fd8e fetchinvoice: include send in path, use pubkeys.
We had sent->path be a list of node_ids, but it makes more sense as
pubkeys so we can avoid conversion.  Also, I find it easier to think
about (especially creating backwards paths) if we include *ourselves*
as the first element in the path.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-22 06:44:26 +09:30
Rusty Russell
37ddf2e829 lightningd: rename sendonionmessage to sendobsonionmessage.
sendonionmessage is going to be the new one, and do much *less*.

As this is an internal experimental-only API, no deprecation cycle
required.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-22 06:44:26 +09:30
Rusty Russell
6aa520bb9b lightningd: remove sendonionmesage parity hack.
offers contain an x-only pubkey: to route to them to need to know the
02 vs 03 prefix.  If they're in the gossmap it's easy, but if they're
a directly-connected peer it's harder.  We used to have
sendonionmessage tweak the key if it found a peer with the matching
key, but this was always a hack.

It turns out that we try to connect to the node anyway, which is
a noop if it's already connected.  So try connecting to the other
parity if the first one fails.

Also, this registers when we fail to connect, and returns an error
rather than waiting for timeout.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-22 06:44:26 +09:30