Commit Graph

665 Commits

Author SHA1 Message Date
Rusty Russell
e7b263304e lightningd: Send updated onion spec messages.
It's very similar to the previous, but there are a few changes:

1. The enctlv fields are numbered differently.
2. The message itself is a different number.

The onionmsg_path type is the same, however, so we keep that constant
at least.

The result is a lot of cut & paste, but we will delete the old one
next release.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-01 05:44:28 +10:30
Rusty Russell
5cf2c2fbd7 lightningd: return both obsolete and modern blindedpaths from "blindedpath" RPC.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-01 05:44:28 +10:30
Rusty Russell
b3af5f5a2c spec: import latest onionmessage spec, based on routeblinding.
This is from 6e99c5feaf60cb797507d181fe583224309318e9

We renamed the enctlv field to encrypted_recipient_data in the spec, and the
new onion_message is message 513.  We don't handle it until the next patch.

Two renames:
1. blinding_seed -> blinding_point.
2. enctlv -> encrypted_recipient_data.

We don't do a compat cycle for our JSON APIs for these experimental
features only used by our own plugins, we just rename.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-01 05:44:28 +10:30
Rusty Russell
1ec6346f3d common: rename current onion message structures to obs2_.
Yes, we changed the spec again.  Hopefully for the last time!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-01 05:44:28 +10:30
Rusty Russell
b74848f6f6 common: remove support for pre v0.10.2 onionmessages.
Temporarily disable sendpay_blinding test which uses obsolete onionmsg;
there's still some debate on the PR about how blinded HTLCs will work.

Changelog-EXPERIMENTAL: onionmessage: removed support for v0.10.1 onion messages.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-01 05:44:28 +10:30
Rusty Russell
166ee4bac8 plugins/Makefile: improve header dependencies.
Not all plugins depended on their headers.  Keep it simple: all
plugins depend on all plugin headers.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-01 05:44:28 +10:30
Simon Vrouwe
e08528b7b5 libplugin-pay: fix valgrind error
for the case rpc "listpeers" returns an error, such as in shutdown
2021-11-30 13:34:44 +10:30
Rusty Russell
6c9b752751 memleak: make notleak use the tal name instead of a boutique struct.
This lets us mark it directly.

Get rid of long-unused "notleaks" member of struct lightningd too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-11-27 10:38:13 +10:30
Rusty Russell
57328fe59e tests: use common_setup/common_shutdown to avoid leaks.
It also does more checks (like taken() checks).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-11-17 10:51:48 +10:30
Rusty Russell
cd36436179 paystatus: remove doubled amount_msat.
It's always been there:

```
{
   "pay": [
      {
         "bolt11": "...",
         "amount_msat": "1000msat",
         "amount_msat": "1000msat",
         "destination": "03...",
         "attempts": [
            {
               "strategy": "Initial attempt",
               "start_time": "2021-11-06T04:20:20.135Z",
               "age_in_seconds": 229032,
               "end_time": "2021-11-06T04:20:27.792Z",
               "state": "completed",
               "success": {
                  "id": 31994,
                  "payment_preimage": "..."
               }
            }
         ]
      }
   ]
}
```

Reported-by: denis2342 on IRC
Changelog-Fixed: JSON-RPC: `paystatus` entries no longer have two identical `amount_msat` entries.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-11-17 10:50:33 +10:30
Vincenzo Palazzo
e0745358d1 Fix typo in the error message.
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2021-10-30 11:12:21 +02:00
Rene Pickhardt
0ba1bc30fe pay: Use log probability based bias in channel selection
Changelog-Changed: pay: The route selection will now use the log-propability-based channel selection to increase success rate and reduce time to completion
2021-10-22 16:04:37 +02:00
Rusty Russell
233d339061 libplugin-pay: bias towards larger channels.
We bias by channel linearly by capacity, scaled by median fee.
This means that we effectively double the fee if we would use the
entire capacity, and only increase it by 50% if we would only use
1/2 the capacity.

This should drive us towards larger channels.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Plugins: `pay` now biases towards larger channels, improving success probability.
2021-10-22 16:04:37 +02:00
Rusty Russell
0b3c79b3c2 common/dijkstra: hand channel direction to path_score callback.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-22 16:04:37 +02:00
Rusty Russell
67220ca31d offers: fix overzealous BROKEN log.
sendonionmessage can fail when sending a reply, either because
the reply had a bad first peer, or because it went offline.  The
latter happens in CI, which is how I found this.

Also fixed typo "onio" -> "onion".

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-14 10:09:03 +10:30
Christian Decker
cd7d87f98e pay: listpays groups by payment_hash and groupid
Fixes #4482
Fixes #4481

Changelog-Added: pay: Payment attempts are now grouped by the pay command that initiated them
Changelog-Fixed: pay: `listpays` returns payments orderd by their creation date
Changelog-Fixed: pay: `listpays` no longer groups attempts from multiple attempts to pay an invoice
2021-10-13 13:41:18 +10:30
Christian Decker
49cddaff3d pay: Stash and forward results for duplicate pay calls 2021-10-13 13:41:18 +10:30
Christian Decker
917bea6764 pay: Mark completed payments as such by nullifying cmd
We want to avoid returning duplicate results when cross-completing, so
mark them as completed when we return a result.
2021-10-13 13:41:18 +10:30
Christian Decker
99f6faaabb libplugin: Add callbacks for successful and failed payments
We're about to suspend duplicate calls to `pay` and this will help us
notify them if the original payment completes.
2021-10-13 13:41:18 +10:30
Christian Decker
a4fb5bceb0 pay: Make pay idempotent 2021-10-13 13:41:18 +10:30
Christian Decker
5f260840ab jsonrpc: Add groupid to waitsendpay 2021-10-13 13:41:18 +10:30
Christian Decker
f384385f8e pay: Call listsendpays to find prior attempts and abort if needed
So far we've always been deferring the deletion, retry and early abort
logic to `sendonion` and `sendpay` which do not have the context to
decide if a call is legitimate or not (they were mostly based on
heuristics). By calling `listsendpays` for the invoice's
`payment_hash` we can identify what our `groupid` should be, but more
importantly we can also abort if another payment is pending or a prior
attempt has already succeeded.
2021-10-13 13:41:18 +10:30
Christian Decker
db84b984e5 pay: Add groupid to the payment struct 2021-10-13 13:41:18 +10:30
Rusty Russell
c92ce59892 BOLT 12: switch invoice_request/invoice to singular chain field.
We keep the now-removed chains field, and in deprecated mode, we set it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: bolt12: `chains` in invoice_request and invoice is deprecated, `chain` is used instead.
2021-10-08 13:47:30 +02:00
Rusty Russell
45bf7a3974 bolt12: update to latest spec.
Main changes are:
1. Uses point32 instead of pubkey32.
2. Uses issuer instead of vendor.
3. Uses byte instead of u8.
4. blinded_path num_hops is now a byte, not u16 (we don't use that yet!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: bolt12: `vendor` is deprecated: the field is now called `issuer`.
2021-10-08 13:47:30 +02:00
Rusty Russell
8f582e770c BOLT12: use point32 instead of pubkey32.
That's the modern BOLT12 term.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-08 13:47:30 +02:00
ZmnSCPxj jxPCSnmZ
d4690358d9 plugins/libplugin.c: Allow freeing notification struct command *.
We always allocate a new `struct command` when we get a full JSON
object from stdin:

b2df01dc73/plugins/libplugin.c (L1229-L1233)

If it happens to be a notification, we pass the `struct command` to
the handler, and not free it ourselves:

b2df01dc73/plugins/libplugin.c (L1270-L1275)

There are only nine points in `plugins/libplugin.c` where we `tal_free`
anything, and only one of them frees a `struct command`:

b2df01dc73/plugins/libplugin.c (L224-L234)

The above function `command_complete` is not appropriate for
notification handlers; the above function sends out a response
to our stdout, which a notification handler should not do.

However, as-is, it does mean that notification handling leaks
`struct command` objects, which can be problematic if we ever
have future built-in plugins which are significantly more
dependent on notifications.

This commit changes notification handlers to return
`struct command_result *`, because possibly in the future
notification handlers may want to perform `send_outreq`, so we
might as well use our standard convention for callbacks, and
to encourage future developers to check how to properly
terminate notification handlers (and free up the
`struct command`).

We also now provide a `notification_handled` function which a
notification handler must eventually call, as well as a
`notification_handler_pending` which is just a snowclone of
`command_still_pending`.
2021-10-08 14:40:04 +10:30
Rusty Russell
368fc07d05 offers: send a modern onion reply in response to a modern request.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Experimental: Protocol: Updated onion_message support to match updated draft specification (with backwards compat for old version)
2021-10-04 11:58:31 +02:00
Rusty Russell
af7e0a1183 offers: save replytok properly.
We carefully copied the buffer, but the tok is inside an array.  We get away
with it for now, but with coming changes it gets freed.  We need to copy
the token and all the tokens within it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-04 11:58:31 +02:00
Rusty Russell
a71f2475d5 fetchinvoice: handle modern onion_message reply.
This comes in via the onion_message_ourpath hook, and we identify the
path by checking the node alias it came to (vs the obsolete version
which used the blinding).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-04 11:58:31 +02:00
Rusty Russell
8dd038c106 fetchinvoice: send modern as well as obsolete messages.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-04 11:58:31 +02:00
Vincenzo Palazzo
9d5c992fc8 plugin: Adding status to the pay plugin
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>

Rebase

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2021-09-22 16:23:24 +09:30
Christian Decker
910e79ddb5 bcli: Accept "already in chain" errors as success 2021-09-22 09:08:48 +09:30
Christian Decker
e6d5df7918 jsonrpc: Simplify the state decision of listpays
It was really different from the way we decide the overall state of a
`pay` command's output. Now we use a more similar state decision,
based on collecting all states and checking them at the end to
determine the outcome.
2021-09-22 09:08:48 +09:30
Christian Decker
6ad8a8a3c9 pay: Fix wrong result being returned by pay
We were checking against the wrong enum value, resulting in an
erroneous `pending` state as a final result.

Changelog-Fixed: pay: `pay` would sometimes misreport a final state of `pending` instead of `failed`
2021-09-22 09:08:48 +09:30
Rusty Russell
2112f4fd8e fetchinvoice: include send in path, use pubkeys.
We had sent->path be a list of node_ids, but it makes more sense as
pubkeys so we can avoid conversion.  Also, I find it easier to think
about (especially creating backwards paths) if we include *ourselves*
as the first element in the path.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-22 06:44:26 +09:30
Rusty Russell
37ddf2e829 lightningd: rename sendonionmessage to sendobsonionmessage.
sendonionmessage is going to be the new one, and do much *less*.

As this is an internal experimental-only API, no deprecation cycle
required.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-22 06:44:26 +09:30
Rusty Russell
6aa520bb9b lightningd: remove sendonionmesage parity hack.
offers contain an x-only pubkey: to route to them to need to know the
02 vs 03 prefix.  If they're in the gossmap it's easy, but if they're
a directly-connected peer it's harder.  We used to have
sendonionmessage tweak the key if it found a peer with the matching
key, but this was always a hack.

It turns out that we try to connect to the node anyway, which is
a noop if it's already connected.  So try connecting to the other
parity if the first one fails.

Also, this registers when we fail to connect, and returns an error
rather than waiting for timeout.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-22 06:44:26 +09:30
Rusty Russell
24536c5561 common/autodata: use instead of ccan/autodata
This means it needs to be linked ~everywhere.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-21 18:04:43 +02:00
Jan Sarenik
284ad2bade Fix for Alpine Linux ond OpenBSD
After recent header files clean-up it was not possible to
build c-lightning 7401b2682. This patch fixes it both for
Alpine Linux and OpenBSD.

Proposed-by: nathanael <nathanael@dalliard.ch>

Changelog-None
2021-09-20 14:44:27 +02:00
Rusty Russell
7401b26824 cleanup: remove unneeded includes in C files.
Before:
 Ten builds, laptop -j5, no ccache:

```
real	0m36.686000-38.956000(38.608+/-0.65)s
user	2m32.864000-42.253000(40.7545+/-2.7)s
sys	0m16.618000-18.316000(17.8531+/-0.48)s
```

 Ten builds, laptop -j5, ccache (warm):

```
real	0m8.212000-8.577000(8.39989+/-0.13)s
user	0m12.731000-13.212000(12.9751+/-0.17)s
sys	0m3.697000-3.902000(3.83722+/-0.064)s
```

After:
 Ten builds, laptop -j5, no ccache: 8% faster

```
real	0m33.802000-35.773000(35.468+/-0.54)s
user	2m19.073000-27.754000(26.2542+/-2.3)s
sys	0m15.784000-17.173000(16.7165+/-0.37)s
```

 Ten builds, laptop -j5, ccache (warm): 1% faster

```
real	0m8.200000-8.485000(8.30138+/-0.097)s
user	0m12.485000-13.100000(12.7344+/-0.19)s
sys	0m3.702000-3.889000(3.78787+/-0.056)s
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-17 09:43:22 +09:30
Rusty Russell
ea30c34d82 cleanup: remove unneeded includes in header files.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-17 09:43:22 +09:30
Rusty Russell
cb22015b2a common/channel_type: wrapper for generated 'struct channel_type'.
We make it a first-class citizen internally, even though we won't use
it over the wire (at least, non-experimental builds).  This scheme
follows the latest draft, in which features are flagged compulsory.

We also add several helper functions.

Since uses the *even* bits (as per latest spec), not the *odd* bits,
we have some other fixups.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-13 15:53:23 +02:00
Rusty Russell
31f439760f libplugin: make leaks log at LOG_BROKEN so they break CI.
Now we've fixed them, this makes sure CI notices if new leaks appear.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-08 19:11:47 +02:00
Rusty Russell
14002915a1 plugins/libplugin: mark timers as not-a-leak.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-08 19:11:47 +02:00
Rusty Russell
4f4c49c88a libplugin: fix leak of struct command when we don't read it all at once.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Plugins: C plugins would could leak memory on every command (esp. seen when hammering topology's listchannels).
2021-09-08 19:11:47 +02:00
Rusty Russell
127539a993 plugins/topology: use memleak annotation instead of global_gossmap hammer.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-08 19:11:47 +02:00
Rusty Russell
68470c8f22 plugins/txprepare: annotate unreleased_txs list against false memleaks.
And fix utx leak in the withdraw case.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-08 19:11:47 +02:00
Rusty Russell
2e0179ffcc plugins/bcli: keep a list of current bitcoind requests.
This lets memleak track them, but makes sure they don't leak; using
notleak could cover up a leak here.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-08 19:11:47 +02:00
Rusty Russell
02553aa68a plugins/spender: don't use global tal context, use take() instead.
Otherwise it looks like a leak.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-08 19:11:47 +02:00