Commit Graph

10469 Commits

Author SHA1 Message Date
Rusty Russell
bdaec48400 wallet: wrap htlc_state enum in db function.
All enums in the db should be wrapped this way on reading/writing them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-15 12:09:36 +02:00
Rusty Russell
2ab4e5b42b utils: add max_unsigned/min_unsigned helpers.
We are usually dealing with unsigned values, so use this.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-15 12:09:36 +02:00
Rusty Russell
012dedc3d8 fuzz: make it build again.
How did this pass CI?  I saw this break in my PR, but it's (long)
broken in master.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-15 12:09:36 +02:00
Vincenzo Palazzo
67b790e3dc tests: Change currency from msat to sat in the regex string
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2021-10-14 10:09:20 +10:30
Vincenzo Palazzo
015627cdee Fixes msat and sat convention in the error message
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2021-10-14 10:09:20 +10:30
Rusty Russell
67220ca31d offers: fix overzealous BROKEN log.
sendonionmessage can fail when sending a reply, either because
the reply had a bad first peer, or because it went offline.  The
latter happens in CI, which is how I found this.

Also fixed typo "onio" -> "onion".

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-14 10:09:03 +10:30
Christian Decker
cd7d87f98e pay: listpays groups by payment_hash and groupid
Fixes #4482
Fixes #4481

Changelog-Added: pay: Payment attempts are now grouped by the pay command that initiated them
Changelog-Fixed: pay: `listpays` returns payments orderd by their creation date
Changelog-Fixed: pay: `listpays` no longer groups attempts from multiple attempts to pay an invoice
2021-10-13 13:41:18 +10:30
Christian Decker
49cddaff3d pay: Stash and forward results for duplicate pay calls 2021-10-13 13:41:18 +10:30
Christian Decker
917bea6764 pay: Mark completed payments as such by nullifying cmd
We want to avoid returning duplicate results when cross-completing, so
mark them as completed when we return a result.
2021-10-13 13:41:18 +10:30
Christian Decker
98f74762a5 pytest: Adjust test_sendpay to the new semantics 2021-10-13 13:41:18 +10:30
Christian Decker
99f6faaabb libplugin: Add callbacks for successful and failed payments
We're about to suspend duplicate calls to `pay` and this will help us
notify them if the original payment completes.
2021-10-13 13:41:18 +10:30
Christian Decker
ce3d3d8e54 pytest: Fix test_onchain_timeout to use groupid 2021-10-13 13:41:18 +10:30
Christian Decker
428982a9e7 pytest: Add groupid to test_partial_payment_{timeout,restart} 2021-10-13 13:41:18 +10:30
Christian Decker
6196d2f559 pay: Fail a sendpay or sendonion that'd produce a DB collision 2021-10-13 13:41:18 +10:30
Christian Decker
bc089d3350 pytest: Fix up test_partial_payment to use a single groupid 2021-10-13 13:41:18 +10:30
Christian Decker
818a4d14cb paycore: Default groupid to increment from last one
This re-establishes the prior behavior where a `sendpay` or
`sendonion` that'd match a prior payment would cause the prior payment
to be deleted. While we no longer delete prior attempts we now avoid a
primary key collision by incrementing once. This helps us not having
to touch all existing tests, and likely avoids breaking other users
too.
2021-10-13 13:41:18 +10:30
Christian Decker
ec9040577e paycore: Prevent multiple concurrent payment groups
One of the fundamental constraints of the payment groups idea is that
there may only ever be one group in flight at any point in time, so if
we find a group that is in flight, any new `sendpay` or `sendonion`
must match its `groupid`.
2021-10-13 13:41:18 +10:30
Christian Decker
39248b5fa7 pay: Do not delete old sendpay attempts if we retry
This was the main cause of the pay states flip-flopping, since we
reset the status on each attempt any final status is not really
final. Let's keep them around, and provide a stable history.
2021-10-13 13:41:18 +10:30
Christian Decker
a4fb5bceb0 pay: Make pay idempotent 2021-10-13 13:41:18 +10:30
Christian Decker
5f260840ab jsonrpc: Add groupid to waitsendpay 2021-10-13 13:41:18 +10:30
Christian Decker
511cb14fbe doc: Add missing amount_sent_msat to listpays 2021-10-13 13:41:18 +10:30
Christian Decker
f384385f8e pay: Call listsendpays to find prior attempts and abort if needed
So far we've always been deferring the deletion, retry and early abort
logic to `sendonion` and `sendpay` which do not have the context to
decide if a call is legitimate or not (they were mostly based on
heuristics). By calling `listsendpays` for the invoice's
`payment_hash` we can identify what our `groupid` should be, but more
importantly we can also abort if another payment is pending or a prior
attempt has already succeeded.
2021-10-13 13:41:18 +10:30
Christian Decker
a9eb7b427f jsonrpc: Add missing partid in listsendpays schema 2021-10-13 13:41:18 +10:30
Christian Decker
817b1b361e db: Add groupid to HTLCs 2021-10-13 13:41:18 +10:30
Christian Decker
0bb0d8a707 wallet: Add function to retrieve the latest groupid for a payment
When doing things like `waitsendpay` without specifying the `groupid`
we likely want to use the latest `groupid` we created, since that's
the one in flight. This adds a function to quickly retrieve that.
2021-10-13 13:41:18 +10:30
Christian Decker
db84b984e5 pay: Add groupid to the payment struct 2021-10-13 13:41:18 +10:30
Christian Decker
24701600ee pyln: Add groupid to sendpay and sendonion 2021-10-13 13:41:18 +10:30
Christian Decker
a1f509e4b5 jsonrpc: Add groupid to sendpay and sendonion
Also add `groupid` to the payment fields so we can retrieve them too.
2021-10-13 13:41:18 +10:30
Christian Decker
ba4be0313b db: Add groupid to the payments table 2021-10-13 13:41:18 +10:30
Christian Decker
324257b2bf pytest: Reproduce #4482 2021-10-13 13:41:18 +10:30
Rusty Russell
d9053ad09f closingd: ignore spurious pong messages.
channeld may have sent a ping before shutdown, now closingd sees it.

```
lightningd-2: 2021-10-11T20:55:18.892Z DEBUG   0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-closingd-chan#1: peer_out WIRE_WARNING
lightningd-2: 2021-10-11T20:55:18.949Z DEBUG   0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-closingd-chan#1: billboard perm: Expected closing_signed: 0013000166
lightningd-2: 2021-10-11T20:55:18.983Z INFO    0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-chan#1: Peer transient failure in CLOSINGD_SIGEXCHANGE: closingd WARNING: Expected closing_signed: 0013000166
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-12 15:03:34 +10:30
Christian Decker
1cc349ef17 pyln: Add missing requirement from pyln-client to pyln-proto
Gossmap makes use of the proto, this ensures we have it.
2021-10-12 09:15:48 +10:30
Christian Decker
cc7985df92 pytest: Use the same TIMEOUT in wait_for_route
This was getting flaky whenever we had multihop routing syncs and had
`DEVELOPER=0`, which uses the slow gossip propagation.
2021-10-12 09:15:48 +10:30
Christian Decker
39022028e9 db: Ensure sqlite3 library minimum version and major version match
We were really strict with the version check, requiring an exact
match. We now check that we are running at least with the version we
were compiled with (distro upgrades continue to work, and repro builds
are built off of an unupdated installation matching this minimum
requirement), and a major version match (since major versions can and
will introduce breaking changes).

Changelog-Fixed: sqlite3: Relaxed the version match requirements to be at least a minimum version and a major version match
2021-10-12 09:15:48 +10:30
Rusty Russell
be3eaa475c doc: update ungenerated file.
Commit c92ce59892 somehow missed this.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-11 10:41:50 +02:00
Rusty Russell
c936540e0b pytest: fix flake in test_upgrade_statickey_onchaind
Reconnect manually, don't wait for automatic reconnect.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-10 15:32:57 +02:00
Rusty Russell
b2206744c0 pytest: fix flake when test reconnects by itself.
If it reconnects by itself, it will get a warning message:

```
lightningd-2: 2021-10-08T01:40:42.446Z DEBUG   0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-channeld-chan#3: billboard: Sent reestablish, waiting for theirs
lightningd-2: 2021-10-08T01:40:42.446Z DEBUG   0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-channeld-chan#3: peer_in WIRE_ERROR
lightningd-2: 2021-10-08T01:40:42.447Z DEBUG   0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-channeld-chan#3: billboard perm: Received error channel 0a6220a3e904d17e72b5c3499928dc8a65720063c6395c999a129a0ff0b06afb: Forcibly closed by `close` command timeout
lightningd-2: 2021-10-08T01:40:42.448Z INFO    0266e4598d1d3c415f572a8488830b60f7e744ed9235eb0b1ba93283b315c03518-chan#3: Peer transient failure in CHANNELD_NORMAL: channeld WARNING: error channel 0a6220a3e904d17e72b5c3499928dc8a65720063c6395c999a129a0ff0b06afb: Forcibly closed by `close` command timeout
```

And this will make CI complain.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-10 15:32:57 +02:00
Rusty Russell
c9b82bf1d2 channeld: restore ping command, but only for channeld.
It's probably not worth fixing for the other daemons.

Changelog-Changed: JSON-RPC: `ping` now only works if we have a channel with the peer.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-10 15:32:57 +02:00
Rusty Russell
be2622a4ff channeld: perform regular keepalive pings.
Send a ping every 15-45 seconds.  If we try to send another one and we
haven't got a reply, hang up.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: Send regular pings to detect dead connections (particularly for Tor).
2021-10-10 15:32:57 +02:00
Rusty Russell
140b2deec0 channeld: fix halting gossip when a timer is active.
We would sleep until the next timer, even if that's long past when we would
send gossip.  Normally we use very short timers, so we didn't notice, but
we will in the next patch, where we use continuous timers for pings.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-10 15:32:57 +02:00
Rusty Russell
9138ebf807 channeld: remove liveness logic pre-commitment.
We're going to continuously ping, so this is redundant.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-10 15:32:57 +02:00
Rusty Russell
1c85b27b4c gossipd: remove ping/pong handling
To minimize the diffs, we #if 0 the code.  We'll reenable it once
channeld is ready.

We also temporarily disable the ping tests.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-10 15:32:57 +02:00
Rusty Russell
c394fd5db2 common/read_peer_msg: handle pings in handle_peer_gossip_or_error().
This is a noop for now, since gossipd handles them.  But that
will change in the next patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-10 15:32:57 +02:00
Dustin Dettmer
6dc7fc6aee Fix memory leak abort in debug mode 2021-10-10 13:22:29 +02:00
Dustin Dettmer
3aa31d7cfd On Mac OS X a different error is raised on failed connection
[ Typo fixed --RR ]
2021-10-10 13:22:29 +02:00
Christian Decker
c963e6b03f db: Don't switch between text and blob fields
The database will yell at you if you try.
2021-10-10 11:05:01 +10:30
Christian Decker
b9c602c7e4 db: Fix a syntax error with the optional parameters
SQL doesn't really allow `a OR 1` as a clause since `1` is not a
boolean expression. Moving it into `a OR 1=1` however is valid again.
2021-10-10 11:05:01 +10:30
Christian Decker
8d3861ee29 ci: Add missing envvar forward for the DB configuration
We were not forwarding the matrix DB configuration, so we were only
ever testing `sqlite3`.
2021-10-10 11:05:01 +10:30
Rusty Russell
1534216aad channeld: handle reestablish from previous release with EXPERIMENTAL_FEATURES.
We switched channel_types from optional to compulsory bits in
cb22015b2a.

The result is infinite reconnects against older nodes; we reject what
they send, and they reject what we send.

The simplest fix is to neither send nor receive the (optional!) tlvs
unless we both advertize option_quiesce, which we now do.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-EXPERIMENTAL: channel_upgrade draft upgraded: cannot upgrade channels until peers also upgrade.
2021-10-08 16:07:21 +02:00
Rusty Russell
7157a92ea6 channeld: import updated channel_upgrade spec.
It now uses raw bitfields instead of a subtype, and only allows a single
option for any upgrade.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-08 16:07:21 +02:00