Commit Graph

454 Commits

Author SHA1 Message Date
niftynei
86f7a179a9 channel utxo: persist the 'csv' lock value to database
Channel leases modify the CSV height that an output is eligible for
being spent at,  persist this to the database
2021-07-20 13:28:38 -04:00
niftynei
265f960cfe liquidity-ads: persist channel blockheight states to disk
Adds new tables to database, backfills, basically copies the fee_rates
state machine for channeld.
2021-07-20 13:28:38 -04:00
niftynei
a396c341cf lease_rates: persist channel's lease info 2021-07-20 13:28:38 -04:00
Christian Decker
a503032bab lightning: Add status field to htlc_in 2021-06-05 17:47:32 +09:30
Rusty Russell
220b6129bf wallet: save thresholds for option_static_remotekey.
Since we will soon be able to activate it on existing channels,
we need to mark the threshold.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-04 16:13:08 +09:30
Nalin Bhardwaj
a88a6e604c wallet: sort listsendpays by ID
Changelog-Changed: JSON: `listsendpays` output is now ordered by `id`.
2021-06-01 11:14:38 -05:00
Vincenzo Palazzo
0ed7c0d083 Suggested code cleanup by TODO comment
This commit introduces the code cleanup suggested by the TODO comment in the code.

Basically, it moves the code from the if-else statement to a switch statement without the default case. I used the basic idea of the code used in PR #4507.

Changelog-Changed: None.

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2021-05-31 11:52:07 +09:30
Rusty Russell
efe29c9db8 sendpsbt, fundpsbt, utxopsbt, reserveinputs, unreserveinputs: allow custom number of blocks to reserve.
Not an API break: reserve=true|false still works for fundpsbt and utxopsbt,
but we also allow a raw number in there.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-26 15:08:01 +09:30
Antoine Poinsot
fe8074c8c3 Refuse to parse v2 onion addresses without deprecated_apis
Tor v2 hidden services have been deprecated for a while:
https://blog.torproject.org/v2-deprecation-timeline .

This prevents user from being able to set them in the configuration
and to connect to them while still letting us be able to parse them
for gossip.

Changelog-Deprecated: lightningd: v2 Tor addresses.  Use v3.  See https://blog.torproject.org/v2-deprecation-timeline.

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-05-24 20:22:45 +09:30
niftynei
d04c373283 rbf: when a channel is open, remove all the inflights
The channel's open has been mined, we don't need to keep all of these around
now.
2021-05-24 12:17:48 +09:30
niftynei
4247ec3a05 inflights: save the whole psbt to the database
Otherwise we're missing info when we go to broadcast these and can't
properly sign the transaction to close it.

Found-by: @jasan
2021-05-24 12:17:48 +09:30
niftynei
82fa3fa2ef df-rbf: order inflights by funding_feerate
When we re-populate from disk, we need to know what order to recreate the
inflights list in.

Fixes #4511
2021-05-24 12:17:48 +09:30
Rusty Russell
9dbac21d3b doc: remove suffix for included-in-master BOLTs.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-04-07 14:34:39 +09:30
Rusty Russell
006300ab96 lightningd: set "direction" correctly for connect which is already connected.
This means remembering the connection direction.  We also use the address to try
to reconnect, which we shouldn't bother with if they connect to us.

For peers from the database, we currently always save the addr: we shouldn't really
do this if they connected to us, since it's not useful for reconnecting (we don't
show the addr in JSON reply to listpeers unless we're connected, so it's only an
internal issue).  This is left for future work.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-03-26 13:22:33 +10:30
niftynei
fba1191f06 df: actually save the peer's sigs transmission to disk
Otherwise we have no idea whether or not they actually sent their sigs
or not.
2021-03-17 10:25:18 +10:30
Rusty Russell
80c2f28373 channeld: accept the 'wrong_funding' shutdown TLV.
If it passes checks, lightningd puts it in the database.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-03-16 13:08:40 +10:30
Rusty Russell
cce280400c wallet: out "wrong_funding" information in the db for persistence.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-03-16 13:08:40 +10:30
Christian Decker
1ed08746d3 wallet: Do not generate basepoints in the hsmd
We now load them from the DB, and don't need the keys to be present at
startup.
2021-03-09 14:58:29 +10:30
Christian Decker
039751b875 wallet: Load local basepoints from the database alongside the hsmd 2021-03-09 14:58:29 +10:30
Christian Decker
ae7d722ec5 wallet: Store local channel basepoints when a new channel is created 2021-03-09 14:58:29 +10:30
niftynei
538fd670ce df: add helper for creating an "unsaved" channel
Channel that's not saved to database yet
2021-03-03 16:19:04 -06:00
niftynei
4baa10ef68 channel: remove psbt and remote_tx_sigs
We just added this, but we've migrated them to the "inflight" struct
instead, as they actually pertain to an single inflight open attempt
2021-03-03 12:28:22 +10:30
niftynei
36f3b13279 inflight: add a 'channel-inflight' concept, in-progress channel stuffs
"inflights" are all potential channel funding transactions that we
currently have commitment transactions for.
2021-03-03 12:28:22 +10:30
Karol Hosiawa
8a8f81175d Added in_channel,out_channel,state params to listforwards 2021-03-03 09:19:20 +10:30
Rusty Russell
874ca99c32 offers: make 'used' flag more useful.
We used to only set it for single-use offers (where it's required),
but it's still interesting for multi-use offers, so let's keep it
there.

We also put this field in the documentation.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-02-25 14:44:03 +10:30
Christian Decker
6252930c82 db: Set the failonion to the default value on creation
This ensures that after the migration in the previous commit we never
insert a new htlc with a null value.

Fixes: #4363

Reported-by: Zoltán Gálli <@gallizoltan>
Changelog-Fixed: db: Fixed an access to a NULL-field in the `channel_htlcs` table and resulting warning.
2021-02-25 11:06:22 +10:30
Christian Decker
87d3818c63 wallet: Fix a potential memory leak when loading state changes
The leak exists if we `tal_free` the result array onto another parent,
but the `ctx` we allocated on is still valid. This leads to a
temporary gap in the ownership tree which is then reported as the
following error:

```text
- Node /tmp/ltests-ufn3ox3p/test_htlc_out_timeout_1/lightning-1/ has memory leaks: [
   {
       "backtrace": [
           "ccan/ccan/tal/tal.c:442 (tal_alloc_)",
           "ccan/ccan/tal/tal.c:471 (tal_alloc_arr_)",
           "ccan/ccan/tal/tal.c:799 (tal_dup_)",
           "ccan/ccan/tal/str/str.c:18 (tal_strdup_)",
           "wallet/wallet.c:1652 (wallet_state_change_get)",
           "lightningd/peer_control.c:869 (json_]add_channel)",
           "lightningd/peer_control.c:1319 (json_add_peer)",
           "lightningd/peer_control.c:1348 (json_listpeers)",
           "lightningd/jsonrpc.c:643 (command_exec)",
           "lightningd/jsonrpc.c:753 (rpc_command_hook_callback)",
           "lightningd/plugin_hook.c:288 (plugin_hook_call_)",
           "lightningd/jsonrpc.c:808 (plugin_hook_call_rpc_command)",
           "lightningd/jsonrpc.c:888 (parse_request)",
           "lightningd/jsonrpc.c:979 (read_json)",
           "ccan/ccan/io/io.c:59 (next_plan)",
           "ccan/ccan/io/io.c:435 (io_do_always)",
           "ccan/ccan/io/poll.c:300 (handle_always)",
           "ccan/ccan/io/poll.c:377 (io_loop)",
           "lightningd/io_loop_with_timers.c:24 (io_loop_with_timers)",
           "lightningd/lightningd.c:1016 (main)"
       ],
       "label": "wallet/wallet.c:1652:char[]",
       "parents": [
           "common/json_stream.c:29:struct json_stream",
           "ccan/ccan/io/io.c:91:struct io_conn",
           "lightningd/lightningd.c:116:struct lightningd"
       ],
       "value": "0x556b0856ab68"
   },
```

Changelog-None
2021-02-11 09:48:55 +01:00
niftynei
8591f11188 wallet nit: consolidate stmt free into one line 2021-01-25 10:05:17 +10:30
niftynei
6778ea5d05 wallet: cleanup fee_states alloc on error 2021-01-25 10:05:17 +10:30
niftynei
fd2e16f8bc db: add flag to database for recving remote's sigs
We need to know if they've sent us their sigs message yet. Ideally, we'd
be able to check the 'finalness' of the PSBT, however if the peer
doesn't have any inputs to the channel this doesn't work.
2021-01-10 13:44:04 +01:00
Christian Decker
8e908ff652 Revert "wallet: Remove in-memory utxoset filter"
This reverts commit c239a7161b.

The goal of c239a716 was to reduce the memory footprint of our
internal UTXO set tracking, and testing against the sqlite3 backend
showed no performance impact. We have since found that the added
roundtrips to the DB server with postgres was having a considerable
performance impact, and backups would also bloat due to the increased
number of queries.

Undoing this change skips the noop updates that were causing this
regression.

Changelog-Fixed: db: Fixed a performance regression during block sync, resulting in many more queries against the DB than necessary.
2021-01-08 12:08:32 +10:30
Rusty Russell
723c16072a cleanups: feedback from Christian Decker review.
1. Hoist 7200 constant into the bolt12 heade2.
2. Make preimage the last createinvoice arg, so we could make it optional.
3. Check the validity of the preimage in createinvoice.
4. Always output used flag in listoffers.
5. Rename wallet offer iterators to offer_id iterators.
6. Fix paramter typos.
7. Rename `local_offer_id` parameter to `localofferid`.
8. Add reference constraints on local_offer_id db fields.
9. Remove cut/paste comment.
10. Clarify source of fatal() messages in wallet.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-15 12:05:02 +01:00
Rusty Russell
a33e39b7e8 pay, wallet: rename internal bolt11 vars to invstring.
And handle bolt12 strings if EXPERIMENTAL_FEATURES.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-15 12:05:02 +01:00
Rusty Russell
3f4683e3f8 sendpay: optional argument to link local offer.
This is for offers which have `send_invoice`: we need to associate the
payment with the original offer, in (the usual) case where it is a single
use offer.  We mark it used when it's paid, to avoid a race.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-15 12:05:02 +01:00
Rusty Russell
963f6b5d67 invoice: add an optional local_offer_id.
This allows us to mark an offer used when an invoice derived from it
is paid, and importantly, avoid any other invoices for the offer being
paid.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-15 12:05:02 +01:00
Rusty Russell
381b0f456c db: offer table.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-15 12:05:02 +01:00
Rusty Russell
1a3c0a0c0d invoice: fix potential race where invoice is paid/expired while we're calling hook.
There's actually a (very unlikely) race here: we would previously have
crashed with an assertion in invoices_resolve.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-02 16:50:12 +10:30
Michael Schmoock
6cc96c07dc feat: adds state_changes to listpeers output
Changelog-Added: RCP: Added 'state_changes' history to listpeers channels
2020-11-06 14:47:04 -06:00
Michael Schmoock
88c1dc56e8 db: adds a state_change history to database 2020-11-06 14:47:04 -06:00
Michael Schmoock
083a856c31 db: persist channel closer and state change cause 2020-11-06 14:47:04 -06:00
niftynei
818f152618 channel: save funding_psbt to database
Only populated until we receive the peer's funding_locked
2020-10-20 12:50:31 +10:30
Christian Decker
77ca07e91d db: Fix statement expansion bugs found through dblog mode 2020-10-10 11:34:43 +10:30
Michael Schmoock
011590b20e fix: broken SQL statement in wallet db_set_utxo
I discovered this accidentally when using the `tests/plugins/dblog.py`
plugin on another testcase: tests/test_connection.py::test_fail_unconfirmed

There the plugin/hook crashes because it can't execute the statement:
```json
{
  "jsonrpc": "2.0",
  "id": 34,
  "error": {
    "code": -32600,
    "message": "Error while processing db_write: unrecognized token: \"174WHERE\"",
    "traceback": "Traceback (most recent call last):\n  File \"/home/will/projects/lightning.git/contrib/pyln-client/pyln/client/plugin.py\", line 535, in _dispatch_request\n    result = self._exec_func(method.func, request)\n  File \"/home/will/projects/lightning.git/contrib/pyln-client/pyln/client/plugin.py\", line 520, in _exec_func\n    return func(*ba.args, **ba.kwargs)\n  File \"/home/will/projects/lightning.git/tests/plugins/dblog.py\", line 45, in db_write\n    plugin.conn.execute(c)\nsqlite3.OperationalError: unrecognized token: \"174WHERE\"\n"
  }
}
```

Changelog-Fixed: plugin: Regression with SQL statement expansion that could result in invalid statements being passed to the `db_write` hook.
2020-10-10 11:34:43 +10:30
Rusty Russell
58f6e316cc lightningd: separate fee_states out of struct channel_info.
It was always kind of weird in there anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-23 15:51:13 +09:30
Vincenzo Palazzo
f62d7bbe45 Added additional fix to code formatting and English spelling.
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2020-09-10 17:27:51 +09:30
Vincenzo Palazzo
9fbeb9bcdc delpay: code style changes and fixed docs
Changelog-Added: JSON-RPC: delpay a new method to delete the payment completed or failed.

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2020-09-10 17:27:51 +09:30
niftynei
864f2f3e21 channel_id: save to database, dont derive from funding_txid
v2 channel open uses a different method to derive the channel_id, so now
we save it to the database so that we dont have to remember how to
derive it for each.

includes a migration for existing channels
2020-09-09 19:54:20 +09:30
Christian Decker
c239a7161b wallet: Remove in-memory utxoset filter
This removes the in-memory utxoset filter, meaning we now just tracks UTXOs in
the DB.
2020-09-09 09:24:29 +09:30
Christian Decker
cab52f1197 topology: Notify gossipd about spends after processing the block
We defer the notification to gossipd till the end of the spends. By itself not
a huge change, but it allows us to later migrate to doing updates blindly and
using the DB as our ground truth. It also allows us to simplify the
`wallet_outpoint_spend` semantics to not return two values in the next commit.
2020-09-09 09:24:29 +09:30
Christian Decker
eeb6e8091b wallet: Add function to retrieve spent UTXOs for a given height
The plan is to eventually use this to tell gossipd about any outputs that were
spent by the last block.
2020-09-09 09:24:29 +09:30