Commit Graph

487 Commits

Author SHA1 Message Date
Rusty Russell
2e6df89274 wallet: remove redundant field counters, ignored fields.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-11-17 10:51:48 +10:30
Rusty Russell
c462ccae1a wallet: have db track what columns are accessed in DEVELOPER mode.
And add db_col_ignore helper for cases where it's deliberate.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-11-17 10:51:48 +10:30
Rusty Russell
b8a240bd15 wallet: convert wallet.c to db_col_ APIs
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-11-17 10:51:48 +10:30
Rusty Russell
787fbb1228 db: create simple hashtable of fields in SELECT.
This simplistically maps names to numbers, eg:

	SELECT foo, bar FROM tbl;

'foo' -> 0
'bar' -> 1

If a statement is too complex for our simple parsing, we treat it as a
single field (which currently it always is).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-11-17 10:51:48 +10:30
niftynei
42e40c1ced htlcs: add flag to 'fail immediately'
If we're over the dust limit, we fail it immediatey *after* commiting
it, but we need a way to signal this throughout the lifecycle, so we add
it to htlc_in struct and persist it through to the database.

If it's supposed to be failed, we fail after the commit cycle is
completed.
2021-10-23 12:59:13 +02:00
niftynei
1fe829c546 lightningd: new option for htlc dust limit
To reduce the surface area of amount of a channel balance that can be
eaten up as htlc dust, we introduce a new config
'--max-dust-htlc-exposure-msat', which sets the max amount that any
channel's balance can be added as dust

Changelog-Added: config: new option --max-dust-htlc-exposure-msat, which limits the total amount of sats to be allowed as dust on a channel
2021-10-23 12:59:13 +02:00
Rusty Russell
da03985996 wallet: only hand onchaind the HTLCs it needs to know.
This will make closing long-lived channels more efficient, and it's
just nicer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-15 12:09:36 +02:00
Rusty Russell
c503232cde common: use bitcoin_outpoint.
I started pulling this thread, and the entire codebase got unravelled.

Oh well, it's done now!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-15 12:09:36 +02:00
Rusty Russell
433e61fcb9 wallet: remove unnecessary data from channel_htlcs when htlc is dead.
In particular, the onion and errors can be large, but now we'll never
need to retransmit them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-15 12:09:36 +02:00
Rusty Russell
8a85bf6880 lightningd: populate min/max commit fields in db.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-15 12:09:36 +02:00
Rusty Russell
bdaec48400 wallet: wrap htlc_state enum in db function.
All enums in the db should be wrapped this way on reading/writing them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-15 12:09:36 +02:00
Christian Decker
5f260840ab jsonrpc: Add groupid to waitsendpay 2021-10-13 13:41:18 +10:30
Christian Decker
817b1b361e db: Add groupid to HTLCs 2021-10-13 13:41:18 +10:30
Christian Decker
0bb0d8a707 wallet: Add function to retrieve the latest groupid for a payment
When doing things like `waitsendpay` without specifying the `groupid`
we likely want to use the latest `groupid` we created, since that's
the one in flight. This adds a function to quickly retrieve that.
2021-10-13 13:41:18 +10:30
Christian Decker
ba4be0313b db: Add groupid to the payments table 2021-10-13 13:41:18 +10:30
Christian Decker
c963e6b03f db: Don't switch between text and blob fields
The database will yell at you if you try.
2021-10-10 11:05:01 +10:30
Christian Decker
b9c602c7e4 db: Fix a syntax error with the optional parameters
SQL doesn't really allow `a OR 1` as a clause since `1` is not a
boolean expression. Moving it into `a OR 1=1` however is valid again.
2021-10-10 11:05:01 +10:30
Vincenzo Palazzo
eb103c15df review 2/2: Use generic query to make the code more readable.
Suggested by @cdecker

P.S: Also this include an API refactoring from my previous solution, also this it is suggested by @cdecker.

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2021-09-22 16:23:24 +09:30
Vincenzo Palazzo
ee8c7252a3 wallet db: Support the query on database with status and payment hash.
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2021-09-22 16:23:24 +09:30
Vincenzo Palazzo
fd33aed4b5 rpc: Integrate the status flow in the listsendpays command
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2021-09-22 16:23:24 +09:30
Rusty Russell
7401b26824 cleanup: remove unneeded includes in C files.
Before:
 Ten builds, laptop -j5, no ccache:

```
real	0m36.686000-38.956000(38.608+/-0.65)s
user	2m32.864000-42.253000(40.7545+/-2.7)s
sys	0m16.618000-18.316000(17.8531+/-0.48)s
```

 Ten builds, laptop -j5, ccache (warm):

```
real	0m8.212000-8.577000(8.39989+/-0.13)s
user	0m12.731000-13.212000(12.9751+/-0.17)s
sys	0m3.697000-3.902000(3.83722+/-0.064)s
```

After:
 Ten builds, laptop -j5, no ccache: 8% faster

```
real	0m33.802000-35.773000(35.468+/-0.54)s
user	2m19.073000-27.754000(26.2542+/-2.3)s
sys	0m15.784000-17.173000(16.7165+/-0.37)s
```

 Ten builds, laptop -j5, ccache (warm): 1% faster

```
real	0m8.200000-8.485000(8.30138+/-0.097)s
user	0m12.485000-13.100000(12.7344+/-0.19)s
sys	0m3.702000-3.889000(3.78787+/-0.056)s
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-17 09:43:22 +09:30
Rusty Russell
ea30c34d82 cleanup: remove unneeded includes in header files.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-17 09:43:22 +09:30
Rusty Russell
183fe107e8 lightningd: use channel_type, pass to-and-from channeld.
Instead of explicit option_static_remotekey and option_anchor_outputs flags.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-13 15:53:23 +02:00
Rusty Russell
fb4edc2938 Makefile: update bolt version to include option_anchors_zero_fee_htlc_tx.
This touches a lot of text, mainly to change "if `option_anchor_outputs`"
to "if `option_anchors`"

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-09 12:04:48 +09:30
Rusty Russell
fe86c117d9 datastore: turn keys into arrays
After some discussion with @shesek, and my own usage, we agreed that
a more comprehensive interface, which explicitly supports grouping,
is desirable.

Thus keys are now arrays, with the semantic that a key is either a
parent or has a value, never both.

For convenience in the JSON schema, we always return them as arrays,
though we accept simple strings as arguments.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-08-25 10:06:31 +02:00
Rusty Russell
533571a655 datastore: add generation, simple atomicity.
We add a generation counter, and allow update or del conditional
on a given generation.

Formalizes error codes, too, since we have more now.

Suggested-by: @shesek
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-08-25 10:06:31 +02:00
Rusty Russell
432508e65e datastore: allow replace/append.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-08-25 10:06:31 +02:00
Rusty Russell
2fb8735f04 wallet: add datastore access routines.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-08-25 10:06:31 +02:00
niftynei
773aa26321 utxos: more comprehensive fix for csv check
we only 'need' to check the CSV for 'option_anchor_outputs' (and even
then, really only ones with csv's greater than 1 but...)

Reported-In: #4697
2021-08-17 13:20:00 +09:30
Rusty Russell
66af5f8a28 chaintopology: tell gossipd that channels no longer exist on reorg.
This actually caused the flake in test_funding_reorg_private, where
l1 and l2 might not mark the original channel disabled.  In fact, they
should *remove* it as it gets reorged out.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-08-14 12:07:38 +09:30
Antoine Poinsot
8157e378f4 wallet: correct CSV check in deep_enough
Return false if the timelock didn't mature yet, not the other way
around.

Also, the check shouldn't be strict: if the CSV is 1 it is valid
at utxo->blockheight + 1.

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-08-09 13:57:36 -05:00
Rusty Russell
44c469d52b disableoffer: fix disabling of already-used offers.
Turns out we didn't actually test this at all, and next commit does :(

    offer_status_in_db: 4 is invalid
    lightningd: FATAL SIGNAL 6 (version v0.10.0-459-g48fbd45-modded)
    0x5608cd360855 send_backtrace
	common/daemon.c:39
    0x5608cd3608ff crashdump
	common/daemon.c:52
    0x7f9af1dae20f ???
	???:0
    0x7f9af1dae18b ???
	???:0
    0x7f9af1d8d858 ???
	???:0
    0x5608cd30a47e fatal
	lightningd/log.c:819
    0x5608cd3430c5 offer_status_in_db
	wallet/wallet.h:1424
    0x5608cd34f1f3 wallet_offer_disable
	wallet/wallet.c:4494
    0x5608cd33ae2e json_disableoffer
	lightningd/offer.c:256
    0x5608cd3038fc command_exec
	lightningd/jsonrpc.c:643

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-07-21 13:27:27 -04:00
niftynei
03ccebc296 utxo: don't add outputs that aren't currently eligible to be spent
If an output's CSV lock hasn't been surpassed yet, don't try to
include it in a transaction
2021-07-20 13:28:38 -04:00
niftynei
86f7a179a9 channel utxo: persist the 'csv' lock value to database
Channel leases modify the CSV height that an output is eligible for
being spent at,  persist this to the database
2021-07-20 13:28:38 -04:00
niftynei
265f960cfe liquidity-ads: persist channel blockheight states to disk
Adds new tables to database, backfills, basically copies the fee_rates
state machine for channeld.
2021-07-20 13:28:38 -04:00
niftynei
a396c341cf lease_rates: persist channel's lease info 2021-07-20 13:28:38 -04:00
Christian Decker
a503032bab lightning: Add status field to htlc_in 2021-06-05 17:47:32 +09:30
Rusty Russell
220b6129bf wallet: save thresholds for option_static_remotekey.
Since we will soon be able to activate it on existing channels,
we need to mark the threshold.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-04 16:13:08 +09:30
Nalin Bhardwaj
a88a6e604c wallet: sort listsendpays by ID
Changelog-Changed: JSON: `listsendpays` output is now ordered by `id`.
2021-06-01 11:14:38 -05:00
Vincenzo Palazzo
0ed7c0d083 Suggested code cleanup by TODO comment
This commit introduces the code cleanup suggested by the TODO comment in the code.

Basically, it moves the code from the if-else statement to a switch statement without the default case. I used the basic idea of the code used in PR #4507.

Changelog-Changed: None.

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2021-05-31 11:52:07 +09:30
Rusty Russell
efe29c9db8 sendpsbt, fundpsbt, utxopsbt, reserveinputs, unreserveinputs: allow custom number of blocks to reserve.
Not an API break: reserve=true|false still works for fundpsbt and utxopsbt,
but we also allow a raw number in there.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-26 15:08:01 +09:30
Antoine Poinsot
fe8074c8c3 Refuse to parse v2 onion addresses without deprecated_apis
Tor v2 hidden services have been deprecated for a while:
https://blog.torproject.org/v2-deprecation-timeline .

This prevents user from being able to set them in the configuration
and to connect to them while still letting us be able to parse them
for gossip.

Changelog-Deprecated: lightningd: v2 Tor addresses.  Use v3.  See https://blog.torproject.org/v2-deprecation-timeline.

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-05-24 20:22:45 +09:30
niftynei
d04c373283 rbf: when a channel is open, remove all the inflights
The channel's open has been mined, we don't need to keep all of these around
now.
2021-05-24 12:17:48 +09:30
niftynei
4247ec3a05 inflights: save the whole psbt to the database
Otherwise we're missing info when we go to broadcast these and can't
properly sign the transaction to close it.

Found-by: @jasan
2021-05-24 12:17:48 +09:30
niftynei
82fa3fa2ef df-rbf: order inflights by funding_feerate
When we re-populate from disk, we need to know what order to recreate the
inflights list in.

Fixes #4511
2021-05-24 12:17:48 +09:30
Rusty Russell
9dbac21d3b doc: remove suffix for included-in-master BOLTs.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-04-07 14:34:39 +09:30
Rusty Russell
006300ab96 lightningd: set "direction" correctly for connect which is already connected.
This means remembering the connection direction.  We also use the address to try
to reconnect, which we shouldn't bother with if they connect to us.

For peers from the database, we currently always save the addr: we shouldn't really
do this if they connected to us, since it's not useful for reconnecting (we don't
show the addr in JSON reply to listpeers unless we're connected, so it's only an
internal issue).  This is left for future work.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-03-26 13:22:33 +10:30
niftynei
fba1191f06 df: actually save the peer's sigs transmission to disk
Otherwise we have no idea whether or not they actually sent their sigs
or not.
2021-03-17 10:25:18 +10:30
Rusty Russell
80c2f28373 channeld: accept the 'wrong_funding' shutdown TLV.
If it passes checks, lightningd puts it in the database.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-03-16 13:08:40 +10:30
Rusty Russell
cce280400c wallet: out "wrong_funding" information in the db for persistence.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-03-16 13:08:40 +10:30