Commit Graph

125 Commits

Author SHA1 Message Date
Rusty Russell
03108e6501 db: fall back to deprecated sqlite3_trace() for db hook on old sqlite3.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-04-08 01:56:17 +00:00
Rusty Russell
7ccf3af51d pytest: test the db hook plugin.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-04-08 01:56:17 +00:00
Rusty Russell
f0200027b8 db: keep track of changes.
For the moment we just dump them to stderr, but later this will go to a hook.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-04-08 01:56:17 +00:00
Rusty Russell
e1e26ca69d db: create explicit separate API for select statements.
I was tempted to create a new db_select_stmt wrapper type, but that means
a lot of boilerplate around binding, which expects to work with db_prepare
*and* db_select_prepare.

This lets us clearly differentiate between db queries (which don't need to
go to a plugin) and db changes (which do).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-04-08 01:56:17 +00:00
Rusty Russell
8c4ef7f059 db: avoid the only failing db_exec.
Replace with an explicit test for existence.  This simplifies our db interface.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-04-08 01:56:17 +00:00
Michael Schmoock
1043df28be adds: new db fields and struct variables
- Intrduce DB update `channel` values: `feerate_base` and `feerate_ppm`
- Make fist use of now context realted DB migration
- Add `struct channel` members of the same name
- Use struct values instead of config when commiting new channels
2019-03-15 02:48:18 +00:00
Rusty Russell
60285187fd db: support more powerful migrations.
Allow a function as well as (or instead of!) an sql statement.  That
will let us do things like set per-channel values to the global
defaults, for example.

Since we remove the NULL termination, the final entry is ARRAY_SIZE()-1
not ARRAY_SIZE()-2.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-03-04 00:04:38 +00:00
Rusty Russell
bef7d45f1c db: remove manual db_migration_count function.
More efficient to measure the ARRAY_SIZE(), which is a runtime
constant.  We move it into the unit test.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-03-04 00:04:38 +00:00
Rusty Russell
b65c279557 db: don't enter into db_upgrades table on every startup.
Below this code appears:

	if (current != orig)
		db_exec(__func__, db,
			"INSERT INTO db_upgrades VALUES (%i, '%s');",
			orig, version());

But since the loop pre-increments current, this is always true.  I wondered
why there were so many duplicates in my db_upgrades table!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-03-04 00:04:38 +00:00
Rusty Russell
919f390a93 sendpay: rename 'description' to 'label'.
This field was used by `pay` to hold the bolt11 description if the bolt11
string used `h` to hash the description (which nobody ever did).  If the
`h` field wasn't present, it could contain anything, as it wasn't checked.

It's really useful to have a label for payments (eg. '1 Cuban'), but adding
yet-another option would be painful, so we simply rename 'description'
to 'label' except inside the db.

This means we need to do some tricky parameter parsing to handle array
and keyword JSON arguments, but only until we remove the old name.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-23 05:45:25 +00:00
Rusty Russell
5e14274f41 sendpay: provide 'bolt11' parameter.
Without this, there's no proof of payment, since it is the signed invoice
that make the receipt valid.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-23 05:45:25 +00:00
Christian Decker
df4e0a0022 wallet: Add scriptPubkey to struct utxo
In order to avoid having to ask the HSM for public keys to
their_unilateral/to_us outputs we just store the `scriptPubkey` with the UTXO,
which can then be converted to the P2WPKH address.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-02-22 11:15:24 -08:00
Rusty Russell
6ed54dd34e db: fix dangling peers on db upgrade.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 12:03:54 +01:00
Rusty Russell
38e7d19dd5 Makefile: check for direct amount_sat/amount_msat access.
We need to do it in various places, but we shouldn't do it lightly:
the primitives are there to help us get overflow handling correct.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 08:01:37 +00:00
Rusty Russell
1d1fcc41b8 db: add amount functions.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 00:44:57 +00:00
Christian Decker
a405c22fc9 daemon: Fix issue 2348, db->filenam not being correctly initialized
We were not correctly allocating the `db->filename`, failing to copy the
null-terminator. This was causing and error when reopening the database after
the call to `fork()`.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
Reported-by: Sean McNally <@sfmcnally>
Changelog-fixed: Fixed a crash when running in daemon-mode due to db filename overrun
2019-02-18 12:01:06 +01:00
Rusty Russell
b99293fbb6 short_channel_id: don't accept :-separated in JSON if --allow-deprecated-apis=false
We need to still accept it when parsing the database, but this flag
should allow upgrade testing for devs building on top

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-08 16:52:30 -08:00
Rusty Russell
be64dd84ca waitsendpay: indicate which channel direction the error was.
You can figure this yourself by knowing the route, but it's better to report
it directly here.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-15 12:01:38 +01:00
Rusty Russell
26dda57cc0 utils: make tal_arr_expand safer.
Christian and I both unwittingly used it in form:

	*tal_arr_expand(&x) = tal(x, ...)

Since '=' isn't a sequence point, the compiler can (and does!) cache
the value of x, handing it to tal *after* tal_arr_expand() moves it
due to tal_resize().

The new version is somewhat less convenient to use, but doesn't have
this problem, since the assignment is always evaluated after the
resize.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-15 12:01:38 +01:00
Rusty Russell
31a375af53 lightningd: add runtime checking for all system-provided libs.
And I tested this by rolling my own libz; make indeed detects
the change and fixes it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-12-14 05:36:34 +00:00
Rusty Russell
d7e233e47d Move json and param core functionality into common, for plugins.
json_escaped.[ch], param.[ch] and jsonrpc_errors.h move from lightningd/
to common/.  Tests moved too.

We add a new 'common/json_tok.[ch]' for the common parameter parsing
routines which a plugin might want, taking them out of
lightningd/json.c (which now only contains the lightningd-specific
ones).

The rest is mainly fixing up includes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-12-10 00:00:50 +00:00
Rusty Russell
305795b01e common/json: move JSON creation routines into lightningd/
It's the only user of them, and it's going to get optimized.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

gossip.pydiff --git a/common/test/run-json.c b/common/test/run-json.c
index 956fdda35..db52d6b01 100644
2018-10-19 22:02:11 +00:00
Christian Decker
07ebc525e9 wallet: Add new forwards table to keep track of forwarded payments
Signed-off-by: Christian Decker <@cdecker>
2018-10-19 21:58:27 +00:00
Rusty Russell
96f05549b2 common/utils.h: add tal_arr_expand helper.
We do this a lot, and had boutique helpers in various places.  So add
a more generic one; for convenience it returns a pointer to the new
end element.

I prefer the name tal_arr_expand to tal_arr_append, since it's up to
the caller to populate the new array entry.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-27 22:57:19 +02:00
Rusty Russell
cefb6925b2 db: save and restore last_sent_commit correctly.
It's an array: we were only saving the single element; if there was more than
one changed HTLC we'd get a bad signature!

The report in #1907 is probably caused by the other side re-requesting
something we considered already finalized; to avoid this particular error,
we should set the field to NULL if there's no last_sent_commit.

I'm increasingly of the opinion we want to just save all the update
packets to the db and blast them out, instead of doing this
second-guessing dance.

Fixes: #1907
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-04 14:43:27 +02:00
Rusty Russell
a5ecc95c42 db: store claimed per_commitment_point from option_data_loss_protect.
This means we don't try to unilaterally close after a restart, *and*
we can tell onchaind to try to use the point to recover funds when the
peer unilaterally closes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-23 14:46:22 +02:00
Rusty Russell
1dde233a48 wallet: put explicit hook in for fatal error testing.
We're currently overriding fatal() with something that actually
returns, which contrasts with its declaration as NORETURN.

This breaks in the next patch which wants a real fatal() in wallet.h.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-10 12:44:28 +02:00
practicalswift
09ec0ae990 Return true or false from sqlite3_bind_* based on result from SQLite instead of always returning true 2018-07-31 12:44:44 +02:00
Rusty Russell
337075dc8c tal: don't access low-level tal functions.
In several places we use low-level tal functions because we want the
label to be something other than the default.  ccan/tal is adding
tal_*_label so replace them and shim it for now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-30 11:31:17 +02:00
Rusty Russell
5cf34d6618 Remove tal_len, use tal_count() or tal_bytelen().
tal_count() is used where there's a type, even if it's char or u8, and
tal_bytelen() is going to replace tal_len() for clarity: it's only needed
where a pointer is void.

We shim tal_bytelen() for now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-30 11:31:17 +02:00
Christian Decker
7119a02932 db: Add description fields to invoices and payments 2018-07-30 03:04:45 +00:00
Rusty Russell
ae17c64c4a db: full location tags for callers, make it implicit.
For better leak tracking.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-27 16:20:35 +02:00
Rusty Russell
b036948219 db: track open sqlite3_stmt in DEVELOPER mode.
I would have liked to make it a tal object, then we'd catch most
things with our memleak detection.  However, sqlite3 doesn't seem to
allow allocator overrides.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-27 16:20:35 +02:00
Christian Decker
89ff46f1e6 db: Added DB migrations to get the correct sync height
The no-rescan change requires us to rescan one last time from the first_blocknum
of our channels (if we have any). The migrations just drop blocks that are
higher, then insert a dummy with the first_blocknum, and then clean up after
us. If we don't have any channels we don't go back at all.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-25 14:33:38 +02:00
Christian Decker
28feb2eb7d db: Add table for onchaind transactions
These transactions being seen on the blockchain triggered some action in
onchaind so we need to replay them when we restore the onchaind.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-25 14:33:38 +02:00
ZmnSCPxj, ZmnSCPxj jxPCSmnZ
11ca729d85 wallet, payalgo: Save detail of payment failures for later reporting. (#1345)
Pointless for remote failures as those are never sent by
the erring node, but for local failures we can give more
detail.
2018-04-16 15:29:40 +02:00
Christian Decker
0f89653ce8 db: Add table for transactions we are interested in
Currently these are either transactions we sent ourselves or transactions that
we are watching because they are part of a channel.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-13 00:04:37 +02:00
Christian Decker
ffbe56847e db: Fixing some clang warnings
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-09 00:21:20 +00:00
ZmnSCPxj
f83c4ff903 wallet: Add msatoshi_to_us_min and msatoshi_to_us_max statistics for channels.
So we know how much counterparty could theoretically steal from us
 (msatoshi_to_us - msatoshi_to_us_min) and how much we could
 theoretically steal from counterparty (msatoshi_to_us_max -
 msatoshi_to_us).
For more piloting goodness.
2018-04-05 19:01:53 +02:00
Rusty Russell
6bb47276ce lightningd: put min/max feerates into db, struct channel.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-04-04 02:31:41 +00:00
Christian Decker
1133820ad6 db: Add a cleanup migration to clear any dangling utxoset entries
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-02 21:07:13 +02:00
Christian Decker
eb531e6da7 db: Make sure to enable foreign keys even after forking
This may be causing #1280, since with `--daemon` the DB is being reopened
without enabling the foreign key relations and hence the delete cascades.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-04-02 21:07:13 +02:00
ZmnSCPxj
b914062465 pay: Also record how much we actually ended up sending. 2018-03-26 05:01:49 +00:00
practicalswift
7e9750ffee Reduce variable scopes 2018-03-26 01:31:21 +00:00
ZmnSCPxj
0bb9bcc0f1 wallet: Track some channel usage statistics.
Fixes: #1049
2018-03-26 01:08:52 +00:00
Rusty Russell
76e8a11380 wallet: use json_escaped for invoice label.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-26 00:20:53 +00:00
practicalswift
98f49c0837 Remove include in file foo.c that is already included in foo.h 2018-03-25 23:54:21 +00:00
practicalswift
0bf1b01425 Fix typos 2018-03-25 15:53:01 +02:00
ZmnSCPxj
d5d31864cf wallet: Remove onion-decoding information from db on payment success/fail.
Fixes: #1177
2018-03-21 13:56:18 +01:00
ZmnSCPxj
693c718160 wallet: Add DB fields for recording routing failures. 2018-03-14 05:33:09 +00:00