Commit Graph

8851 Commits

Author SHA1 Message Date
Christian Decker
1d9e7cf079 db: Add support for key-value pair DSNs in postgresql
These are simple space-separated key-value pair sets of options instead of the
URI style DSNs, but they are also much more flexible allowing the user to
specify client SSL certificates, server certificates, compression and
encryption levels, and much more (see [1] for more information)

[1]: https://www.postgresql.org/docs/9.1/libpq-connect.html

Changelog-Added: db: Added support for key-value DSNs for postgresql, allowing for a wider variety of configurations and environments.
2020-09-23 20:20:40 +09:30
Antoine Poinsot
15adcc915f Remove varint typedef for bigsize
It's not part of the spec anymore

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-09-23 16:30:36 +09:30
Rusty Russell
7db8680530 lightningd: don't hand around partially-assembled channel_info.
wallet_commit_channel would fill in the old_remote_per_commit and
fee_states, which is weird since the caller doesn't care.

Make the caller set all the channel_info fields, so wallet_commit_channel
is a simple consumer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-23 15:51:13 +09:30
Rusty Russell
58f6e316cc lightningd: separate fee_states out of struct channel_info.
It was always kind of weird in there anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-23 15:51:13 +09:30
niftynei
cf2a807d3c dual_open: don't release uc here, it gets freed elsewhere 2020-09-23 15:51:13 +09:30
niftynei
db2b013990 dual-fund: call daemon_shutdown only
daemon_shutdown calls common_shutdown internally; calling both frees the
same context twice
2020-09-23 15:51:13 +09:30
niftynei
56b3711bbc dual_open: alignment nit 2020-09-23 15:51:13 +09:30
niftynei
ed9f661e07 dual_open: memleak problem with fee_states 2020-09-23 15:51:13 +09:30
niftynei
7d836a5945 dual_open_control: nit, reword comment 2020-09-23 15:51:13 +09:30
niftynei
aa6420cb58 dual_open: split logic for deserializing a changed vs signed PSBT
We need to run different checks on these, so split them out for finer
grained control
2020-09-23 15:51:13 +09:30
niftynei
1f165c00ae psbt_txid: it's possible a psbt may already have the finalized scriptsig
If we've already got a scriptSig field filled out for a PSBT input, we
use that instead of 'deriving' the scriptSig from the redeemscript
(finalizing a PSBT removes the redeemscript field)
2020-09-23 15:51:13 +09:30
niftynei
c5f748e716 dualopend: tiny billboard capitalization change 2020-09-23 15:51:13 +09:30
niftynei
5f6a2c05bd dualopend: check that we're not exceeding max allowable capacity 2020-09-23 15:51:13 +09:30
niftynei
8761468f42 dualopend: use status failed instead of peer failed 2020-09-23 15:51:13 +09:30
Christian Decker
668debb74f pyln: Rename LightningNode.fund_channel to fundchannel
It is inconsistent with the RPC method, which was bothering me for some time.
2020-09-23 14:45:12 +09:30
Christian Decker
8ecb157af6 pyln: Add type annotations to lightning.py 2020-09-23 14:45:12 +09:30
Christian Decker
49ec800a07 pyln: Add type-annotations to plugin.py
This should help users that have type-checking enabled.
2020-09-23 14:45:12 +09:30
Christian Decker
d27da4d152 pyln: Fix two formatting string not matching parameters 2020-09-23 14:45:12 +09:30
Christian Decker
748caf91d3 pyln-client: Add support to monkey patch the JSONEncoder
Several times we had issues with plugins not being able to re-encode an RPC
result because they forgot to use the custom encoder class. This allows us to
patch the JSONEncoder when we start the RPC or the plugin and automagically
support classes that provide a `to_json` method.
2020-09-23 14:45:12 +09:30
Christian Decker
382230509b pyln: Add stubs to generate documentation for pyln-client 2020-09-23 14:45:12 +09:30
Rusty Russell
75ce380e0e Travis: disable postgres test.
I don't have time to diagnose why it's failing all the time right now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-23 14:03:03 +09:30
Christian Decker
1cb527d46a topo: Do not keep txids in memory indefinitely
I mistakenly assumed the block would be freed after processing completed. That
is not true since chaintopology keeps headers and stubs around for reorgs. So
we need to remove the precomputed txids along with the full_txs.
2020-09-23 13:52:49 +09:30
Rusty Russell
f37f2b6193 common/memleak: simplify and document API.
1. Rename memleak_enter_allocations to memleak_find_allocations.
2. Unify scanning for pointers into memleak_remove_region / memleak_remove_pointer.
3. Document the functions.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-23 13:52:49 +09:30
Rusty Russell
0a50301d51 pytest: don't test for memleaks under valgrind.
The next patch perturbed things enough that we suddenly started
getting (with --track-origins=yes):

Valgrind error file: valgrind-errors.120470
==120470== Use of uninitialised value of size 8
==120470==    at 0x14EBD5: htable_val (htable.c:150)
==120470==    by 0x14EC3C: htable_firstval_ (htable.c:165)
==120470==    by 0x14F583: htable_del_ (htable.c:349)
==120470==    by 0x11825D: pointer_referenced (memleak.c:65)
==120470==    by 0x118485: scan_for_pointers (memleak.c:121)
==120470==    by 0x118500: memleak_remove_region (memleak.c:130)
==120470==    by 0x118A30: call_memleak_helpers (memleak.c:257)
==120470==    by 0x118A8B: call_memleak_helpers (memleak.c:262)
==120470==    by 0x118A8B: call_memleak_helpers (memleak.c:262)
==120470==    by 0x118B25: memleak_find_allocations (memleak.c:278)
==120470==    by 0x10EB12: closing_dev_memleak (closingd.c:584)
==120470==    by 0x10F3E2: main (closingd.c:783)
==120470==  Uninitialised value was created by a heap allocation
==120470==    at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==120470==    by 0x1604E8: allocate (tal.c:250)
==120470==    by 0x160AA9: tal_alloc_ (tal.c:428)
==120470==    by 0x119BE0: new_per_peer_state (per_peer_state.c:24)
==120470==    by 0x11A101: fromwire_per_peer_state (per_peer_state.c:95)
==120470==    by 0x10FB7C: fromwire_closingd_init (closingd_wiregen.c:103)
==120470==    by 0x10ED15: main (closingd.c:626)
==120470==

This is because there is uninitialized padding at the end of struct
peer_state.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-23 13:52:49 +09:30
Rusty Russell
bba4fd0d70 common: don't suppress leak detection for libwally allocations.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-23 13:52:49 +09:30
Rusty Russell
161cc3e840 memleak: make "_notleak" names less powerful.
They previously prevented any child from being detected as leaks, now
they just mark the tal allocation itself as not being a leak.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-23 13:52:49 +09:30
Rusty Russell
7b55d6cfe2 common: enforce that we have collected all wally allocations.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-23 13:52:49 +09:30
Rusty Russell
c23001d508 bitcoin: use tal_gather_wally() so we don't leave unattached allocations.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-23 13:52:49 +09:30
Rusty Russell
a316c517c7 common: add tal_gather_wally() function to reparent libwally objs.
This lets us reduce leaks, and ease their detection.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-23 13:52:49 +09:30
Rusty Russell
3c8049f32c bitcoin/psbt: psbt_input_add_unknown/psbt_output_add_unknown needs a tal ctx.
Since it allocates something, it needs a context (used in the next patch!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-23 13:52:49 +09:30
Rusty Russell
77b62d9e42 bitcoin/psbt: psbt_finalize needs a tal ctx.
Since it returns a wally_tx.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-23 13:52:49 +09:30
Rusty Russell
607075a3d4 bitcoin/psbt: wallt_tx_output needs a tal ctx.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-23 13:52:49 +09:30
Rusty Russell
480f671e91 bitcoin/psbt: psbt_txid needs a tal ctx.
It returns a wally_tx; it's an anti-pattern not to hand in a tal context.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-23 13:52:49 +09:30
Rusty Russell
67fbb349b6 bitcoin/tx: trivial cleanups.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-23 13:52:49 +09:30
Rusty Russell
11ebb84e7d bitcoin/psbt: attach destructors to wally allocations to avoid leaks.
This covers the obvious ones, but the later patches fix this more
seriously.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Some memory leaks in transaction and PSBT manipulate closed.
2020-09-23 13:52:49 +09:30
Rusty Russell
80c705d528 bitcoin/psbt: more const pointers.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-23 13:52:49 +09:30
Antoine Poinsot
4b4fe2f4fb dual-openingd: msg_type should be dualopend_wire not u8
clang did the hard work here:

openingd/dualopend.c:958:42: error: result of comparison of constant 'WIRE_DUAL_OPEN_FAIL' (7003) with expression of type 'u8' (aka 'unsigned char') is always false
      [-Werror,-Wtautological-constant-out-of-range-compare]
        if ((msg_type = fromwire_peektype(msg)) == WIRE_DUAL_OPEN_FAIL) {
            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^  ~~~~~~~~~~~~~~~~~~~

Co-Authored-by: the clang compiler
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-09-23 13:46:02 +09:30
Christian Decker
de7b41695f db: Fix size mismatch on postgres in a migration
Changelog-Fixed: db: Fixed a broken migration on postgres DBs that had really old channels.
2020-09-23 10:48:31 +09:30
Rusty Russell
a08e83522b configure: clean up tmpfile if we fail.
Reported-by: @whitslack
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-21 13:07:01 +02:00
Rusty Russell
d56f8b6c5d configure: refuse to create config.vars without any database.
We alter the script to do the move to config.vars right at the end,
which is probably a good idea anyway.

Fixes: #4054
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-19 12:38:54 +09:30
Rusty Russell
34a6294031 lightningd: really remove relative plugins dirs.
We promised this in 0.8.1!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: Plugin: Relative plugin paths are not relative to startup (deprecated v0.7.2.1)
2020-09-18 12:08:07 +09:30
Rusty Russell
288aa397fc lightningd: remove deprecated result for plugin stop.
Changelog-Removed: JSON API: `plugin stop` result with an empty ("") key (deprecated 0.8.1)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-18 12:08:07 +09:30
Rusty Russell
00115ee7f0 lightningd: remove deprecated rpc_command hook return.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: JSON API: The hook `rpc_command` returning `{"continue": true}` (deprecated 0.8.1)
2020-09-18 12:08:07 +09:30
Rusty Russell
8aa660e938 lightningd: remove deprecated return for db_write hook.
Changelog-Removed: JSON API: The hook `db_write` can no longer return `true` (deprecated in 0.8.1)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-18 12:08:07 +09:30
Rusty Russell
349f245f8c lightningd: remove deprecated per_hop_v0 in htlc hook.
Changelog-Removed: JSON API: `htlc_accepted` hook `per_hop_v0` object removed (deprecated 0.8.0)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-18 12:08:07 +09:30
Rusty Russell
99ff474308 lightningd: remove deprecated plugin entries.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: JSON API: `listconfigs` duplicated "plugin" paths (deprecated 0.8.0)
2020-09-18 12:08:07 +09:30
Matt Whitlock
eab14768a8 update SHA256STAMPs using sorted dependencies 2020-09-17 10:23:40 +09:30
Matt Whitlock
ec29f04b5f Makefile: sort input filenames to SHA256STAMP
Make's 'wildcard' function returns matching file paths in an
unpredictable order, thereby causing false mismatches in the produced
SHA256STAMPs from system to system. This commit sorts the file paths
given to 'cat' to make the stamps deterministic.

Changelog-Fixed: The build system no longer spuriously regenerates generated sources due to differences in `readdir`(3) sort order.
2020-09-17 10:23:40 +09:30
niftynei
b4989b9e15 listpeers: show correct amounts for starting funding allocation
With dual-funded channels, it's possible for the funding allocation of
the non-opener to not be zero
2020-09-17 10:15:29 +09:30
niftynei
d0e42e2c36 contrib: make bitcoin polling and gossip propagation snappier
Faster stronger better local testing
2020-09-17 10:14:22 +09:30