Commit Graph

426 Commits

Author SHA1 Message Date
Christian Decker
668debb74f pyln: Rename LightningNode.fund_channel to fundchannel
It is inconsistent with the RPC method, which was bothering me for some time.
2020-09-23 14:45:12 +09:30
Christian Decker
8ecb157af6 pyln: Add type annotations to lightning.py 2020-09-23 14:45:12 +09:30
Christian Decker
49ec800a07 pyln: Add type-annotations to plugin.py
This should help users that have type-checking enabled.
2020-09-23 14:45:12 +09:30
Christian Decker
d27da4d152 pyln: Fix two formatting string not matching parameters 2020-09-23 14:45:12 +09:30
Christian Decker
748caf91d3 pyln-client: Add support to monkey patch the JSONEncoder
Several times we had issues with plugins not being able to re-encode an RPC
result because they forgot to use the custom encoder class. This allows us to
patch the JSONEncoder when we start the RPC or the plugin and automagically
support classes that provide a `to_json` method.
2020-09-23 14:45:12 +09:30
Christian Decker
382230509b pyln: Add stubs to generate documentation for pyln-client 2020-09-23 14:45:12 +09:30
Rusty Russell
0a50301d51 pytest: don't test for memleaks under valgrind.
The next patch perturbed things enough that we suddenly started
getting (with --track-origins=yes):

Valgrind error file: valgrind-errors.120470
==120470== Use of uninitialised value of size 8
==120470==    at 0x14EBD5: htable_val (htable.c:150)
==120470==    by 0x14EC3C: htable_firstval_ (htable.c:165)
==120470==    by 0x14F583: htable_del_ (htable.c:349)
==120470==    by 0x11825D: pointer_referenced (memleak.c:65)
==120470==    by 0x118485: scan_for_pointers (memleak.c:121)
==120470==    by 0x118500: memleak_remove_region (memleak.c:130)
==120470==    by 0x118A30: call_memleak_helpers (memleak.c:257)
==120470==    by 0x118A8B: call_memleak_helpers (memleak.c:262)
==120470==    by 0x118A8B: call_memleak_helpers (memleak.c:262)
==120470==    by 0x118B25: memleak_find_allocations (memleak.c:278)
==120470==    by 0x10EB12: closing_dev_memleak (closingd.c:584)
==120470==    by 0x10F3E2: main (closingd.c:783)
==120470==  Uninitialised value was created by a heap allocation
==120470==    at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==120470==    by 0x1604E8: allocate (tal.c:250)
==120470==    by 0x160AA9: tal_alloc_ (tal.c:428)
==120470==    by 0x119BE0: new_per_peer_state (per_peer_state.c:24)
==120470==    by 0x11A101: fromwire_per_peer_state (per_peer_state.c:95)
==120470==    by 0x10FB7C: fromwire_closingd_init (closingd_wiregen.c:103)
==120470==    by 0x10ED15: main (closingd.c:626)
==120470==

This is because there is uninitialized padding at the end of struct
peer_state.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-23 13:52:49 +09:30
niftynei
d0e42e2c36 contrib: make bitcoin polling and gossip propagation snappier
Faster stronger better local testing
2020-09-17 10:14:22 +09:30
Christian Decker
a88073f6fb repro: Add missing Dockerfiles 2020-09-16 06:27:12 +09:30
Sergi Delgado Segura
f1d5fdf323 pyln: Sets remote_pubkey for LightningConnection if node is not the initiator
Currently ``LightningConnection.remote_pubkey`` is set to ``None`` if the node is not the handshake initiator. This sets it to ``rs`` in act three from the receiver side
2020-09-12 15:06:08 +09:30
Christian Decker
f5dd393113 pyln: Bump proto version to match pyln-bolt*
Since pyln-bolt* specify the 0.8.4 version which we didn't upload, and the
requirements.txt specify ==0.8.4, we need to backfill that version, even if we
could just bump it directly to 0.9.1.
2020-09-10 20:01:41 +09:30
Christian Decker
bd811fbd1a pyln: Add safe fallback results for hooks
Hooks do not tolerate failures at all. If we return a JSON-RPC error to a hook
call the only thing the main daemon can really do is to crash. This commit
adds a mapping of error to a safe fallback result, including a warning to the
node operator that this should be addressed in the plugin. The warning is
reported as a `**BROKEN**` message, and should therefore fail any testing done
on the plugin.

Changelog-Fixed: pyln: Fixed HTLCs hanging indefinitely if the hook function raises an exception. A safe fallback result is now returned instead.
2020-09-10 10:27:24 +09:30
Michael Schmoock
218a9efaec pyln-testing: add get_channel_id helper 2020-09-10 10:24:06 +09:30
ZmnSCPxj jxPCSnmZ
277ff0f44c plugins/spender/multiwithdraw.c: Implement multiwithdraw command.
Fixes: #2679

Changelog-Added: JSON-RPC: New `multiwithdraw` command to batch multiple onchain sends in a single transaction.  Note it shuffles inputs and outputs, does not use BIP69.
2020-09-09 20:36:08 +09:30
Christian Decker
d46d82eeff pyln-testing: Allow up to 1000 connections to postgresql in tests
This was causing some errors when testing with a lot of parallelism (30+ tests
in parallel)
2020-09-09 20:17:26 +09:30
niftynei
b3cbb0b653 tests: plugin for dual-funding tests in ln-prototest
FIXME: requires wallycore
2020-09-09 19:54:20 +09:30
niftynei
171b9857de contrib: add ability to start/stop elements nodeset
sometimes it's handy to be able to get access to an elements daemon and
node set also
2020-09-09 16:49:25 +09:30
ZmnSCPxj jxPCSnmZ
45e1b3828a plugins/multifundchannel.c: Add minchannels flag to multifundchannel.
[ changed from best_effort binary to minchannels counter -- RR]
2020-09-09 16:45:56 +09:30
ZmnSCPxj jxPCSnmZ
9460838650 doc/, contrib/: Additional support etc. for multifundchannel. 2020-09-09 16:45:56 +09:30
Christian Decker
71e713c722 pyln-testing: Bump version to 0.9.0 and add Makefile for release 2020-08-30 20:03:42 +02:00
Christian Decker
0b50fd0f96 pyln-testing: Add a dummy test to check functionality 2020-08-30 20:03:42 +02:00
Christian Decker
81d8525d86 pyln: Make the pytest requirement less stringent 2020-08-30 20:03:42 +02:00
niftynei
26f116041c contrib: start_ln now creates variable number of nodes
Re-write start_ln such that we can create up to 10 nodes locally for
testing. Useful for scenarios where more than two nodes are needed

Changelog-Changed: contrib: startup_regtest.sh `startup_ln` now takes a number of nodes to create as a parameter
2020-08-28 14:27:56 +09:30
niftynei
c8579b99d0 pyln-proto: use vals for subtype parsing 2020-08-27 10:20:16 +09:30
Christian Decker
437c411769 pyln-testing: Have LightningNode.fund_channel poll 2020-08-27 09:45:35 +09:30
Christian Decker
0431a29a26 pyln-testing: Poll for channel active instead of watching logs 2020-08-27 09:45:35 +09:30
ZmnSCPxj jxPCSnmZ
8c763bb7be contrib/pyln-testing/pyln/testing/utils.py: Add a fundbalancedchannel method to LightningNode, for when you want to Thanos your channels. 2020-08-26 09:29:46 +09:30
Sergi Delgado Segura
3edbacbb3c pyln: hexlify -> bytes.hex() in pyln.proto.wire 2020-08-26 06:09:53 +09:30
Sergi Delgado Segura
27e495efa5 pyln: Updates proto to use coincurve for Public and Private keys 2020-08-26 06:09:53 +09:30
nicolas.dorier
2e51f23a95 Add python to runtime docker image (arm32/64) 2020-08-24 20:23:25 +02:00
Rusty Russell
7435d50970 signpsbt: add signonly parameter to restrict/enforce what inputs to sign.
This is an extra safety check for dual funding, where we only want to sign
the inputs we provided!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `signpsbt` takes an optional `signonly` array to limit what inputs to sign.
2020-08-18 11:38:25 -05:00
Rusty Russell
aab3808668 utxopsbt: let caller specify locktime, add tests and python binding.
Changelog-Added: JSON-RPC: `utxopsbt` takes a new `locktime` parameter
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-18 11:38:25 -05:00
Rusty Russell
14baaaa8ba fundpsbt: let caller specify locktime.
Required for dual funding where the opener sets it.

Changelog-Added: JSON-RPC: `fundpsbt` takes a new `locktime` parameter
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-18 11:38:25 -05:00
Rusty Russell
624df64330 pyln-client: allow commands and options to mark themselves deprecated.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: pyln-client: commands and options can now mark themselves deprecated.
2020-08-11 08:43:18 +09:30
Rusty Russell
4ce8f56327 pyln.client.plugin: set deprcated_apis based on getmanifest.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-11 08:43:18 +09:30
Rusty Russell
151bc47583 JSON-RPC: getmanifest passes allow-deprecated-apis flag.
This allows plugins to choose how to present things in getmanifest.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: plugins: `getmanifest` may now include "allow-deprecated-apis" boolean flag.
Changelog-Deprecated: plugins: `getmanifest` without any parameters; plugins should accept any parameters for future use.
2020-08-11 08:43:18 +09:30
Rusty Russell
01a82d38f7 pytest: add slow_test marker.
And when it's set, and we're SLOW_MACHINE, simply disable valgrind.

Since Travis (SLOW_MACHINE=1) only does VALGRIND=1 DEVELOPER=1 tests,
and VALGRIND=0 DEVELOPER=0 tests, it was missing tests which needed
DEVELOPER and !VALGRIND.

Instead, this demotes them to non-valgrind tests for SLOW_MACHINEs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-07 13:57:31 +02:00
Rusty Russell
51aae9cce7 pytest: make valgrind a per-node option.
Next patch will turn it off for slow-marked tests.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-07 13:57:31 +02:00
Rusty Russell
79278b880f pytest: optimize join_nodes a little.
We can query all the txids at once, rather than one at a time.

Doesn't make any measurable difference to full runtime testing here
though.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-07 13:57:31 +02:00
Rusty Russell
929fd3e2f6 pytest: make sure all nodes see funds using sync_blockheight
We might have funds prior to calling join_nodes(), so testing that
we've all seen the block is better.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-07 13:57:31 +02:00
Rusty Russell
02b413a4dc pytest: make join_nodes / line_graph wait for updates in both dirs.
This is what fund_channel() does, which is more thorough than what
we were doing.  But since the order of the logs is undefined, we need
to be a little careful.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-07 13:57:31 +02:00
Rusty Russell
8f455c8b98 pytest: new join_nodes to allow you to get all the nodes then join some of them.
This lets you get_nodes() and join some later.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-07 13:57:31 +02:00
Rusty Russell
06372e13d8 pyln.proto.message: don't let Message() init set implicit lengths.
We'll override them from field length anyway!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-06 17:53:25 +02:00
Antoine Poinsot
5b45334d93 pyln: allow to signal may_fail in get_node()
This or `expect_fail`, as it implies that, well, the node may fail.

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-08-03 12:46:37 +09:30
Rusty Russell
1274d34822 lightningd: add --dev-no-version-checks, use if SLOW_MACHINE and VALGRIND
Reduces VALGRIND=1 node_factory.line_graph(5) time on my laptop from 42s to 36s.

This is simply because forking all the subdaemons just to check the
version is very expensive under valgrind.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-22 16:13:44 +02:00
Rusty Russell
c85a433d9a pytest: reduce accuracy of valgrind if SLOW_MACHINE.
Reduces node_factory.line_graph(5) time on my laptop from 48s to 42s.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-22 16:13:44 +02:00
Rusty Russell
73a5f5b313 fundpsbt: make parameters more usable.
fundpsbt forces the caller to manually add their weight * feerate
to the satoshis they ask for.  That means no named feerates.

Instead, create a startweight parameter and do the calc for them
internally, and return the feerate we used (and, while we're at it,
the estimated final weight).

This API change is best done now, as it would otherwise have to
be appended as a parameter.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-17 13:53:46 +02:00
Rusty Russell
31d7e013bd pytest: test fundpsbt.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-15 18:49:02 +09:30
Rusty Russell
4ee527a59c pytest: test reserve and unreserve.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-15 18:49:02 +09:30
Christian Decker
be9b55fcb5 pyln: Add a traceback if a hook or rpc method fails 2020-07-02 11:54:12 +02:00