Commit Graph

8383 Commits

Author SHA1 Message Date
Rusty Russell
496c0dd1e6 common/random_select: central place for reservoir sampling.
Turns out we can make quite a simple API out of it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-28 10:56:50 +09:30
Rusty Russell
12d0d5c185 amount: cleanup usage.
We've got some recently-added primitives which help.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-27 18:16:28 +02:00
Rusty Russell
4a9c162211 pytest: disable test_mpp_interference_2.
We're failing this too often: we'd fail it more but it's disabled
with VALGRIND (it shouldn't be: @slow_test removes VALGRIND if SLOW_MACHINE
is set).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-27 13:51:22 +02:00
niftynei
c8579b99d0 pyln-proto: use vals for subtype parsing 2020-08-27 10:20:16 +09:30
Christian Decker
21d87f7075 pay: Implement simple presplit fix for ludicrous amounts
This is the simplest possible fix: increase the target amount until we get
the desired number of parts, while still bucketizing payments together that
are in approximately the same size.

The current logic puts all payments that are in the range x < amount <= 16*x
in the same bucket, making them harder to distinguish.

Changelog-Fixed: pay: The `presplit` modifier now supports large payments without exhausting the available HTLCs.
2020-08-27 10:19:21 +09:30
Christian Decker
3f6809c934 pytest: Reproduce ludicrous presplit issue
Huge payment + presplit modifier = too many HTLCs...
2020-08-27 10:19:21 +09:30
Christian Decker
f5985171bd gossipd: Make gossipd much quieter
We are logging way too much from gossipd, causing noisy logs. This PR reduces
logs for incoming messages to those that actually caused a change in our
internal state (duplicate and old messages are just dropped silently now).

Changelog-Changed: gossipd: The `gossipd` is now a lot quieter, and will log only when a message changed our network topology.
2020-08-27 09:45:35 +09:30
Christian Decker
bdad5a6f94 pytest: Stabilize test_onchain_middleman
It wasn't waiting for the tx to be in the mempool
2020-08-27 09:45:35 +09:30
Christian Decker
8f028272f4 pytest: Stabilize test_pay_get_error_with_update 2020-08-27 09:45:35 +09:30
Christian Decker
437c411769 pyln-testing: Have LightningNode.fund_channel poll 2020-08-27 09:45:35 +09:30
Christian Decker
0431a29a26 pyln-testing: Poll for channel active instead of watching logs 2020-08-27 09:45:35 +09:30
Christian Decker
8f05d28049 pytest: Stabilize test_onchaind_replay 2020-08-27 09:45:35 +09:30
ZmnSCPxj jxPCSnmZ
128adf0938 plugins/libplugin-pay.c: Round-robin routehints when splitting.
This improves the success rate of `test_mpp_interference_2`, though
still not quite up to the level that we can remove `@flaky` from it.
2020-08-26 09:29:46 +09:30
ZmnSCPxj jxPCSnmZ
ba3f38077e lightningd/invoice.c: Use round-robin channel selection.
Changelog-Changed: We now make MPP-aware routehints in invoices.
2020-08-26 09:29:46 +09:30
ZmnSCPxj jxPCSnmZ
a9a11265dd lightningd/lightningd.h: Maintain a round-robin list of channels. 2020-08-26 09:29:46 +09:30
ZmnSCPxj jxPCSnmZ
f50951a0d6 tests/test_pay.py: Motivating scenario for round-robin routehinting. 2020-08-26 09:29:46 +09:30
ZmnSCPxj jxPCSnmZ
8c763bb7be contrib/pyln-testing/pyln/testing/utils.py: Add a fundbalancedchannel method to LightningNode, for when you want to Thanos your channels. 2020-08-26 09:29:46 +09:30
Sergi Delgado Segura
3edbacbb3c pyln: hexlify -> bytes.hex() in pyln.proto.wire 2020-08-26 06:09:53 +09:30
Sergi Delgado Segura
27e495efa5 pyln: Updates proto to use coincurve for Public and Private keys 2020-08-26 06:09:53 +09:30
ZmnSCPxj jxPCSnmZ
c27d7a3110 plugins/libplugin-pay.c: Store the route description, and re-report on failure. 2020-08-25 12:17:18 +02:00
ZmnSCPxj jxPCSnmZ
6468616c02 plugins/libplugin-pay.c: Propagate local_id from parent to child payment object. 2020-08-25 12:17:18 +02:00
ZmnSCPxj jxPCSnmZ
d8678467fa plugins/libplugin-pay.c: Show routes being tried, also print updates to channel hints not just initial creations. 2020-08-25 12:17:18 +02:00
ZmnSCPxj jxPCSnmZ
d89c77c0ce plugins/libplugin-pay.c: Describe the bits of unrecognized failure codes. 2020-08-25 12:17:18 +02:00
ZmnSCPxj jxPCSnmZ
0d2d85ab5f plugins/libplugin-pay.c: Also print events that create new sub-payments. 2020-08-25 12:17:18 +02:00
ZmnSCPxj jxPCSnmZ
98583e84b5 plugins/libplugin-pay.c: Give cmd id and partid for each log message.
Changelog-None: internal debugging

Makes it easier to debug payments with tons of splits.
2020-08-25 12:17:18 +02:00
ZmnSCPxj jxPCSnmZ
e80d733399 .gitignore: Add missing gitignores. 2020-08-25 12:10:05 +02:00
Rusty Russell
dc8458d1e2 tools: fix comment in template.
It's (usually) a .csv not an _csv file.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-25 12:53:13 +09:30
Rusty Russell
3e52d4100d common: convert to new wire generation style.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-25 12:53:13 +09:30
Rusty Russell
398b4806b9 connectd: convert to new wire generation style.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-25 12:53:13 +09:30
Rusty Russell
fe8564555f closingd: convert to new wire generation style.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-25 12:53:13 +09:30
Rusty Russell
6d09c0eba9 openingd: convert wire to new scheme.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-25 12:53:13 +09:30
Rusty Russell
dffbf8de85 gossipd: convert wire to new scheme.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-25 12:53:13 +09:30
Rusty Russell
1702c7a69a hsmd: convert to new wire generation style.
Note that other directories were explicitly depending on the generated
file, instead of relying on their (already existing) dependency on 
$(LIGHTNINGD_HSM_CLIENT_OBJS), so we remove that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-25 12:53:13 +09:30
Rusty Russell
8ae6740bb1 Makefile: create generic wiregen rules, use for channeld.
This means some files get renamed, and I took the opportunity to clarify
our naming (the *d* is important!)

1. channeld/channel_wire.csv -> channeld/channeld_wire.csv
2. channeld/gen_channel_wire.h -> channeld/channeld_wiregen.h
3. enum channel_wire_type -> enum channeld_wire
4. WIRE_CHANNEL_FUNDING_DEPTH -> WIRE_CHANNELD_FUNDING_DEPTH.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-25 12:53:13 +09:30
Rusty Russell
7804d89ef1 Makefile: don't output on stderr if file not yet checking into git.
And if both aren't checked in, consider them changed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-25 12:53:13 +09:30
Rusty Russell
b38b913694 Makefile: reduce clutter if they haven't run ./configure
It's hard to see the error:

Before:

    $ make
    File config.vars not found: you must run ./configure before running make.
    CC: cc -DBINTOPKGLIBEXECDIR="../libexec/c-lightning"    -I ccan -I external/libwally-core/include/ -I external/libwally-core/src/secp256k1/include/ -I external/jsmn/ -I external/libbacktrace/ -I external/x86_64-linux-gnu/libbacktrace-build -I external/libsodium/src/libsodium/include -I external/libsodium/src/libsodium/include/sodium -I external/x86_64-linux-gnu/libsodium-build/src/libsodium/include -I . -I/usr/local/include    -DSHACHAIN_BITS=48 -DJSMN_PARENT_LINKS   -DBUILD_ELEMENTS=1 -c -o
    LD: cc  -fsanitize=address     -Lexternal/x86_64-linux-gnu -lwallycore -lsecp256k1 -ljsmn -lbacktrace -lsodium -L/usr/local/lib -lm -lgmp -lsqlite3 -lz  -o

After:

    $ make
    File config.vars not found: you must run ./configure before running make.
    make: *** No rule to make target 'config.vars', needed by 'show-flags'.  Stop.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-25 12:53:13 +09:30
Rusty Russell
b9ac032329 doc: force refresh of all manpages.
The idea is that you regenerate the man pages in the same commit you
alter them: that's how we know whether to try regenerating them or not
(git doesn't store timestamps, so it can't really tell).

Travis will now check this, so force them all to sync to this commit.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-25 12:53:13 +09:30
Rusty Russell
67cde96397 Makefile: respect --quiet and V=1 in doc/Makefile.
And don't regen doc/index.rst unless there's good reason.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-25 12:53:13 +09:30
niftynei
6f114e06a6 bugfix: save max_witness_len as big-endian 2020-08-25 12:34:56 +09:30
niftynei
0305c0a57e psbt: if psbt is null, return null for get_bytes 2020-08-25 12:34:56 +09:30
niftynei
df7c122cb8 json: add "json-to-psbt" helper
Pull a PSBT out of a json buffer
2020-08-25 12:34:56 +09:30
niftynei
10237ccb8e psbt: handle 'NULL' or empty byte PSBTs
Don't crash if the PSBT is null / zero-byte len.
2020-08-25 12:34:56 +09:30
Rusty Russell
ef3fbab551 invoice: don't allow zero-value invoices.
You can't pay them anyway, and at least one person used 0 instead of "any".

Closes: #3808
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: `invoice` no longer accepts zero amounts (did you mean "any"?)
2020-08-25 12:22:45 +09:30
nicolas.dorier
2e51f23a95 Add python to runtime docker image (arm32/64) 2020-08-24 20:23:25 +02:00
Rusty Russell
57dd5be2fd gossipd: prune channels unless *both* peers have refreshed.
See https://github.com/lightningnetwork/lightning-rfc/pull/767

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Protocol: channels now pruned after two weeks unless both peers refresh it (see lightning-rfc#767)
2020-08-24 20:55:47 +09:30
Rusty Russell
7ad8fde060 bolt11: update ctlv expiry, always write it.
As per https://github.com/lightningnetwork/lightning-rfc/pull/785

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: protocol: bolt11 invoices always include CLTV fields (see lightning-rfc#785)
2020-08-24 20:55:47 +09:30
Rusty Russell
b09e519669 options: be more conservative with timeouts on mainnet.
As per https://github.com/lightningnetwork/lightning-rfc/pull/785

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: config: the default CLTV expiry is now 34 blocks, and final expiry 18 blocks as per new BOLT recommendations.
2020-08-24 20:55:47 +09:30
Rusty Russell
6e4e267f2c doc: update BOLT now option_anchor_outputs is merged.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-24 20:55:47 +09:30
Rusty Russell
007daf6b9f doc: update bolt version
And sweep through and remove git qualifiers from many BOLT strings.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-24 20:55:47 +09:30
Rusty Russell
95294e25ea wallet: do valgrind checks when binding db statements.
Otherwise valgrind gets upset when we *run* the statements: better
to get a backtrace when we bind, so we can tell which field it is!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-21 11:25:51 +09:30