Commit Graph

6139 Commits

Author SHA1 Message Date
trueptolemy
a449a91ae2 JSON: Warp the process of forward payment json object
Warp this process as a new function: 'void json_format_forwarding_object()'. This function will be used in 'forward_event' next, and can ensure the consistent json object structure for forward_payment between 'listforwards' API and 'forward_event' notification.
2019-08-01 18:49:25 +08:00
trueptolemy
bcec6bb6cc API: 'listforwards' now include 'payment_hash' field
'payment_hash' can help users learn more about the forward payment.
2019-08-01 18:49:25 +08:00
trueptolemy
ab60bcab7c wallet: Set the fee to 0 when forward doesn't have out channel with LOCAL_FAILED status 2019-08-01 18:49:25 +08:00
Rusty Russell
2255dd4dda
lightningd: avoid thundering herd on restart.
The reason lnd was sending sync error was that we were taking more than
30 seconds to send the channel_reestablish after connect.  That's
understandable on my test node under valgrind, but shouldn't happen normally.

However, it seems it has at least once,
(see https://github.com/ElementsProject/lightning/issues/2847)
: space out startup so it's less likely to happen.

Suggested-by: @cfromknecht
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-01 01:10:11 +00:00
Rusty Russell
cc70b9c4ec wire: use common/bigsize routines
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-31 23:25:59 +00:00
Rusty Russell
2241e25cb4 test: move run-bigsize into common to match common/bigsize.
And make it use bigsize accessors directly, not via fromwire/towire.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-31 23:25:59 +00:00
Rusty Russell
8349c1a0c2 bigsize: insist in minimal representation in bigsize_get().
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-31 23:25:59 +00:00
Rusty Russell
3fa375881a bigsize: make it a proper first-class type.
It doesn't belong in bitcoin, and should not be confused with varint_t.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-31 23:25:59 +00:00
Rusty Russell
e23f183468 common/sphinx: use bigsize_get not varint_get.
These are not the same if it's more than one byte.  Testing would have
caught this, I assume.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-31 23:25:59 +00:00
Simon Vrouwe
1ae8b73f0e doc: invoice manpage: add some details about route hint selection
Also add more detail to the warnings returned by the invoice command.
2019-07-31 23:25:37 +00:00
Christian Decker
5dff67900e tx: Add chainparams when deserializing transactions from wire msgs
This is the other origin, besides `bitcoin_tx`, where we create `bitcoin_tx`
instances, so add the context as soon as possible. Sadly I can't weave the
chainparams into the deserialization code since that'd need to change all the
generated wire code as well.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-07-31 23:22:54 +00:00
Christian Decker
d14bd286ba bitcoin: Add chainparams to transactions from blocks
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-07-31 23:22:54 +00:00
Christian Decker
9288a7906b tx: Add chainparams to struct bitcoin_tx as context
The way we build transactions, serialize them, and compute fees depends on the
chain we are working on, so let's add some context to the transactions.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-07-31 23:22:54 +00:00
Christian Decker
2537bd5afa closingd: Tell closingd which chain we are working on
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-07-31 23:22:54 +00:00
Christian Decker
c5ac6c7b1f hsmd: Tell hsmd which chain we are working on
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-07-31 23:22:54 +00:00
Christian Decker
15e73e3dc3 onchaind: Pass genesis hash to onchaind so it knows the chainparams
It'll be creating quite a few transactions and we will have to know which
params to use.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-07-31 23:22:54 +00:00
Rene Pickhardt
dbc0265a5d added mako dependency to tests/requirements.txt
this should partially fix #2879 

this dependency seems to be needed in `tools/generate-wire.py`
2019-07-31 07:46:49 +08:00
trueptolemy
0ae20399bd CHANGELOG: check the bitcoind version when setup_topology 2019-07-30 17:38:54 +08:00
trueptolemy
130bf20516 lightningd: check bitcoind version when setup_topology 2019-07-30 17:38:54 +08:00
trueptolemy
cdbfa4229c bitcoin: Add the cli_min_supported_version field in chainparams
Set the min supported numeric version of cli as 150000.
2019-07-30 17:38:54 +08:00
Rusty Russell
201b531eb6 devtools: fix credit script.
Lisa's git name is lower-case, whereas CHANGELOG.md uses upper case,
so it doesn't realize she's named a commit already.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-30 17:38:11 +08:00
Chirimen-Jako
b7119150a7 Fix build fail on 32bit environment.
cc -DBINTOPKGLIBEXECDIR="\"../libexec/c-lightning\"" -Wall -Wundef -Wmissing-prototypes -Wmissing-declarations -Wstrict-prototypes -Wold-style-definition -Werror -Wno-error=maybe-uninitialized -std=gnu11 -g -fstack-protector -Og -I ccan -I external/libwally-core/include/ -I external/libwally-core/src/secp256k1/include/ -I external/jsmn/ -I external/libbacktrace/ -I external/libbacktrace-build -I . -I/usr/local/include    -DSHACHAIN_BITS=48 -DJSMN_PARENT_LINKS  -DCOMPAT_V052=1 -DCOMPAT_V060=1 -DCOMPAT_V061=1 -DCOMPAT_V062=1 -DCOMPAT_V070=1 -DBINTOPKGLIBEXECDIR="\"../libexec/c-lightning\""  -c -o common/sphinx.o common/sphinx.c
common/sphinx.c: In function 'sphinx_parse_payload':
common/sphinx.c:488:30: error: passing argument 3 of 'varint_get' from incompatible pointer type [-Werror=incompatible-pointer-types]
   vsize = varint_get(src, 3, &raw_size);
                              ^
In file included from common/sphinx.c:3:0:
./bitcoin/varint.h:16:8: note: expected 'u64 * {aka long long unsigned int *}' but argument is of type 'size_t * {aka unsigned int *}'
 size_t varint_get(const u8 *p, size_t max_len, varint_t *val);
        ^~~~~~~~~~
common/sphinx.c: In function 'process_onionpacket':
common/sphinx.c:621:40: error: passing argument 3 of 'bigsize_get' from incompatible pointer type [-Werror=incompatible-pointer-types]
   vsize = bigsize_get(paddedheader, 3, &shift_size);
                                        ^
In file included from common/sphinx.c:3:0:
./bitcoin/varint.h:23:8: note: expected 'u64 * {aka long long unsigned int *}' but argument is of type 'size_t * {aka unsigned int *}'
 size_t bigsize_get(const u8 *p, size_t max, varint_t *val);
2019-07-30 05:22:38 +00:00
Rusty Russell
488ef67b9b doc: fix typo in default port number.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-30 05:05:45 +00:00
Rusty Russell
af5b3203b5 Makefile: fix parallel update-mocks build.
We always rebuild headerversions to examine critical system headers,
however that stomps on parallel builds with:

make[1]: execvp: tools/headerversions: Text file busy

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-30 05:05:24 +00:00
Rusty Russell
928e3964f2 Makefile: fix update-mocks
Broken by 9e5b9a31c9: it needs $MAKE set.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-30 05:05:24 +00:00
Christian Decker
581694fdda devtools: Minor cleanup of the onion command line tool
Simplifying some operations, erroring in some cases and moving to global
defines for constants.

Suggested-by: Rusty Russell <@rustyrussell>
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-07-30 02:14:49 +00:00
Christian Decker
78c7edb889 sphinx: Switch to big-endian number encoding
See https://github.com/lightningnetwork/lightning-rfc/pull/619 and
https://github.com/lightningnetwork/lightning-rfc/pull/619 for discussion.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-07-30 02:14:49 +00:00
Christian Decker
c752c3318d sphinx: Cleanup sphinx onion construction, remove realm
The realm has lost significance, so let's unify this into the type.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-07-30 02:14:49 +00:00
Christian Decker
660921a9dd sphinx: Introduce a runtest command to the onion tool
The `runtest` command takes a JSON onion spec, creates the onion and decodes
it with the provided private keys. It is fully configurable and can be used
for the test-vectors in the spec.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-07-30 02:14:49 +00:00
Christian Decker
b83d15ea4a sphinx: Remove standalone v0 payload in favor of the unionized one
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-07-30 02:14:49 +00:00
Christian Decker
d607afd195 sphinx: Variable left-shift when unwrapping onion
This is all it takes on the read side to use multiple frames. We are
overshooting the padding a bit since we can at most use 16 additional frames,
but ChaCha20 is cheap.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-07-30 02:14:49 +00:00
Christian Decker
7bc4cf83b1 sphinx: Expose sphinx_add_raw_hop for testing
Shouldn't be used directly, but really useful for testing, since we can just
cram a huge payload in without having to be valid. And we don't have a TLV
spec yet.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-07-30 02:14:49 +00:00
Christian Decker
75c0160c08 sphinx: Simplify the filler generation code
Just some reorganizations and clarifications before we add the multi-frame
support.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-07-30 02:14:49 +00:00
Christian Decker
6831db62f7 sphinx: Clean up after migrating to the sphinx_path struct
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-07-30 02:14:49 +00:00
Christian Decker
ca0dc01bee sphinx: Fix the onion cli tool to take public keys
This was a mismatch between the go tool and this test tool so far. Just
aligning the tools to allows for easier testing.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-07-30 02:14:49 +00:00
Christian Decker
a0a1a1f752 sphinx: Add function to add a new v0 hop to a sphinx_path
This is just taking the existing serialization code and repackaging it in a
more useful form.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-07-30 02:14:49 +00:00
Christian Decker
71cf4e1b39 sphinx: Add sphinx_path struct to encapsulate routing related info
`struct sphinx_path` serves as a container for all the routing related
information, with a couple of constructors that can be used for normal
operation or testing (with pre-determined `session_key`).

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-07-30 02:14:49 +00:00
Christian Decker
ea4831363e sphinx: Consolidate some of the hard-coded parameter naming
For the multi-frame support we need to introduce the FRAME_SIZE parameter and
I took the opportunity to fix up some of the naming.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-07-30 02:14:49 +00:00
Rusty Russell
79d32ec2f2 plugin: notice when plugin has *started* configuring.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-30 08:44:01 +08:00
Rusty Russell
df8a6f615b pytest: test for crash when we do a plugin op and a plugin has not finished init
We try to start it twice, with predictable hilarious results:

   DEBUG plugin-manager started(20701) /home/rusty/devel/cvs/lightning/tests/plugins/slow_init.py
   DEBUG lightningd(20670): Adding block 101: 0f26301caaae390d1b956c3eb73827d1f87a2af306561348045bf1313b7551b0
   INFO plugin-slow_init.py slow_init.py initializing
   DEBUG plugin-manager started(20704) /home/rusty/devel/cvs/lightning/tests/plugins/slow_init.py
   **BROKEN** lightningd(20670): FATAL SIGNAL 6 (version v0.7.1-146-g7d3b352-modded)
   **BROKEN** lightningd(20670): backtrace: common/daemon.c:45 (send_backtrace) 0x55a29c3ec539
   **BROKEN** lightningd(20670): backtrace: common/daemon.c:53 (crashdump) 0x55a29c3ec589
   **BROKEN** lightningd(20670): backtrace: (null):0 ((null)) 0x7fa281ebcf5f
   **BROKEN** lightningd(20670): backtrace: (null):0 ((null)) 0x7fa281ebced7
   **BROKEN** lightningd(20670): backtrace: (null):0 ((null)) 0x7fa281e9e534
   **BROKEN** lightningd(20670): backtrace: (null):0 ((null)) 0x7fa281e9e40e
   **BROKEN** lightningd(20670): backtrace: (null):0 ((null)) 0x7fa281eae011
   **BROKEN** lightningd(20670): backtrace: lightningd/json_stream.c:225 (json_stream_output_) 0x55a29c3bff5f
   **BROKEN** lightningd(20670): backtrace: lightningd/plugin.c:383 (plugin_write_json) 0x55a29c3e103f
   **BROKEN** lightningd(20670): backtrace: ccan/ccan/io/io.c:59 (next_plan) 0x55a29c43316b
   **BROKEN** lightningd(20670): backtrace: ccan/ccan/io/io.c:435 (io_do_always) 0x55a29c433df1
   **BROKEN** lightningd(20670): backtrace: ccan/ccan/io/poll.c:300 (handle_always) 0x55a29c435865
   **BROKEN** lightningd(20670): backtrace: ccan/ccan/io/poll.c:377 (io_loop) 0x55a29c435b3d
   **BROKEN** lightningd(20670): backtrace: lightningd/io_loop_with_timers.c:24 (io_loop_with_timers) 0x55a29c3bdfcf
   **BROKEN** lightningd(20670): backtrace: lightningd/lightningd.c:830 (main) 0x55a29c3c44b6
   **BROKEN** lightningd(20670): backtrace: (null):0 ((null)) 0x7fa281e9fb6a
   **BROKEN** lightningd(20670): backtrace: (null):0 ((null)) 0x55a29c3ac0f9
   **BROKEN** lightningd(20670): backtrace: (null):0 ((null)) 0xffffffffffffffff

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-30 08:44:01 +08:00
Rusty Russell
a477c69d46 lightningd: fix crash on dynamic plugin startup
2019-07-29T04:25:31.503Z DEBUG plugin-manager started(25413) /home/rusty/text/webinar/lightning-2019-07/helloworld.py
2019-07-29T04:25:39.168Z UNUSUAL lightningd(24972): Unable to estimate CONSERVATIVE/2 fee
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): FATAL SIGNAL 6 (version v0.7.1-144-g6bb8525)
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): backtrace: common/daemon.c:45 (send_backtrace) 0x5581b809d7d6
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): backtrace: common/daemon.c:53 (crashdump) 0x5581b809d823
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): backtrace: (null):0 ((null)) 0x7f4ecd46af5f
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): backtrace: (null):0 ((null)) 0x7f4ecd46aed7
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): backtrace: (null):0 ((null)) 0x7f4ecd44c534
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): backtrace: ccan/ccan/tal/tal.c:93 (call_error) 0x5581b80da741
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): backtrace: ccan/ccan/tal/tal.c:165 (check_bounds) 0x5581b80da81d
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): backtrace: ccan/ccan/tal/tal.c:174 (to_tal_hdr) 0x5581b80da845
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): backtrace: ccan/ccan/tal/tal.c:186 (to_tal_hdr_or_null) 0x5581b80da88e
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): backtrace: ccan/ccan/tal/tal.c:632 (tal_bytelen) 0x5581b80db59c
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): backtrace: lightningd/plugin.c:1056 (plugin_subscriptions_contains) 0x5581b8095aca
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): backtrace: lightningd/plugin.c:1071 (plugins_notify) 0x5581b80972f4
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): backtrace: lightningd/notification.c:63 (notify_warning) 0x5581b808615a
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): backtrace: lightningd/log.c:269 (logv) 0x5581b8085393
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): backtrace: lightningd/log.c:329 (log_) 0x5581b8085233
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): backtrace: lightningd/bitcoind.c:372 (process_estimatefee) 0x5581b80759d7
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): backtrace: lightningd/bitcoind.c:226 (bcli_finished) 0x5581b8075cfb
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): backtrace: ccan/ccan/io/poll.c:244 (destroy_conn) 0x5581b80d2413
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): backtrace: ccan/ccan/io/poll.c:250 (destroy_conn_close_fd) 0x5581b80d2435
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): backtrace: ccan/ccan/tal/tal.c:235 (notify) 0x5581b80dabb0
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): backtrace: ccan/ccan/tal/tal.c:397 (del_tree) 0x5581b80dac7a
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): backtrace: ccan/ccan/tal/tal.c:481 (tal_free) 0x5581b80db1c5
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): backtrace: ccan/ccan/io/io.c:450 (io_close) 0x5581b80d1147
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): backtrace: ccan/ccan/io/poll.c:449 (io_loop) 0x5581b80d29ff
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): backtrace: lightningd/io_loop_with_timers.c:24 (io_loop_with_timers) 0x5581b808058d
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): backtrace: lightningd/lightningd.c:830 (main) 0x5581b8084396
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): backtrace: (null):0 ((null)) 0x7f4ecd44db6a
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): backtrace: (null):0 ((null)) 0x5581b8075059
2019-07-29T04:25:39.193Z **BROKEN** lightningd(24972): backtrace: (null):0 ((null)) 0xffffffffffffffff

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-30 08:44:01 +08:00
Rusty Russell
310d806dd3 plugins: document and extend the ~/.lightning/plugins/ dir.
Load any plugins directly as well as subdirs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-29 14:32:07 +02:00
Rusty Russell
4f937ea1f0 fixup! FreeBSD: fix build. 2019-07-29 14:24:29 +02:00
Rusty Russell
9e5b9a31c9 FreeBSD: fix build.
This makes it build for me on FreeBSD 11:
1. $(MAKE) has to passed through into update-mocks.
2. FreeBSD sed doesn't turn \n into a newline on RHS.
3. Bash and mako dependencies were missing from INSTALL.md

Fixes: #2850
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-29 14:24:29 +02:00
Rusty Russell
6bb8525e5d gossipd: fix crash when we prune old, un-updated channel announcements.
We added a random channel to the list, but we can just free it immediately
(since traversal of a uintmap isn't altered by deletion).

This was introduced in d1f43d993a where we explicitly call free_chan
rather than relying on destructors.

Fixes: #2837
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-28 14:15:32 +02:00
darosior
1e2f379e1b Add a CHANGELOG bragline for the plugin command and new fields in
'init' and 'getmanifest'
2019-07-28 07:24:04 +00:00
darosior
dda154612c doc: add the new init and getmanifest fields 2019-07-28 07:24:04 +00:00
darosior
307fb0708e lightningd/plugin_control: don't control non-dynamic plugins 2019-07-28 07:24:04 +00:00
darosior
12e28c2554 lightningd/plugin: Add a 'dynamic' field to getmanifest and a 'startup' field to init
This lets a plugin specify whether it can be restarted, and to know if it is started at lightningd startup
2019-07-28 07:24:04 +00:00
darosior
f81db6fb47 Travis: export PYTHONPATH to add pylightning 2019-07-28 07:24:04 +00:00