Commit Graph

4140 Commits

Author SHA1 Message Date
Christian Decker
356dcbba97 pylightning: Don't attempt to deserialize after every read
We make use of the structure of the final read to decide when to deserialize.
2018-07-16 00:24:04 +00:00
Bertrand Marlier
760b053d7b pylightning: fix incorrect description for disconnect() method 2018-07-15 18:40:48 +02:00
SimonVrouwe
f2ffb6d03e improves exponential smoothing of feerate estimates (#1699)
- fixes problem with polling interval > 150 * 0.9
- fixes log message 'feerate hit floor' at every feerate change
- smoothed fee now reaches 90% of (exp weighted) fee estimates polled in last
120s, independent of polling interval
- only apply smoothing when effect > 10 percent so it doesn't correct forever
- fix indentation
2018-07-15 18:30:43 +02:00
Rusty Russell
e0c21debc2 pytest: fix sleep(1) in test_channel_persistence.
It's obviously not enough under load, as Travis demonstrates.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-15 17:37:29 +02:00
Rusty Russell
b1182702ae pytest: increase timeout for test_payment_failed_persistence
This happened under travis.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-15 17:37:29 +02:00
arowser
97118c558e add --conf parameter 2018-07-15 09:45:55 +00:00
Mark Beckwith
cf12130627 params: shortened names
This is a cosmetic change only. No functional changes.

I shortened the names of macros and changed param_parse() to param().

Also went through params.h with a fine-toothed comb and updated the comments
to reflect the current API.

I wanted to change the files:

	params.c -> param.c
	params.h -> param.h
	run-params.c -> run->param.c

but that confused `git diff` for params.h so its best left for another PR.

I'm keeping #1682 updated locally with all these changes.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-07-13 23:44:50 +00:00
Mark Beckwith
891dee739d params: Fix arg size
Fixes #1668

Reported-by: @jsarenik

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-07-13 11:30:52 +00:00
Mark Beckwith
7ce19b0bc8 Improved comment.
Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-07-13 11:21:39 +00:00
Rusty Russell
cf86c74870 params: add helper to provide default initialization.
@wythe points out that many cases want a default value, not NULL.
Nicer to do it in the param_parse() call.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-13 11:21:39 +00:00
gallizoltan
1220a4eb1d pylightning: add missing arguments to getroute 2018-07-13 13:12:49 +02:00
Christian Decker
14c6310a4f gossip: Fix concurrent PR merge issue with structeq
PR #1618 in parallel with the migration to macro `structeq` created this.

Fixes #1674
2018-07-08 19:04:46 +02:00
Jonathan Zernik
ec55bf3b3f README.md: 18332 is default bitcoind rpc port for testnet. 2018-07-08 16:08:56 +02:00
Rusty Russell
68a8eeea21 htlc_wire: rename malformed to failcode in struct failed_htlc.
I'm not completely convinced that it's only ever set to a failcode
with the BADONION bit set, especially after the previous patches in
this series.  Now that channeld can handle arbitrary failcodes passed
this way, simply rename it.

We add marshalling assertions that only one of failcode and failreason
is set, and we unmarshal an empty 'fail' to NULL (just the the
generated unmarshalling code does).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-08 15:56:34 +02:00
Rusty Russell
5a184c24e8 channeld: add extra check to channel_force_htlcs.
None of these sanity checks should fail, but let's be thorough: we
were testing for htlc->fail but not failcode when fulfilling an HTLC.
The failing-htlc case had this correct already.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-08 15:56:34 +02:00
Rusty Russell
efee948d3a channeld: handle HTLCs failed by failcode uniformly.
'struct htlc' in channeld has a 'malformed' field, which is really only
used in the "retransmit updates on reconnect" case.  That's quite confusing,
and I'm not entirely convinced that it can only be set to a failcode
with the BADONION bit set.

So generalize it, using the same logic we use in the master daemon:

failcode: a locally generated error, for channeld to turn into the appropriate
          error message.
fail: a remotely generated onion error, for forwarding.

Either of these being non-zero/non-NULL means we've failed, and only one
should be set at any time.

We unify the "send htlc fail/fulfill update due to retransmit" and the
normal send update paths, by always calling send_fail_or_fulfill.

This unification revealed that we accidentally skipped the
onion-wrapping stage when we retransmit failed htlcs!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-08 15:56:34 +02:00
Rusty Russell
8155bfcf18 channeld: make channel_fulfill_htlc return the HTLC it fulfulled.
This is the same pattern as channel_fail_htlc, and in fact one caller
wanted it already.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-08 15:56:34 +02:00
Rusty Russell
e92f244b80 channeld: hoist make_failmsg above send_fail_or_fulfill.
Move only.  Needed for next patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-08 15:56:34 +02:00
Rusty Russell
6e9ae98e1e lightningd: don't send uninialized malformed fields to channeld.
==1224== Uninitialised byte(s) found during client check request
==1224==    at 0x152CAD: memcheck_ (mem.h:247)
==1224==    by 0x152D18: towire (towire.c:17)
==1224==    by 0x152DA1: towire_u16 (towire.c:28)
==1224==    by 0x142189: towire_failed_htlc (htlc_wire.c:29)
==1224==    by 0x16343F: towire_channel_init (gen_channel_wire.c:596)
==1224==    by 0x115C2C: peer_start_channeld (channel_control.c:249)
==1224==    by 0x131701: peer_connected (peer_control.c:503)
==1224==    by 0x117820: gossip_msg (gossip_control.c:182)
==1224==    by 0x139D97: sd_msg_read (subd.c:500)
==1224==    by 0x139676: read_fds (subd.c:327)
==1224==    by 0x179D52: next_plan (io.c:59)
==1224==    by 0x17A84F: do_plan (io.c:387)
==1224==  Address 0x1ffefffabe is on thread 1's stack
==1224==  in frame #2, created by towire_u16 (towire.c:26)

Followed by:

2018-06-18T21:53:04.129Z lightningd(1224): 03933884aaf1d6b108397e5efe5c86bcf2d8ca8d2f700eda99db9214fc2712b134 chan #1: Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: received ERROR channel d0101486543e1a8b6871556a4fe1fba4ad4d83ce7f6f92919fd17bd1545d2fd5: UpdateFailMalformedHtlc message doesn't have BADONION bit set

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-08 15:56:34 +02:00
Rusty Russell
b14cc0c9f7 lightningd/params: fix typesafe check.
typesafe_cb isn't suitable here, as it is simply a conditional cast,
and the result is passed through '...' and doesn't matter.

Reported-by: @wythe
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-07 23:49:25 +00:00
Rusty Russell
dd7afc33ee pytest: fix flaky test in test_htlc_sig_persistence
We can shutdown before the tx hits bitcoind.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-07 16:26:23 +02:00
Rusty Russell
ed83bbe623 pytest: fix flaky race in test_gossip_query_channel_range.
We weren't waiting for gossipd to actually process the
dev_set_max_scids_encode_size message, so under Travis it sometimes
split the reply before processing that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-07 16:26:23 +02:00
Rusty Russell
57794b9285 gossipd: also delay locally-generated disables when peer vanishes.
Note that we mark both directions of the channel disabled immediately,
it's just the broadcast of the update which is delayed, just like the
ones generated when channeld tells us to.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-07 16:07:53 +02:00
Rusty Russell
f9b8237d50 gossipd: delay generation of local updates.
We disable the channel every time the peer disconnects; if it reconnects
we get two updates.

The simplest solution: delay all updates by 15 seconds.  Replace any
pending delayed update.  If update is redundant after 15 seconds,
discard.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-07 16:07:53 +02:00
Rusty Russell
55a09d79b9 pytest: make test_gossip_no_empty_announcements robust when update delayed.
In this test we tell l3 to disconnect on sending WIRE_CHANNEL_ANNOUNCEMENT.
This is hit by gossipd (to disconnect from l2) but *also* channeld to
disconnect from l4.  That's OK, because normally by this point l4 has
sent its real channel_update.

However, the next patch introduces a delay in sending channel_updates,
meaning l4 hasn't sent it yet.  If l3 doesn't reconnect to l4, we
never get the channel_update and the test which expects l1 to eventually
see both sides of the channel fails.

So we manually reconnect then.  Note that we remove the redundant
'dev-no-reconnect' option from l2: it's added automatically as it
doesn't set 'may_reconnect'.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-07 16:07:53 +02:00
Rusty Russell
ef59a8f4aa gossipd: suppress redundant local updates which we would generate.
This doesn't do anything for us now, since we actually tend to produce
DISABLE/ENABLE update pairs.  But the infrastructure is useful for the
next patch.

We also add more details to the trace message in the core update code.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-07 16:07:53 +02:00
Rusty Russell
2d533dc82e channeld: don't manually disable channel.
gossipd will do it when peer dies anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-07 16:07:53 +02:00
Rusty Russell
8e571ba688 listnodes: expose global features.
Since nobody sets these yet, it's a bit moot, but it will be great in
future.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-07 16:07:53 +02:00
Rusty Russell
9fa738a741 listpeers: expose peer features as 'local_features' and 'global_features'
For now, just the connected peers.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-07 16:07:53 +02:00
Rusty Russell
7b735fbeee gossipd: fix json_listpeers printing node information.
json_listpeers returns an array of peers, and an array of nodes: the latter
is a subset of the former, and is used for printing alias/color information.

This changes it so there is a 1:1 correspondance between the peer information
and nodes, meaning no more O(n^2) search.

If there is no node_announce for a peer, we use a negative timestamp
(already used to indicate that the rest of the gossip_getnodes_entry
is not valid).

Other fixes:
1. Use get_node instead of iterating through the node map.
2. A node without addresses is perfectly valid: we have to use the timestamp
   to see if the alias/color are set.  Previously we wouldn't print that
   if it didn't also advertize an address.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-07 16:07:53 +02:00
William Casarin
a9cf73380a cli: launch a manpage when help is passed an argument
example

  lightning-cli help pay
  lightning-cli help cli

Signed-off-by: William Casarin <jb55@jb55.com>
2018-07-07 05:01:31 +00:00
Christian Decker
13796f0d0f travis: Add xmllint to the builder image
Seems that it is required for the manpage conversion.
2018-07-05 00:24:33 +00:00
Mark Beckwith
fc2d955b01 Fixed spacing in run-params
Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-07-05 00:19:16 +00:00
Mark Beckwith
7d9ad89010 params: removed the param_opt_tok macro
There doesn't seeem to be a need for this anymore (unless I'm missing something).
I added the sendpay_nulltok() unit test to confirm.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-07-05 00:19:16 +00:00
Mark Beckwith
1b50ea2abd params: removed tal context.
@rustyrussell showed we don't need temporary objects for params.

This means params no longer need a tal context.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-07-05 00:19:16 +00:00
Rusty Russell
32ccfa5b29 test/run-params: suppress stderr for expected failures.
And use err() instead of perror/exit(0) (we should have exit(1) there anyway).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-05 00:19:16 +00:00
Rusty Russell
6ff901d7b0 params: simplify lifetimes of params.
@wythe points out we don't need to keep the around now param_is_set()
is removed.  We can in fact go further and avoid marshalling them into
temporary objects at the caller altogether.

This means internally we have an array of struct param, rather than an
array of 'struct param *', which causes most of the noise in this
patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-05 00:19:16 +00:00
Rusty Russell
3f6f9e6fe0 param: make sure the name is a string literal.
We're using a macro anyway, so appending "" make it a compile-time check.

Complicates testing a bit, since we actually use generated names there.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-05 00:19:16 +00:00
Rusty Russell
899ff02e36 params: use asort.
It's a little neater than qsort here.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-05 00:19:16 +00:00
Rusty Russell
9f83a9ae4d params: make optional args do allocation for you.
This is a bit more natural, IMHO.  The only issue is that json_tok_tok is
special, so we end up with param_opt_tok() if you really want an optional
generic token.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-05 00:19:16 +00:00
Mark Beckwith
4d1d0438e1 Typesafe callback system for parsing json
This is part of #1464 and incorporates Rusty's suggested updates from #1569.

See comment in param.h for description, here's the basics:

	unsigned cltv;
	const jsmntok_t *note;
	u64 msatoshi;
	struct param * mp;

	if (!param_parse(cmd, buffer, tokens,
			 param_req("cltv", json_tok_number, &cltv),
			 param_opt("note", json_tok_tok, &note),
			 mp = param_opt("msatoshi", json_tok_u64, &msatoshi),
			 NULL))
		return;

	if (param_is_set(mp))
		do_something()

There is a lot of developer mode code to make sure we don't make mistakes,
like trying to unmarshal into the same variable twice or adding a required param
after optional.

During testing, I found a bug (of sorts) in the current system.  It allows you
to provide two named parameters with the same name without error; e.g.:

	# cli/lightning-cli -k newaddr addresstype=p2sh-segwit addresstype=bech32
	{
		  "address": "2N3r6fT65PhfhE1mcMS6TtcdaEurud6M7pA"
	}

It just takes the first and ignores the second.  The new system reports this as an
error for now.  We can always change this later.
2018-07-05 00:19:16 +00:00
Rusty Russell
fed5a117e7 Update ccan/structeq.
structeq() is too dangerous: if a structure has padding, it can fail
silently.

The new ccan/structeq instead provides a macro to define foo_eq(),
which does the right thing in case of padding (which none of our
structures currently have anyway).

Upgrade ccan, and use it everywhere.  Except run-peer-wire.c, which
is only testing code and can use raw memcmp(): valgrind will tell us
if padding exists.

Interestingly, we still declared short_channel_id_eq, even though
we didn't define it any more!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-04 23:57:00 +02:00
Rusty Russell
4a1ca0fb99 gossipd: don't use raw secp256k1_pubkey in routing.
We wrap it in 'struct pubkey' for typesafety and consistency, and the
next patch takes advantage of that when we move to pubkey_eq.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-04 23:57:00 +02:00
Jan Sarenik
9f519afc5d tools/check-includes.sh: shellcheck recommended fixes
$ shellcheck --version
    ShellCheck - shell script analysis tool
    version: 0.5.0
    license: GNU General Public License, version 3
    website: https://www.shellcheck.net
    $ make check-source
    ...

    In tools/check-includes.sh line 14:
        if [[ $(grep -cE "^#((ifndef|define) ${HEADER_ID}|endif /\* ${HEADER_ID} \*/)$" "${HEADER_FILE}") != 3 ]]; then
                                                                 ^-- SC1117: Backslash is literal in "\*". Prefer explicit escaping: "\\*".

    In tools/check-includes.sh line 28:
        git ls-files | grep -v 'ccan/' | grep -E "\.${1}"'$'
                                                  ^-- SC1117: Backslash is literal in "\.". Prefer explicit escaping: "\\.".
2018-07-04 01:48:21 +00:00
Christian Decker
582ea1a33b jsonrpc: Remove dev-blockheight in favor of getinfo
`getinfo` has been providing the blockheight for a good while and doesn't
require the `DEVELOPER=1` flag during compilation, so it should be the preferred
method to retrieve the blockchain height.
2018-07-04 00:08:14 +00:00
William Casarin
4a1b68502e wallet: fix up semi-cryptic wallet error message
There is evidence that this message can be confusing[1].
Suggest a way to resolve the error in the message.

[1] https://www.reddit.com/r/Bitcoin/comments/7ybcq9/programming_how_can_i_resolve_the_below_error/

Signed-off-by: William Casarin <jb55@jb55.com>
2018-07-03 23:58:55 +00:00
Sjors Provoost
f7dabd4fd8 Armbian build instructions 2018-07-03 11:49:55 +02:00
Jan Sarenik
cb53ad0706 wallet/wallet.c: bind_null if (payment_key) 2018-07-02 05:00:08 +00:00
Jan Sarenik
da80447ba3 wallet/wallet.com: bind_null if (out->preimage) 2018-07-02 05:00:08 +00:00
Jan Sarenik
5205de7b8e wallet/wallet.c: bind_null if (in->preimage) 2018-07-02 05:00:08 +00:00