Commit graph

321 commits

Author SHA1 Message Date
Rusty Russell
6af8f29392 bolt11: better message when you try to 'lightning-cli pay' an onchain addr.
Give a clear error at the beginning if it's not bolt11 payment,
rather than falling foul of other checks.

This will work at least until some altcoin adapts the 'ln' prefix :)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-17 22:59:20 +00:00
lisa neigut
66ca2a333f channeld: use u64 fo htlc_minimum_msat
As per spec.
2018-10-16 03:32:27 +00:00
Christian Decker
0ee0da0eb8 common: Add a test for io_lock
Totally forgot to add this test. It just shows how a writer can take
exclusive access of a socket over multiple `io_write` calls, and
queuing all others behind it.
2018-10-11 01:43:55 +00:00
Christian Decker
bb38541d9e common: Added a small locking mechanism to simplify how we lock IO
We've done this a number of times already where we're getting
exclusive access to either the out direction of a connection, or we
try to lock out the read side while we are responding to a previous
request. They usually are really cumbersome because we reach around to
the other direction to stop it from proceeding, or we flag our
exclusive access somewhere, and we always need to know whom to notify.

PR ElementsProject/lightning#1970 adds two new instances of this:

 - Streaming a JSON response requires that nothing else should write
   while the stream is active.
 - We also want to stop reading new requests while we are responding
   to one.

To remove the complexity of having to know whom to stop and notify
when we're done, this adds a simple `io_lock` primitive that can be
used to get exclusive access to a connection. This inverts the
requirement for notifications, since everybody registers interest in
the lock and they get notified if the lock holder releases it.
2018-10-11 01:43:55 +00:00
Rusty Russell
df27fc55af More renaming of gfeatures to globalfeatures.
Use the BOLT #1 naming.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-09 08:40:52 +00:00
Rusty Russell
76e7f8aa5c wireaddr: add wireaddr_eq.
We can't STRUCTEQ_DEF() it since addrlen is variable, so open-code it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-28 18:20:17 +02:00
Rusty Russell
41b0872f58 Use localfeatures and globalfeatures consistently.
That's what BOLT #1 calls them; make it easier for people to grep.

Reported-by: @niftynei
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-28 04:14:28 +00:00
Rusty Russell
a24ed20dee common/cryptomsg: remove unused async routines.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-28 04:14:28 +00:00
Rusty Russell
a1bdaa8f99 connectd/peer_exchange_initmsg: handle peer comms ourselves.
connectd is the only user of the cryptomsg async APIs; better to
open-code it here.  We need to expose a little from cryptomsg(),
but we remove the 'struct peer' entirely from connectd.

One trick is that we still need to defer telling lightningd when a
peer reconnects (until it tells us the old one is disconnected).  So
now we generate the message for lightningd and send it once we're woken.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-28 04:14:28 +00:00
Rusty Russell
d1552bd1eb test/run-cryptomsg.c: move out of lightningd/ into common/
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-28 04:14:28 +00:00
Rusty Russell
dfaaa09bc6 common/cryptomsg: remove unused 'reading_body' flag and 'peer_in_started'.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-28 04:14:28 +00:00
Rusty Russell
96f05549b2 common/utils.h: add tal_arr_expand helper.
We do this a lot, and had boutique helpers in various places.  So add
a more generic one; for convenience it returns a pointer to the new
end element.

I prefer the name tal_arr_expand to tal_arr_append, since it's up to
the caller to populate the new array entry.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-27 22:57:19 +02:00
Christian Decker
c3f433ec66 json: Support streaming JSON messages
It turns out we were heavily relying on the fact that after each message from
the client there'd be a flush, and that there would not be anything after the
JSON object we read. This will no longer be the case once we start streaming
things or we are very quick in issuing the JSON-RPC requests.

This just takes one of the error paths (incomplete read) and makes it into a
successful path if we have indeed read a full root element.
2018-09-25 05:47:23 +00:00
lisa neigut
b1f15c2605 BOLT updates: broken link fixes
See a9195a84d0
2018-09-21 00:24:12 +00:00
lisa neigut
b287f2f007 BOLT 11 human-readable formatting changes 2018-09-21 00:24:12 +00:00
Rusty Russell
e012e94ab2 hsmd: rename hsm_client_wire_csv to hsm_wire.csv
That matches the other CSV names (HSM was the first, so it was written
before the pattern emerged).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-20 09:49:39 +02:00
Saibato
d5c3626fa7 parse autotor: address before separate_address_and_port
this enables addr like --addr=autotor:127.0.0.1 or
--addr=autotor:localhost to just use the default tor service port

Signed-off-by: Saibato <Saibato.naga@pm.me>
2018-09-20 09:09:11 +02:00
Christian Decker
dc88c35d7f channeld: Do not fail if we get a chain_hash we don't know 2018-09-14 21:18:11 +02:00
Christian Decker
2402c524cc channeld: Keep track of the chainparams for the chain we are using 2018-09-14 21:18:11 +02:00
Rusty Russell
7b9341e762 subdaemon: better GDB support.
It was annoying me, so I made it much nicer to use.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-04 14:36:05 +02:00
Rusty Russell
76f116daf1 lightningd: minor cleanups
Code changes:
1. Expose daemon_poll() so lightningd can call it directly, which avoids us
   having store a global and document it.
2. Remove the (undocumented, unused, forgotten) --rpc-file="" option to disable
   JSON RPC.
3. Move the ickiness of finding the executable path into subd.c, so it doesn't
   distract from lightningd.c overview.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-03 05:01:40 +00:00
Mark Beckwith
aa60057134 param: upgraded json_tok_escaped_string
Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-09-03 00:40:27 +00:00
Mark Beckwith
c32f7910cc param: upgraded json_tok_label
Added utility function json_tok_is_num so I would avoid using goto.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-09-03 00:40:27 +00:00
Rusty Russell
90d1062a55 daemon_conn: fix memory leak when passing an fd.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-24 19:54:32 +02:00
Rusty Russell
a2828ed40b memleak: allow for scanning non-talloc regions.
For some daemons we'll be handing it non-talloc memory to scan for pointers.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-24 19:54:32 +02:00
Rusty Russell
8733015836 memleak: don't require a root pointer.
We can just track everything from NULL (the ultimate parent) down.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-24 19:54:32 +02:00
Rusty Russell
37ea0d3c7f memleak: fix exclude check.
We want to exclude the child from being entered into the htable:
if we wanted the parent we could do this outside the loop.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-24 19:54:32 +02:00
Rusty Russell
112b7336a3 memleak: create and use a generic htable helper and generic intmap helper.
memleak can't see into htables, as it overloads unused pointer bits.
And it can't see into intmap, since they use malloc (it only looks for tal
pointers).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-24 19:54:32 +02:00
Rusty Russell
12a39b8a79 lightningd: fix backtraces in memleak detection.
We were using a *different* backtrace_state var, which was always NULL.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-24 19:54:32 +02:00
Rusty Russell
7f37fa3263 derive_basepoints: harden checking.
I managed to crash the HSM by asking for point -1 (shachain_index has an
assert).  Fail in this case, instead.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-23 14:46:22 +02:00
Rusty Russell
8340d8c070 secret_eq: remove in favor of constant time variant.
To be safe, we should never memcmp secrets.  We don't do this
currently outside tests, but we're about to.

The tests to prove this as constant time are the tricky bit.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-23 14:46:22 +02:00
Rusty Russell
4c891f4661 common: log when we toggle IO logging, don't edit env in tests!
Tests were failing when in the same thread after a test which set
log_all_io=True, because SIGUSR1 seemed to be turning logging *off*.

This is due to Python using references not copies for assignment.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-23 14:46:22 +02:00
Rusty Russell
9f044305db pytest: dev env var LIGHTNINGD_DEV_LOG_IO turns io logging on immediately.
This is required for the next test, which has to log messages from channeld
as soon as it starts (so might be too late if it sends SIGUSR1).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-23 14:46:22 +02:00
Rusty Russell
ebaf5eaf2e channeld: send option_data_loss_protect fields.
We ignore incoming for now, but this means we advertize the option and
we send the required fields.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-23 14:46:22 +02:00
Rusty Russell
b123b1867d shachain: shachain_get_secret helper.
This is a wrapper around shachain_get_hash, which converts the
commit_num to an index and returns a 'struct secret' rather than a
'struct sha256' (which is really an internal detail).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-23 14:46:22 +02:00
Mark Beckwith
6254d15efd param: upgraded json_tok_sha256
Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith
947752b9cc param: upgraded json_tok_percent
Made it a local static since its only used once.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith
8ebc95b7b0 param: upgraded json_tok_bool
Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith
e5918f4e5a param: upgraded json_tok_double
Also renamed old version to json_to_double for use as a utility funciton.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith
bab8ff991a param: upgraded json_tok_u64
Also renamed old version to json_to_u64 for use as a utility funciton.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith
294dc06de9 param: upgraded json_tok_number
Also renamed old version to json_to_number for use as a utility function.

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-20 01:02:25 +00:00
Mark Beckwith
8f17191099 param: upgraded json_tok_tok to advanced callback
This was a very simple change and allowed us to remove the special
`json_opt_tok` macro.

Moved the callback out of `common/json.c` to `lightningd/json.c` because the new
callbacks are dependent on `struct command` etc.
(I already started on `json_tok_number`)

My plan is to:
	1. upgrade json_tok_X one a time, maybe a PR for each one.
	2. When done, rename macros (i.e, remove "_tal").
	3. Remove all vestiges of the old callbacks
	4. Add new callbacks so that we no longer need json_tok_tok!
	   (e.g., json_tok_label, json_tok_str, json_tok_msat)

Signed-off-by: Mark Beckwith <wythe@intrig.com>
2018-08-13 23:46:35 +00:00
Rusty Russell
db22d2366e crypto_sync: sync_crypto_write_no_delay to flush TCP after critical packets.
Avoid that 200ms loss.  We don't want to disable nagle generally,
since it's great for gossip and other traffic; we just want to push at
critical times.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-10 12:46:45 +02:00
Rusty Russell
b4e6a0fcad peer_failed: write error message to peer directly.
We currently hand the error back to the master, who then stores it for
future connections and hands it back to another openingd to send and exit.

Just send directly; it's more reliable and simpler.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-09 19:44:27 +02:00
Rusty Russell
17f7f50814 wireaddr: correctly parse ':portnum' (meaning IPv4 and IPv6)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-08 15:04:06 +02:00
Rusty Russell
136f10e4a3 common/read_peer_msg: remove.
Also means we simplify the handle_gossip_msg() since everyone wants it to
use sync_crypto_write().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-05 02:03:58 +00:00
Rusty Russell
0b08601951 sync_crypto_write/sync_crypto_read: just fail, don't return NULL.
There's only one thing the caller ever does, just do that internally.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-05 02:03:58 +00:00
Rusty Russell
09cce4a9c7 common/read_peer_msg: deconstruct into individual helper routines.
The One Big API is confusing, and has enough corner cases that we should
ditch it rather than add more.

See: https://www.sandimetz.com/blog/2016/1/20/the-wrong-abstraction

In particular, when openingd is changed to chat to peers even when
it's not actively opening a channel, it wants to handle (most) errors
by continuing, not calling peer_failed().

This exposes the constituent parts.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-05 02:03:58 +00:00
Rusty Russell
a409cc26d9 memleak: reenable CCAN_TAL_DEBUG in DEVELOPER mode.
It no longer has any effect on tal_len(), but it *does* give file and line
of allocations which is much nicer for tracking memory leaks!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-05 02:03:58 +00:00
Rusty Russell
4bdacea7b5 common/utils: don't free tmpctx, just the children.
In some daemons I want to hand it into a loop, which would call
clean_tmpctx().  This causes a subtle bug.

So just free the children directly: the pointer itself remains valid.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-08-05 02:03:58 +00:00