Commit Graph

6969 Commits

Author SHA1 Message Date
Rusty Russell
3437f7e25d devtools/gossipwith: change timeout to seconds.
I always get this wrong, then wonder why it's dying!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-18 11:01:20 +01:00
Rusty Russell
709c98f539 logging: remove spaces from subsystem names.
Spaces just make life a little harder for everyone.

(Plus, fix documentation: it's 'jsonrpc' not 'json' subsystem).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-18 04:50:22 +00:00
Rusty Russell
0aad532db1 logging: add examples for logging.
As suggested by @cdecker, but also did a bit of minor reformatting.

I don't like the excessive indenting in our man pages though,
but that's a separate problem.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-18 04:50:22 +00:00
Rusty Russell
24f708ee7b lightningd: kill per-peer daemons if they claim a different peer id.
Suggested-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-18 04:50:22 +00:00
Rusty Russell
f91ff993ea pytest: deflake test_htlc_send_timeout
We can definitely get a pong from l1 (should test be slow enough):
it's l3 we are concerned about.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-18 04:50:22 +00:00
Rusty Russell
b210c9d91f lightningd: use a simple array for less memory usage.
The tal overhead of 5 pointers, the linked list node is 2; and we also
tal'd the string.  That's 96 bytes per entry.

Use a simple array instead, though it means more work on deletion
since each log_entry is no longer a tal object.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-18 04:50:22 +00:00
Rusty Russell
39d537b495 lightningd: remove log_add functions.
They added complexity, and were only used in a few places.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-18 04:50:22 +00:00
Rusty Russell
a83fd16840 lightningd: perform better log pruning.
1. Don't prune the last 10%.
2. Be more aggressive on pruning IO and DEBUG.
3. Account for skipped entries correctly across multiple prunes.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-18 04:50:22 +00:00
Rusty Russell
0c6d8996fc lightningd: avoid keeping multiple copies of nodeid.
Simple refcount FTW.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-18 04:50:22 +00:00
Rusty Russell
4fc498f901 lightningd: enable io logging on subdaemons iff we're going to print it.
This simplifies our tests, too, since we don't need a magic option to
enable io logging in subdaemons.

Note that test_bad_onion still takes too long, due to a separate minor
bug, so that's marked and left dev-only for now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-18 04:50:22 +00:00
Rusty Russell
0607f998d1 options: allow --log-level <level>:<prefix> for finegrained log control.
This allows finegrained logging control of particular subdaemons or
subsystems.

To do this, we defer setting the logging levels for each log object
until after early argument parsing (since e.g. "bitcoind" log object
is created early).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-changed: Options: log-level can now specify different levels for different subsystems.
2019-11-18 04:50:22 +00:00
Rusty Russell
ef7a820ab1 log: make formatting more consistent.
1. Printed form is always "[<nodeid>-]<prefix>: <string>"
2. "jcon fd %i" becomes "jsonrpc #%i".
3. "jsonrpc" log is only used once, and is removed.
4. "database" log prefix is use for db accesses.
5. "lightningd(%i)" becomes simply "lightningd" without the pid.
6. The "lightningd_" prefix is stripped from subd log prefixes, and pid removed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-changed: Logging: formatting made uniform: [NODEID-]SUBSYSTEM: MESSAGE
Changelog-removed: `lightning_` prefixes removed from subdaemon names, including in listpeers `owner` field.
2019-11-18 04:50:22 +00:00
Rusty Russell
86fb54a33b lightningd: remove per-peer log book.
We had a separate logbook for each peer, and copy log entries above
the printable log level into the master logbook.  This didn't always
work well, since we didn't dump it on crash for example.

Keep a single global logbook instead, and remove this infrastructure.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-18 04:50:22 +00:00
Rusty Russell
3270e5c843 lightningd: move log structs into log.c.
Simply better encapsulation.   We still need to expose log_entry, since the
notification hook uses it though.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-18 04:50:22 +00:00
Rusty Russell
eed654f684 connectd, gossipd: use per-peer logging.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-18 04:50:22 +00:00
Rusty Russell
00cb5adfe6 common: allow subdaemons to specify the node_id in status messages.
This is ignored in subdaemons which are per-peer, but very useful for
multi-peer daemons like connectd and gossipd.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-18 04:50:22 +00:00
Rusty Russell
e433d4ddc1 lightningd: have logging include an optional node_id for each entry.
A log can have a default node_id, which can be overridden on a per-entry
basis.  This changes the format of logging, so some tests need rework.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-18 04:50:22 +00:00
Rusty Russell
4fa7b30836 lightningd: have optional node_id associated with subdaemons.
We'll use this for logging it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-18 04:50:22 +00:00
Rusty Russell
b0a72a67e8 doc/lightningd-config.5.txt: remove
Replaced by doc/lightningd-config.5.md.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-18 04:50:22 +00:00
Rusty Russell
7f0a36600a lightningd: fix uninitialized variable
==1310== Conditional jump or move depends on uninitialised value(s)
==1310==    at 0x127C7F: io_loop_with_timers (io_loop_with_timers.c:30)
==1310==    by 0x14F0E1: plugins_init (plugin.c:1019)
==1310==    by 0x12E4B1: main (lightningd.c:694)
==1310==

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-17 12:38:33 +01:00
darosior
f075b87137 bitcoind: remove the chainparams member
We now have a global constant, prefer to use it instead of having
two variables with the same utility.
2019-11-15 13:14:08 +01:00
Rusty Russell
fdd69af1f6 gossipd: don't discard node_announcements with old timestamps.
It really, really doesn't matter.  But we were dramatically reducing
our view of the network:

In my gossip_store (mainnet):
  channel_announcement: 30349
  channel_update: 55119
  node_announcment: 1783

Changelog-Fixed: No longer discard most node_announcements (fixes #3194)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-15 13:01:34 +01:00
Michael Schmoock
d7609e61b5 pytest: add test for raising listtransactions after funding 2019-11-15 03:53:56 +00:00
Michael Schmoock
fe4a25a780 wallet: fix skipping tx dups memory corruption
Changelog-Fixed: #3231 listtransactions crash
2019-11-15 03:53:56 +00:00
Michael Schmoock
ed238758a2 wallet: fix accessing blockheight of unconfirmed transaction 2019-11-15 03:53:56 +00:00
nicolas.dorier
0d55dca50f Revert "docker: Use system libraries, don't bring your own"
This reverts commit 48728d3828.
2019-11-14 13:37:04 +01:00
arowser
39fa486cb2 remove unnecessary software package from arm Dockerfile 2019-11-14 10:20:28 +01:00
Rusty Russell
323e4f6288 dev: add option to prevent HTLC timeouts.
This is required for the protocol tests, which can be slow.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-14 10:19:01 +01:00
Rusty Russell
21555b277d tests: test that we only use tlv onion when advertized.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-14 10:15:33 +01:00
Rusty Russell
bb06bec891 features: set OPT_VAR_ONION (bit 9) iff EXPERIMENTAL_FEATURES
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-14 10:15:33 +01:00
Rusty Russell
9dc8cff9b4 devtools/onion: use raw sphinx helper or new style, allow TLV.
This means we can make sphinx_add_v0_hop static, too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-14 10:15:33 +01:00
Rusty Russell
0211712f5e sphinx: separate nonfinal from final interface, add tlv option.
For legacy, they were the same, but for TLV we care whether it's the
final hop or not.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-14 10:15:33 +01:00
Rusty Russell
c83834ca82 lightningd: expose/accept "style" parameter in routes.
Default is legacy.  If we have future styles, new strings can be defined,
but for now it's "tlv" or "legacy".

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-14 10:15:33 +01:00
Rusty Russell
997e3f7fe6 lightningd: move json_add_route into gossip_control.c and make static.
There's only one caller.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-14 10:15:33 +01:00
Rusty Russell
3a25e9b8d6 gossipd: add hop-style to nodes to mark whether they speak TLV onion.
We keep the feature bitmap on disk, so we cache this in the struct
explicitly.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-14 10:15:33 +01:00
Rusty Russell
2a2259083a lightningd: handle tlv-style payloads.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-changed: JSON API: `htlc_accepted` hook has `type` (currently `legacy` or `tlv`) and other fields directly inside `onion`.
Changelog-deprecated: JSON API: `htlc_accepted` hook `per_hop_v0` object deprecated, as is `short_channel_id` for the final hop.
2019-11-14 10:15:33 +01:00
Rusty Russell
b7bbccd6fa common/sphinx: handle decoding of TLV payload.
We add routines to decode the expected fields from both legacy and tlv
hop formats.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-14 10:15:33 +01:00
Rusty Russell
a76518a029 common/sphinx: rename hop_data to hop_data_legacy.
This highlights the various places we need to change.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-14 10:15:33 +01:00
Richard Myers
2821187fdc trim whitespace from end of onion before decoding 2019-11-14 00:12:53 +00:00
Richard Myers
1c6ac953af fixed to properly pass outgoing cltv to hopdata 2019-11-14 00:12:53 +00:00
Richard Myers
8dbb32afec fixed generate/decode to use assocdata 2019-11-14 00:12:53 +00:00
Richard Myers
d4c074676e fixed argv indexes for parsing commands 2019-11-14 00:12:53 +00:00
Rusty Russell
eaa72edbd6 pytest: fix make pytest
Travis adds the correct PYTHONPATH, but "make check" doesn't.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-13 10:59:05 +01:00
lisa neigut
28cdccfb11 dev: add option flag for specifying temporary channel id
--dev-force-tmp-channel-id flag takes a 64-character hex string
to use as the temporary channel id. Useful for spec tests

[ Fixed crash in non-DEVELOPER mode --RR ]
Changelog-None
2019-11-13 05:51:02 +00:00
lisa neigut
a333df449a listpeers: show close_to address
If a 'upfront_shutdown_script' was specified, show the address +
scriptpubky in `listpeers`

Changelog-added: JSON API: `listpeers` channels now include `close_to` and `close_to_addr` iff a `close_to` address was specified at channel open
2019-11-13 03:31:20 +00:00
lisa neigut
963a1da958 addr: handle P2SH/P2PKH in scriptpubkey encoding
Previously, returned null if a scriptpubkey was not Segwit; now
handles encoding to Base58 for other types.
2019-11-13 03:31:20 +00:00
lisa neigut
6f215a70e5 contrib: fixup documentation of l1/2-log 2019-11-13 03:31:20 +00:00
Rusty Russell
7679f25aec devtools: fix mkcommit crash.
Needs to initialize global now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-13 01:45:32 +00:00
Christian Decker
9378be72a9 pyln: Allow users to override the LightningNode class
Quite a few of the things in the LightningNode class are tailored to their use
in the c-lightning tests, so I decided to split those customizations out into
a sub-class, and adding one more fixture that just serves the class. This
allows us to override the LightningNode implementation in our own tests, while
still having sane defaults for other users.
2019-11-12 21:23:55 +01:00
Christian Decker
bc9b1c405e pyln: Move RPC and daemon instantiation into the LightningNode
We were relying heavily on NodeFactory to do some magic before instantiating
the Node with rpc and daemon initialized, that meant that we'd have to replace
all 3 classes when customizing the node to our needs. Moving that
initialization into the node itself means that the LightningNode class now can
be swapped out and customized, without having to wire everything else through.
2019-11-12 21:23:55 +01:00