Commit Graph

11029 Commits

Author SHA1 Message Date
Simon Vrouwe
175e0b54b3 testing: fix flake in test_ping_timeout
[ Folded 2 fixups --RR ]
2022-03-01 09:38:40 +10:30
Simon Vrouwe
c286eb053f pyln-client: Plugin, improve background comment 2022-03-01 09:38:40 +10:30
Simon Vrouwe
ad22535d34 pyln-testing: improve description of wait_for_logs 2022-03-01 09:38:40 +10:30
Simon Vrouwe
84bead9396 pyln-testing: in LightningNode.openchannel, make wait_for_announce more reliable
it now waits for 'alias' in node_announcement, not just block confirms.
more cleanup
2022-03-01 09:38:40 +10:30
Simon Vrouwe
f84e1a0536 pyln-testing: in utils, update method fundbalancedchannel, now it also works with dualfund
and some cleanup, fundchannel has been returning tx, txid and outnum for years
2022-03-01 09:38:40 +10:30
Simon Vrouwe
2d06c38997 plugin-funder: fix typos in option lease-fee-base-msat and funder-fuzz-percent
Changelog-Experimental: option `--lease-fee-base-msat` renamed to `--lease-fee-base-sat`

Changelog-Experimental: option `--lease-fee-base-msat` deprecated and will be removed next release
2022-03-01 09:38:40 +10:30
Simon Vrouwe
0173717b6a doc: add details about: log-level, experimental options and features field returned by listnodes,
listchannels and connect.
2022-03-01 09:38:40 +10:30
Simon Vrouwe
9bbb52097e lightningd: fix parsing of option dev-force-features
It did not pick-up the last (bolt11) bits set.
2022-03-01 09:38:40 +10:30
Simon Vrouwe
905a85dc99 JSON-RPC: getinfo, add field our_features
All build flags and (experimental) options make it hard to find
out what features are supported or enabled.

And the undocumented `--list-features-only`, does not account for all
our featurebits, for example bit 55 (keysend).

Changelog-Added: JSON-RPC: `getinfo` result now includes `our_features` (bits) for various Bolt #9 contexts
2022-03-01 09:38:40 +10:30
Simon Vrouwe
32fb8a42fd testing: cleanup in test_plugin_shutdown 2022-03-01 09:38:40 +10:30
Simon Vrouwe
760c271381 devtools: in decodemsg, add newline and fix decoding a node_announcement with tlvs
Terminal prompt got messed up because missing newline in case of empty fields.

printwire_addresses expected it to be the last field, which is not
the case of a node_announcement with tlv
2022-03-01 09:38:40 +10:30
Rusty Russell
05bd62fee4 configure: restore -Og default on non-developer builds.
Reverts change in 1ef77504b1.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-28 14:42:41 -06:00
Rusty Russell
dd495b2b21 memleak: fix handling of excluded pointers.
We often hand an exclude pointer (usually the current command) to
memleak.  But when we encountered this we would stop iterating, rather
than just ignore it: this means we would often ignore significant siblings.

In particular, fixing this (which has always been there) reveals many
previously-undetected leaks.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-27 07:44:26 +10:30
Rusty Russell
ea71e9fe5b memleak: fix handling of notleak pointers.
We were ignoring the *parent* of the notleak pointers, not the notleak
pointer itself!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-27 07:44:26 +10:30
Rusty Russell
fe86395e7c memleak: restore functionality accidentally removed.
Accidentally removed in 6c9b752751.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-27 07:44:26 +10:30
Rusty Russell
7081413e20 plugins/funder: actually free the pending_opens.
It's a small leak, but real.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-27 07:44:26 +10:30
Rusty Russell
d5064ff56c libplugin: make memleak see current requests.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-27 07:44:26 +10:30
Rusty Russell
9bf2aa65cd common/param: don't keep around the temporary array off cmd.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-27 07:44:26 +10:30
Rusty Russell
3218e5332f common/param: fix leak with opt_param_def.
We allocate the default, then callback allocates over the top.  Mark
params with a default, so we can free that when it's called.

(We can't do this generally, since not all param args are actually
pointers to pointers, though opt_param_def has to be).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-27 07:44:26 +10:30
Rusty Russell
f0ea4d60b9 onchaind: fix minor leaks.
Not actually leaks, but they do live longer than they need.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-27 07:44:26 +10:30
Rusty Russell
c5b424b6ce plugins/pay: make memleak happy.
1. Tell memleak about our linked-list of current payments.
2. Don't remove them from list until we actually free them (in destructor, naturally).
3. Decode invoices into tmpctx (we steal / copy what we want anyway).
4. Free params after we've used them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-27 07:44:26 +10:30
Rusty Russell
246b724dbe libplugin-pay: fix spurious leak reports.
1. The dijkstra can be temporary, doesn't need to last as long as pay cmd.
2. We fail multiple times in several places, so don't leak old failreason.
3. Make payments findable by our memleak detector.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-27 07:44:26 +10:30
Rusty Russell
4b7d2dc5b8 common/bolt11: make decoded routes hang off routes arr, not b11.
This causes weirdness in pay, which tal_steal's b11->routes and expects
to get it all.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-27 07:44:26 +10:30
Rusty Russell
b3338a174e plugins/keysend: prevent leak detecting getting upset.
1. We don't keep a pointer to payments (unlike pay, which keeps a
   linked list), so mark it notleak.
2. plugin->our_features is overloaded for "features we want to set" (used by keysend)
   and then "features we have".  Create a new field, which is cleaner.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-27 07:44:26 +10:30
Rusty Russell
6c8ea95f6e hsmd: remove unnecessary memleak_remove_region.
The current client is in dbid_zero_clients (i.e. it's lightningd).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-27 07:44:26 +10:30
Rusty Russell
b67329bec2 lightningd: don't leak address for printing.
Not really a leak, but it would last as long as the peer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-27 07:44:26 +10:30
Rusty Russell
899f54894f gossipd: move deferred_update list ptr to head to ease memleak checks.
They can only follow pointers (without help) if they point to the
*start* of a tal object.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-27 07:44:26 +10:30
Rusty Russell
a901402e66 plugins/pay: fix unnecessary code.
1. p is a child of cmd, so it's freed by command_failed.
2. cltv_budget is set a few lines up to the same thing already.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-27 07:44:26 +10:30
Rusty Russell
2a6d3cad01 libbacktrace: update to latest.
Seeing if this helps my build box which previously gives:

```
/home/rusty/lightning-ltest/lightningd/lightning_connectd: libbacktrace: unrecognized DWARF version in .debug_info at 6
/home/rusty/lightning-ltest/lightningd/lightning_connectd: libbacktrace: no debug info in ELF executable
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-27 07:44:26 +10:30
Christian Decker
86b83e473b pytest: Remove 3 stress-tests
These tests have proven to be:

 a) very expensive, as they spin up many nodes, and perform long setup
 b) are not testing anything specific, they just fuzz functionality
    that is already tested otherwise
 c) have not helped pinpoint any issues in living memory
 d) are very flaky, making for really bad signal-to-noise, so much
    that devs usually just restart without even looking at the logs
 e) even if we were to look at the logs, we'd be unable to reproduce
    due to the inherent randomness involved in these tests
 f) are really noisy neighbors, causing other tests to flake as well,
    further muddying the water

All in all, these tests are a waste of time, and source of
frustration.

[ Cleaned up python unused imports --RR ]
Changelog-None
2022-02-26 14:36:50 +10:30
Matt Whitlock
ec6d91fd54 hsmd: don't leak message buffers
client_read_next(…) calls io_read_wire(…), passing &c->msg_in as the
address of a pointer that will be set to the address of the buffer that
io_read_wire_(…) will allocate, and passing c (a pointer to the struct
client instance) as the parent for the new allocation. As long as the
struct client instance eventually gets freed, the allocated message
buffer will be freed too, so there is no "leak" in the strict sense of
the term, but the freeing of the buffer may not occur for an arbitrarily
long time after the buffer has become disused, and indeed many millions
of message buffers may be allocated within the lifetime of one struct
client instance.

handle_client(…) ultimately hands off the c->msg_in to one of several
message-type-specific handler functions, and those functions are not
TAKES or STEALS on their message buffer parameters and do not free their
message buffer arguments. Consequently, each successive call to
client_read_next(…) will cause io_read_wire_(…) to overwrite the
c->msg_in pointer with the address of a newly allocated message buffer,
and the old buffer will be left dangling off of the struct client
instance indefinitely.

Fix this by initializing c->msg_in to NULL in new_client(…) and then
having client_read_next(…) do `c->msg_in = tal_free(c->msg_in)` prior to
calling io_read_wire(…). That way, the previous message buffer will be
freed just before beginning to read the next message. The same strategy
is already employed in common/daemon_conn.c, albeit without nulling out
dc->msg_in after freeing it.

Fixes: #5035
Changelog-Fixed: hsmd: Fixed a significant memory leak
2022-02-26 11:15:22 +10:30
Rusty Russell
1da9b30b9a gossipd: don't send updates in error messages for unannounced channels.
This restores the behaviour prior to `lightningd: use our cached
channel_update for errors instead of asking gossipd.`, where gossipd
would refuse to give us channel_updates for unannounced channels.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-23 14:47:39 +10:30
Dustin Dettmer
d56904db80 channeld: Comment update for channeld
Receiving gossip is now handled globally by connectd
2022-02-23 14:47:11 +10:30
Rusty Russell
01f2ca4fe7 pytest: remove test_htlc_rexmit_while_closing as too flaky.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-23 14:46:10 +10:30
Dustin Dettmer
bdeeaab631 Clean up outdated comments about gossipd 2022-02-23 10:11:33 +10:30
William Casarin
2b92ac4236 bolt11: mark when expiry is decoded
It looks like the x tag isn't marked when parsed either?
2022-02-22 09:58:48 +10:30
William Casarin
af16b9b9f4 bolt11: mark when we decode min_final_cltv_expiry
It looks like decode_c doesn't set have_c unlike the other decode_
methods. At the start of the function, decode_c checks have_c to see if
it's set, but it is never set. It seems like this could allow for
duplicate c tags, which is probably not intended.

Signed-off-by: William Casarin <jb55@jb55.com>
2022-02-22 09:58:48 +10:30
William Casarin
ff84b3f773 json_add_invoice: fix crash if missing invstring
If this field is missing for whatever reason (weird db state?)
clightning will crash when listing invoices.

Signed-off-by: William Casarin <jb55@jb55.com>
2022-02-22 09:57:09 +10:30
azuchi
167fade0fa Add LIGHTNINGD_NETWORK env variable to Dockerfile for ARM
Changelog-Added: Docker build for ARM defaults to `bitcoin`, but can be overridden with the `LIGHTNINGD_NETWORK` envvar.
2022-02-22 09:51:08 +10:30
Vincenzo Palazzo
11c94528b3 doc: reintroduce the fmt command to fmt the schema
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2022-02-22 09:48:16 +10:30
Christian Decker
1ef77504b1 misc: Add build targets for the tarball and debian packages
The tarball needs to materialze all submodules, and git needs to be
removed as a build dependency.
2022-02-22 09:43:56 +10:30
Michael Schmoock
df9a34b81e chore: use EXPERIMENTAL for BOLT1 remote_addr #917 2022-02-22 05:45:47 +10:30
Michael Schmoock
6db97b4235 pytest: check for remote_addr 2022-02-22 05:45:47 +10:30
Michael Schmoock
38e2abf68a peer_exchange: set, read and log remote_addr
Changelog-Added: Protocol: set remote_addr on init tlvs
2022-02-22 05:45:47 +10:30
Michael Schmoock
03a1df074a peer_wire: add remote_addr to init_tlv
Unfortunately we can't do any smart parsing here since
wiregen does not support switch/type cases for different
substructure unions yet. So just give us a pointer we can use.
2022-02-22 05:45:47 +10:30
Michael Schmoock
7c8b12db0b wiregen: adds note about what <count> can be 2022-02-22 05:45:47 +10:30
Rusty Russell
d0c7e18995 bitcoind: importmulti fails (bitcoin master), use importdescriptors
But this requires a watch-only wallet, and python-bitcoinlib doesn't support
multiple wallets, so we need to unload the original one, but then we need
to generate a block, so that can't generate a new address, so we need
an address arg to generate_block.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-20 19:29:44 +10:30
Rusty Russell
c4ec1d576e pytest: fix test_statictor_onions
It wasn't binding to .onion:<l2.port>, but .onion:9735.  Test both cases.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-20 19:29:44 +10:30
Rusty Russell
00cbe6959b pytest: YA bitcoind API break.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-02-20 19:29:44 +10:30
Sebastian Falbesoner
53d4e9d2b8 bitcoin/tx: remove unused SEGREGATED_WITNESS_FLAG
This define is unused since commit
509bb2c7ae.

Changelog-None
2022-02-16 15:58:10 -06:00