Commit Graph

11511 Commits

Author SHA1 Message Date
Simon Vrouwe
94f16f14b7 test: test_invoices add a stress test 2022-07-16 14:19:05 +09:30
Rusty Russell
dbae5ae569 common/json_stream.c: provide explicit json_add_primitive_fmt and json_add_str_fmt routines.
Rather than a generic "add member", provide two routines: one which
doesn't quote, and one which does.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-15 12:24:00 -05:00
Rusty Russell
36a29fbfbc lightningd/json.h: remove.
There are hardly any lightningd-specific JSON functions: all that's left
are the feerate ones, and there's already a comment that we should have
a lightningd/feerate.h.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-15 12:24:00 -05:00
Rusty Russell
401f1debc5 common: clean up json routine locations.
We have them split over common/param.c, common/json.c,
common/json_helpers.c, common/json_tok.c and common/json_stream.c.

Change that to:
* common/json_parse (all the json_to_xxx routines)
* common/json_parse_simple (simplest the json parsing routines, for cli too)
* common/json_stream (all the json_add_xxx routines)
* common/json_param (all the param and param_xxx routines)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-15 12:24:00 -05:00
Rusty Russell
f12f0d6929 common: remove json_tok.
Make local copies for tests, and fundchannel plugin (which just passes
params through).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-15 12:24:00 -05:00
Rusty Russell
a9b992ff4a plugins/spender/multifundchannel: remove json_tok.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-15 12:24:00 -05:00
Rusty Russell
e621b8b24e lightningd: remove gratuitous param_tok from help and config.
They predate json_string!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-15 12:24:00 -05:00
Rusty Russell
ec76ba3895 lightningd/connect_control: remove param_tok from connect.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-15 12:24:00 -05:00
Rusty Russell
814cde5623 lightningd/closing_control: remove param_tok().
It's bad form; if we do parsing inside params() then the `check`
command is much more effective.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-15 12:24:00 -05:00
Rusty Russell
0236d4e4da common/json_stream: remove useless attempt at oom handling.
We tell membuf to use tal, and tal never returns NULL, so this
code can never be triggered.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-15 12:24:00 -05:00
Rusty Russell
9685c1adaf lightningd: remove getsharedsecret.
This was introduced to allow creating a shared secret, but it's better to use
makesecret which creates unique secrets.  getsharedsecret being a generic ECDH
function allows the caller to initiate conversations as if it was us; this
is generally OK, since we don't allow untrusted API access, but the commando
plugin had to blacklist this for read-only runes explicitly.

Since @ZmnSCPxj never ended up using this after introducing it, simply
remove it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: JSONRPC: `getsharedsecret` API: use `makesecret`
2022-07-15 22:17:58 +09:30
Rusty Russell
c34a0a22ad makesecret: change info_hex arg to simply "hex" to match datastore command.
And fix schema: it wasn't tested as there was no test-by-parameter-name.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-15 22:17:58 +09:30
niftynei
769efe8d54 tests: massively speed up our wait for enormous feerates
We do smoothing, waiting for this to hit the target (50k+) was taking
longer than my TIMEOUT=15; here we increase the speed at which it hits
exit velocity, so to speak.
2022-07-15 22:16:27 +09:30
niftynei
f2e7e9d919 coin-moves: only log htlc_timeout pair for penalty txs
We cleanup our output tracking for timeout txs when the peer's
htlc_timeout self-expiry is hit; we'd also log its spend if happen to
see it get spent.

This is a bit of a race as they can't spend it until the locktime is
available. Hence the flakiness in tests that expected the `htlc_timeout`
to *not* be spent.

Instead, we only log an external's `htlc_timeout` spend in the case
where we also immediately register another output to track for it (only
happens when said htlc is stealable)

Fixes #5405
In-Collab-With: @ddustin
2022-07-15 22:16:27 +09:30
Rusty Russell
6fd8fa4d95 connectd: optimize requests for "recent" gossip.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-15 21:18:29 +09:30
Rusty Russell
62a5183fb5 common: add ability for gossip_store to track by timestamp.
We'll use this to keep a two-hour-ago "recent gossip offset" as an
optimization.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-15 21:18:29 +09:30
Rusty Russell
dcf2916dcb devtools: add --print-timestamps to dump-gossipstore.
Allowed me to sanity check the next patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-15 21:18:29 +09:30
Rusty Russell
92fe871467 connectd: optimize case where peer doesn't want gossip.
LND and us send 0xFFFFFFFF to turn off gossip.  LDK and Eclair don't
seem to turn off gossip at all, but that's OK.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-15 21:18:29 +09:30
Christian Decker
50180e040c msggen: Update generated files to add Listpeers.peers[].remote_addr
Changelog-None
2022-07-15 15:27:45 +09:30
Christian Decker
1a9979784f contrib: Remove obsolete libhsmd_python class
This has been replaced with better rust bindings that can then be
consumed via pyo3, consolidating the C interface in a portable
wrapper.

Changelog-Removed: libhsmd: Removed the `libhsmd_python` wrapper as it was unused
2022-07-15 15:27:45 +09:30
Christian Decker
99367dbb32 make: Add msggen generated files to check-gen-updated target
Changelog-None
2022-07-15 15:27:45 +09:30
Igor Bubelov
5ef6729224 Use correct naming 2022-07-14 22:39:18 -05:00
Vincenzo Palazzo
b624c53051 plugin: add check on the type json object during the IO message handling 2022-07-14 12:49:23 -05:00
niftynei
d284b98911 notify: channel_state_changed now receives notice when channel opens
Previously we wouldn't notify when a channel moves into state
"CHANNELD_AWAITING_LOCKIN", as this is the original state (so there's
no movement btw states). This meant that it's impossible to track when a
channel's commitment txs have been exchanged and we're waiting for
onchain confirmation.

It's useful to have notice of this initialization though, all in one
place so that the `channel_state_changed` notification can successfully
track the entire lifecycle of a channel, from inception to close.

Note that for v2 "dual-funded" channels, we already notify at the same place, at
"DUALOPEND_AWAITING_LOCKIN" (the initial state for a dualopend channel
is "DUALOPEND_OPEN_INIT" -- this is the only state we don't get notified
at now...)

Changelog-Added: Plugins: `channel_state_changed` now triggers for a v1 channel's initial "CHANNELD_AWAITING_LOCKIN" state transition (from prior state "unknown")
2022-07-14 12:42:48 -05:00
Rusty Russell
bdefbabbef lightningd: re-transmit all closing transactions on startup.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-14 12:40:57 -05:00
Rusty Russell
d544ae075b pytest: test (failing) that we rexmit closing transactions on startup.
Usually bitcoind will have them, but it's best to re-xmit.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-14 12:40:57 -05:00
adi2011
7b160b203a tests: Add tests for the RPCs
Changelog-Added: Static channel backup, to enable smooth fund recovery in case of complete data loss
2022-07-14 12:24:48 -05:00
adi2011
829fe09c13 doc: Add documentation for new RPCs and a FIXME: in fromschema.py 2022-07-14 12:24:48 -05:00
adi2011
1450f1758c Plugin: Add new plugin for SCB, Updated makefile and gitignore as well 2022-07-14 12:24:48 -05:00
adi2011
6ba8abb0de lightningd: Add RPC for populating DB with stub channels and set an error on reconnecting 2022-07-14 12:24:48 -05:00
adi2011
286d6c3165 tools/gen: Always return bool! 2022-07-14 12:24:48 -05:00
adi2011
1a1be6abd6 lightningd/peer_control: Add RPC for fetching scb for all the in-memory channels. 2022-07-14 12:24:48 -05:00
adi2011
eca844eb36 channel: Add struct scb_chan in channel and making last tx optional. 2022-07-14 12:24:48 -05:00
adi2011
e42ba8366b common: Add scb_wire for serializing the static_chan_backup 2022-07-14 12:24:48 -05:00
adi2011
64c03f8990 hsmd: Create derive_secret and makesecret RPC for deriving pseudorandom keys from HSM 2022-07-14 12:24:48 -05:00
niftynei
ba7d4a8f6b make-schema: don't include tools/fromschema.py in SHASUMS
We were including the entire list of prerequisites when generating a
shastamp, which for schemas includes the `tools/fromschema.py` doc.

This meant all of our shasums were updating anytime this tool file
changed.

Instead, we just include the first prerequisite.

See: https://www.gnu.org/software/make/manual/html_node/Automatic-Variables.html#Automatic-Variables
2022-07-14 12:24:48 -05:00
Alex Myers
27e5ddc7b7 gossipd: fix of gossip v10 channel removal handling
Node announcement indices should be reinitialized after removing the
referenced announcement.  Debugging output also corrected.

Changelog-None
2022-07-14 09:51:12 +09:30
niftynei
95ec897ac0 dual-fund: Fail if you try to buy a liquidity ad w/o dualfunding on
Fixes #5271

In-Collaboration-With: Base58 'n Coding Seminar Participants

Changelog-Changed: `fundchannel` now errors if you try to buy a liquidity ad but dont' have `experimental-dual-fund` enabled
2022-07-13 12:29:50 -05:00
Michael Schmoock
f67ab2a86e test: run-gossmap update test store instructions
Changelog-None
2022-07-12 13:59:46 -05:00
Michael Schmoock
b3177e945f doc: mention ip discovery only active when no addresses are announced
Changelog-None
2022-07-12 13:59:46 -05:00
Michael Schmoock
301acc9556 gossipd: only use IP discovery if no addresses are announced
This will only add the discovered `remote_addr` IPs if no other
addresses would be announced. Meaning whenever a public address was
found by autobind or an address was specified via commandline or config,
IP discovery will be disabled.

Addresses: #5305

Note from the author: We could/should also enable IP discovery when we only
have a TOR address (but without --always-use-proxy ofc). This will give
nodes an option to have a bootstrap way to be reached until IP discovery
can do the job in a more stable way.

Changelog-Changed: Only use IP discovery as fallback when no addresses would be announced
2022-07-12 13:59:46 -05:00
Michael Schmoock
da85014fdf gossipd: nit: update a misleading comment
This commit got reduced to just changing a comment because
the stuff it initially did was already merged in before by
commit 7ff62b4a

So I just kept the changed comment as its more precise.

Changelog-None
2022-07-12 13:59:46 -05:00
Rusty Russell
1217f479df sendpay: allow route to contain both amount_msat and (deprecated) msatoshi.
Since it's a deprecation, we simply ignore one, rather than properly
checking they match etc.

Fixes: #5386
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-12 12:42:05 -05:00
Rusty Russell
1d78911b29 pytest: test that we allow both msatoshi and amount_msat is route for sendpay.
We deprecated msatoshi, but getroute() still returns both (unless
--allow-deprecated-apis=False), so we need to accept both.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-12 12:42:05 -05:00
Rusty Russell
f98df63b75 Revert "pytest: fix test_gossip_no_backtalk flake."
This reverts commit 6cc4858847.

No longer needed now we don't flush gossip rcvd_filter as aggressively.
2022-07-12 21:41:19 +09:30
Rusty Russell
06e1e119aa pytest: fix test_gossip_no_empty_announcements flake.
This is a side-effect of fixing aging: sometimes, we age our
rcvd_filter cache too fast, and thus re-xmit.  This breaks
our test, since it used dev-disconnect on the channel_announce,
but that closes to l3, not l1!

```
>       assert l1.rpc.listchannels()['channels'] == []
E       AssertionError: assert [{'active': T...ags': 1, ...}] == []
E         Left contains 2 more items, first extra item: {'active': True, 'amount_msat': 100000000msat, 'base_fee_millisatoshi': 1, 'channel_flags': 0, ...}
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes: #5403
2022-07-12 21:41:19 +09:30
Alex Myers
ddf8fbdb5d gossipd: fix crash from gossip_store v10 changes
routing.c fixed to properly remove rate-limited gossip_store entries
when channels are closed. This caused gossipd to crash on a subsequent
gossip_store_load. Also corrects an overzealous limit of one gossip_store
entry per message (should now allow one broadcastable and one
rate-limited). Addresses issues 5387, 5395.

Changelog-None
2022-07-12 14:18:06 +09:30
Rusty Russell
312751075c lightningd: save outgoing information for more forwards.
There's one case where we can present more infomation, so do that, and
fix documentation.

Changelog-Added: JSON-RPC: `listforwards` now shows `out_channel` in more cases: even if it couldn't actually send to it.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Fixes: #5329
2022-07-12 06:38:11 +09:30
Rusty Russell
3a1a7eb93f wallet: allow saving forwarding scid even if we don't have amount.
They're not logically connected: we can know where they wanted to
go, but we didn't send it.

Where possible, it's the scid *they asked for*; otherwise, it's the
scid or fallback to the alias, but do this in the *caller*, not by
overriding inside wallet_forwarded_payment_add.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-12 06:38:11 +09:30
Rusty Russell
d6afb0cd8d lightningd: allow outgoing_scid without outgoing amount.
This (will) happen if they ask is to forward to an unknown scid.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2022-07-12 06:38:11 +09:30