Commit Graph

3032 Commits

Author SHA1 Message Date
Rusty Russell
eb4062bb14 setchannelfee: fix crash when channel is not in valid state.
You can't fail a cmd when you've already started streaming
a successful response:

lightningd: ccan/ccan/json_out/json_out.c:343: json_out_finished: Assertion `tal_count(jout->wrapping) == 0' failed.
lightningd: FATAL SIGNAL 6 (version v0.9.2-119-gf7cdf1d)
0x1847d1 send_backtrace
	common/daemon.c:38
0x184877 crashdump
	common/daemon.c:51
0x5bda03f ???
	???:0
0x5bd9fb7 ???
	???:0
0x5bdb920 ???
	???:0
0x5bcb489 ???
	???:0
0x5bcb501 ???
	???:0
0x1e07a8 json_out_finished
	ccan/ccan/json_out/json_out.c:343
0x18db0a json_stream_double_cr
	common/json_stream.c:95
0x18dbf3 json_stream_close
	common/json_stream.c:117
0x12fd98 command_raw_complete
	lightningd/jsonrpc.c:459
0x12fec9 command_failed
	lightningd/jsonrpc.c:488
0x12ffb9 command_fail
	lightningd/jsonrpc.c:503
0x14dc20 json_setchannelfee
	lightningd/peer_control.c:2052

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: JSONRPC: `setchannelfee` would fail an assertion if channel wasn't in normal state.
2020-12-21 17:10:28 -06:00
Rusty Russell
723c16072a cleanups: feedback from Christian Decker review.
1. Hoist 7200 constant into the bolt12 heade2.
2. Make preimage the last createinvoice arg, so we could make it optional.
3. Check the validity of the preimage in createinvoice.
4. Always output used flag in listoffers.
5. Rename wallet offer iterators to offer_id iterators.
6. Fix paramter typos.
7. Rename `local_offer_id` parameter to `localofferid`.
8. Add reference constraints on local_offer_id db fields.
9. Remove cut/paste comment.
10. Clarify source of fatal() messages in wallet.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-15 12:05:02 +01:00
Rusty Russell
a33e39b7e8 pay, wallet: rename internal bolt11 vars to invstring.
And handle bolt12 strings if EXPERIMENTAL_FEATURES.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-15 12:05:02 +01:00
Rusty Russell
4c4288e3e5 invoice: handle bolt12 strings if EXPERIMENTAL_FEATURES.
The database still calls them `bolt11`, but we treat them depending on
prefix.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-15 12:05:02 +01:00
Rusty Russell
fab0842d31 lightningd: createinvoice low-level invoice creation command.
This takes an unsigned bolt11 (or bolt12 if EXPERIMENTAL_FEATURES) string
and signs it and puts it in the database.

The invoice command could now be moved out to a plugin, in fact.

Changelog-Added: JSON-RPC: `createinvoice` new low-level invoice creation API.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-15 12:05:02 +01:00
Rusty Russell
d66dbd473a lightningd: add low-level offer interfaces.
The real work is done in a plugin, but provide enough API that we can
manipulate the db.
2020-12-15 12:05:02 +01:00
Rusty Russell
3f4683e3f8 sendpay: optional argument to link local offer.
This is for offers which have `send_invoice`: we need to associate the
payment with the original offer, in (the usual) case where it is a single
use offer.  We mark it used when it's paid, to avoid a race.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-15 12:05:02 +01:00
Rusty Russell
963f6b5d67 invoice: add an optional local_offer_id.
This allows us to mark an offer used when an invoice derived from it
is paid, and importantly, avoid any other invoices for the offer being
paid.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-15 12:05:02 +01:00
Rusty Russell
d971e3de98 Plugin: support extra args to "start".
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Plugins: `start` command can now take plugin-specific parameters.
2020-12-15 09:28:56 +10:30
Rusty Russell
8a9976c4c1 plugins: support concatenation of multiple args.
"multi" means that specifying a parameter twice will append, not override.
Multi args are always given as a JSON array, even if only one.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Plugins: new "multi" field allows an option to be specified multiple times.
2020-12-15 09:28:56 +10:30
Rusty Russell
646c564ec5 plugins: remove deprecated string plugin options.
This was fixed in 0.8.2.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: plugins: options to init are no longer given as strings if they are bool or int types (deprecated in 0.8.2).
2020-12-15 09:28:56 +10:30
Rusty Russell
32c7c133f4 common/sphinx: make onionpacket.routinginfo a dynamic member.
Still asserts that it's the standard size, but makes it a dynamic
member.  For simpliciy, changes the parse_onionpacket API (it must be
a tal object now, so we might as well allocate it here to catch all
the callers).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-11 15:51:24 +01:00
Rusty Russell
3776af4a35 common/sphinx: make TOTAL_PACKET_SIZE a macro.
This paves the way for using it on different-sized onions.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-11 15:51:24 +01:00
Rusty Russell
28a903c917 channeld: hand up onionmessage fields.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-09 16:40:13 +10:30
Rusty Russell
c1bdaa27bc sendonionmessage: add invoice, invoice_request and invoice_error fields.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-09 16:40:13 +10:30
Christian Decker
b2a5cf422f jsonrpc: Forward errors on malformed requests to cli
We were masquerading errors when parsing the request by reporting only
a bogus malformed `id` field in the response, when the real issue was
that we were unable to parse the request in the first place (which
caused the null-id error to be returned).

Fixes #4238
2020-12-09 06:56:21 +10:30
Michael Schmoock
c4b7f4d9d2 feat: adds local halfchan fees to listpeers
This will add `fee_base` (msat) and `fee_ppm` (u32 num) to the RPC
`listpeers` output.

Changelog-Added: fee_base and fee_ppm to listpeers
2020-12-07 14:26:36 +01:00
Rusty Russell
695a8bd868 lightningd: split onion_message hook.
Only way to be sure that plugins don't accidentally respond to onion_message
sent via reply path from another message (which would potentially leak our
identity!).

To quote BOLT #7 (Onion Messages) in the offers PR:

```markdown
The reader:
- MUST ignore any message which contains a `blinding` which it did not expect, or does not contain
  a `blinding` when one is expected.
...
`blinding` is critical to the use of blinded paths: there are various
means by which a blinded path is passed to a node.  The receipt of an
expected `blinding` indicates that blinded path has been used: it is
important that a node not accept unblinded messages when it is expecting
a blinded message, as this implies the sender is probing to detect if
the recipient is the terminus of the blinded path.

Similarly, since blinded paths don't expire, a node could try to use
a blinded path to send an unexpected message hoping for a response.
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-04 20:16:54 -06:00
Rusty Russell
61422193d9 channeld: hand input blinding to lightningd.
Required to determine if this msg used expected reply path.

Also remove FIXME (om->enctlv is handled above).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-04 20:16:54 -06:00
Rusty Russell
dc745cdf2f peer_htlcs.c: close taken() leak.
Found in tests/test_connection.py::test_restart_many_payments:

`lightningd: outstanding taken(): lightningd/peer_htlcs.c:532:towire_temporary_channel_failure(((void *)0), ((void *)0))`

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-03 11:35:30 +01:00
Rusty Russell
1a3c0a0c0d invoice: fix potential race where invoice is paid/expired while we're calling hook.
There's actually a (very unlikely) race here: we would previously have
crashed with an assertion in invoices_resolve.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-02 16:50:12 +10:30
ZmnSCPxj jxPCSnmZ
a437936c78 lightningd/log.c: Fix up handling of SIGHUP.
Fixes: #4240

ChangeLog-Fixed: log: Do not terminate on the second received SIGHUP.
2020-12-02 16:48:48 +10:30
niftynei
9f74dd9fb3 df: rename the dual_open wires to match other daemon wires 2020-12-02 14:19:08 +10:30
Rusty Russell
0ad269f5b6 invoice: make invoice_payment hook a multi-user hook.
We register on it for offers, and without this nobody else can.

Changelog-Changed: plugins: more than one plugin can now register invoice_payment hook.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-02 12:56:15 +10:30
Rusty Russell
ca2bd98082 unittest: use common_setup / common_shutdown almost everywhere.
Avoids much cut & paste.  Some tests don't need any of it, but most
want at least some of this infrastructure.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-02 12:55:09 +10:30
Rusty Russell
f0621cec0d JSON-RPC: don't allow any strings which aren't valid UTF-8.
We already do some sanity checks, add this one.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: invalid UTF-8 strings now rejected.
2020-12-02 10:38:04 +10:30
ZmnSCPxj jxPCSnmZ
32de621886 lightningd/plugin_hook.c: Make db_write a chained hook.
Fixes: #4219

Changelog-Changed: Plugins: Multiple plugins can now register `db_write` hooks.
2020-11-30 10:40:11 +10:30
ZmnSCPxj jxPCSnmZ
904e110554 lightningd/plugin.c: Make plugin-exclusive loop support multiple plugins. 2020-11-30 10:40:11 +10:30
niftynei
6077eca660 df: pass back 'close_to' for completed/commitment secured channels
When commitments are secured, also return the 'close_to' script if we've
got a local_upfront_shutdown_script set.
2020-11-23 12:41:05 -06:00
YOSHIDA Masanori
ff2535651e lightningd: remove unused pid_fd member in struct lightningd
Signed-off-by: YOSHIDA Masanori <masanori.yoshida@gmail.com>
Changelog-None
2020-11-23 12:13:28 -06:00
Rusty Russell
f56266c1c1 plugins: undeprecate old form of hooks.
This effectively reverts ac93b780d5.

Christian points out that plugins need time before we deprecate
the old options (probably 6 months) as they need to work with
both old and new.

Changelog-Deprecated: **UNDO** plugins: hooks should now be specified using objects, not raw names.
Suggested-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-11-17 14:42:58 +10:30
Michael Schmoock
36499e6779 fix: closer set to remote for onchain settlement
Incase we have been offline while a channel was force closed on us we
now set the 'closer' to 'remote' instead of null because this is by far
the most probable reason.

Changelog-None
2020-11-13 14:25:02 -06:00
Michael Schmoock
3e42d08c89 fix: hangup in plugin rescan
This adds a missing plugins_send_getmanifest() call in the rescan function
that lead to a RPC hangup. Not sure though if this is the proper fix.

Changelog-None
2020-11-12 15:27:04 -06:00
niftynei
cefb64cd68 htlc_accepted: log BROKEN if both 'failure_msg' + 'failure_onion'
Log an error for incorrect use of API

Suggested-By: @cdecker
2020-11-10 19:03:23 -06:00
fiatjaf
65cdb78bb4 refactor htlc_accepted_hook_deserialize. 2020-11-10 19:03:23 -06:00
fiatjaf
9e4bed73d9 optional "failure_onion" in reply to htlc_accepted hook.
Changelog-Added: `htlc_accepted` hook can now return custom `failure_onion`.
2020-11-10 19:03:23 -06:00
Rusty Russell
fb295ffb51 plugin: sort topological candidates by specified order.
We previously registered hooks up in who-replies-to-getmanifest-first
order, but then if any had dependencies it would scatter that order.

This allows users to manually set dependencies developers have
forgotten by specifying the plugins manually in their configuration or
cmdline.  This was an excellent consideration by @mschmook.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-11-09 15:22:33 -06:00
Rusty Russell
852e14c947 plugins: check order once all plugins have returned from getmanifest.
This means we need to stop at this stage even in the runtime-loaded
case.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-11-09 15:22:33 -06:00
Rusty Russell
ac93b780d5 plugins: deprecate old form of hooks.
Now both python and c libraries are updated, we can officially
deprecate the old form.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Deprecated: plugins: hooks should now be specified using objects, not raw names.
2020-11-09 15:22:33 -06:00
Rusty Russell
6a55b4367e lightningd: actually order the hooks.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-11-09 15:22:33 -06:00
Rusty Russell
e2a31f42f2 plugins: allow 'before' and 'after' arrays for hooks.
The next patch will use these to order the hooks.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: plugins: hooks can now specify that they must be called 'before' or 'after' other plugins.
2020-11-09 15:22:33 -06:00
ZmnSCPxj jxPCSnmZ
2604a81aba lightningd/opening_control.c: Remove predeclaration. 2020-11-07 17:18:49 +00:00
ZmnSCPxj jxPCSnmZ
0fd87b85da openingd/: Fail fundchannel_start if we already are, or will become, the fundee.
Fixes: #4108

Changelog-Fixed: Network: Fixed a race condition when us and a peer attempt to make channels to each other at nearly the same time.
2020-11-07 17:18:49 +00:00
Michael Schmoock
6cc96c07dc feat: adds state_changes to listpeers output
Changelog-Added: RCP: Added 'state_changes' history to listpeers channels
2020-11-06 14:47:04 -06:00
Michael Schmoock
88c1dc56e8 db: adds a state_change history to database 2020-11-06 14:47:04 -06:00
Michael Schmoock
68ce25c92d feat: adds timestamp to state_change notification 2020-11-06 14:47:04 -06:00
Michael Schmoock
b7c18517df rpc: adds opener and closer to listpeers channels
Changelog-Added: RPC: Added 'opener' and 'closer' to listpeers channels
2020-11-06 14:47:04 -06:00
Michael Schmoock
083a856c31 db: persist channel closer and state change cause 2020-11-06 14:47:04 -06:00
Michael Schmoock
8a8dabaa58 feat: adds state change cause and message
This adds a `state_change` 'cause' to a channel.
A 'cause' is some initial 'reason' a channel was created or closed by:

  /* Anything other than the reasons below. Should not happen. */
  REASON_UNKNOWN,
  /* Unconscious internal reasons, e.g. dev fail of a channel. */
  REASON_LOCAL,
  /* The operator or a plugin opened or closed a channel by intention. */
  REASON_USER,
  /* The remote closed or funded a channel with us by intention. */
  REASON_REMOTE,
  /* E.g. We need to close a channel because of bad signatures and such. */
  REASON_PROTOCOL,
  /* A channel was closed onchain, while we were offline. */
  /* Note: This is very likely a conscious remote decision. */
  REASON_ONCHAIN

If a 'cause' is known and a subsequent state change is made with
`REASON_UNKNOWN` the preceding cause will be used as reason, since a lot
(all `REASON_UNKNOWN`) state changes are a subsequent consequences of a prior
cause: local, user, remote, protocol or onchain.

Changelog-Added: Plugins: Channel closure resaon/cause to channel_state_changed notification
2020-11-06 14:47:04 -06:00
Rusty Russell
d5d9858b7b lightningd: fix similar race in stop.
Tested by putting a sleep in the rpc_command hook.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-11-06 14:05:44 -06:00
Rusty Russell
5e6b0f9445 lightningd: fix crash if we abort after enabling notifications.
The rpc_command hook means that we have a delay between receiving
a JSON command and actually calling the handler.  In this case, the
caller can go away:

```
==1348== Invalid write of size 1
==1348==    at 0x130EA6: json_notifications (jsonrpc.c:1350)
==1348==    by 0x12EE9E: command_exec (jsonrpc.c:636)
==1348==    by 0x12F3C6: rpc_command_hook_callback (jsonrpc.c:752)
==1348==    by 0x15AA08: plugin_hook_callback (plugin_hook.c:210)
==1348==    by 0x155C9D: plugin_response_handle (plugin.c:398)
==1348==    by 0x155E84: plugin_read_json_one (plugin.c:504)
==1348==    by 0x15603D: plugin_read_json (plugin.c:548)
==1348==    by 0x1D4AB3: next_plan (io.c:59)
==1348==    by 0x1D5630: do_plan (io.c:407)
==1348==    by 0x1D566E: io_ready (io.c:417)
==1348==    by 0x1D7834: io_loop (poll.c:445)
==1348==    by 0x12CFAC: io_loop_with_timers (io_loop_with_timers.c:24)
==1348==  Address 0x58 is not stack'd, malloc'd or (recently) free'd
==1348==
lightningd: FATAL SIGNAL 11 (version v0.9.1-266-ga4df315)
0x180f7e send_backtrace
	common/daemon.c:38
0x181024 crashdump
	common/daemon.c:51
0x5bd7fcf ???
	???:0
0x130ea6 json_notifications
	lightningd/jsonrpc.c:1350
0x12ee9e command_exec
	lightningd/jsonrpc.c:636
0x12f3c6 rpc_command_hook_callback
	lightningd/jsonrpc.c:752
0x15aa08 plugin_hook_callback
	lightningd/plugin_hook.c:210
0x155c9d plugin_response_handle
	lightningd/plugin.c:398
0x155e84 plugin_read_json_one
	lightningd/plugin.c:504
0x15603d plugin_read_json
	lightningd/plugin.c:548
0x1d4ab3 next_plan
	ccan/ccan/io/io.c:59
0x1d5630 do_plan
	ccan/ccan/io/io.c:407
0x1d566e io_ready
	ccan/ccan/io/io.c:417
0x1d7834 io_loop
	ccan/ccan/io/poll.c:445
0x12cfac io_loop_with_timers
	lightningd/io_loop_with_timers.c:24
0x132825 main
	lightningd/lightningd.c:1016
0x5bbab96 ???
	???:0
0x1159e9 ???
	???:0
0xffffffffffffffff ???
	???:0
Log dumped in crash.log.20201106001723
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-11-06 14:05:44 -06:00
Antoine Poinsot
8846c35ac6 dual_open: correct silent enum conversions
We were silently converting a side enum (3 variants) to a tx_role enum
(2 variants).

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-11-04 14:29:22 -06:00
Jan Sarenik
895722fd8a nit: lightningd.c: fix "Richard Stevens' advice"
See https://github.com/rustyrussell/ccan/pull/88

Changelog-None
2020-10-28 14:32:35 -05:00
Rusty Russell
ae4139ba10 lightningd: note whether payment secret is used or not.
This will allow nodes (with log-level=debug) to gather how many payments
are made without payment_secrets.  We need to know this so we know when
we can make them compulsory.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-28 12:03:14 +01:00
niftynei
4ea9d9e928 df: Pass new feerate options through to plugin, set reasonable bounds
We let the plugin decide what feerate to accept/whether or not to add
funds to the open. To aid this decision, we also send the plugin what we
(c-lightning) currently have as our max and min acceptable feerates.

We also now use these as our default for max/min acceptable feerate
range when sending an openchannel offer to a peer.

In the future, it might be a good idea to make these more easily
changeable, either via a config setting (?) or a command param.
2020-10-27 19:52:05 -05:00
niftynei
97fd18f0b5 df: incorporate a few spec changes -- serial_id is now 64-bits
And we pass 3-params for feerate so it's a 'pick a range' conversation.
2020-10-27 19:52:05 -05:00
Jan Sarenik
d46427234b nit: lightningd.c: Fix a couple of SATTS comments
SATTS stands for Shared Adventure Through The Code

 - fix "must declared"
 - fix "an zero-length"
 - fix redundant space before dot
2020-10-27 10:55:11 -05:00
niftynei
21122af3a8 dev-force-features: adds a second valid format for forcing features
The previous dev-force-features forced you to explicitly declare every
desired feature bit in an array, for each set.

Here, we allow you to also denote adding/subtracing a feature bit
by just passing in the number of the bit to flip and the direction to
turn it. e.g.

	'dev-force-features': '+223'

Will turn on opt_dual_fund/odd.

	'dev-force-features': '-16'

Will flag off opt_basic_mpp.
2020-10-26 21:31:24 -05:00
YOSHIDA Masanori
4f12ff0967 Makefile: add $(LIGHTNINGD_SRC_NOHDR) to $(ALL_C_SOURCES) for "make clean" to work properly
"make clean" removes all object files listed in $(ALL_OBJS), which is derived from $(ALL_C_SOURCES).
Source files in $(LIGHTNINGD_SRC_NOHDR) are missing from $(ALL_C_SOURCES) therefore some object files are not removed by "make clean".
This commit fixes this point.

Signed-off-by: YOSHIDA Masanori <masanori.yoshida@gmail.com>
Changelog-None
2020-10-27 10:42:48 +10:30
Rusty Russell
1e5789d421 close: add notification for slow closes.
For compatibility, we only do this if `allow-deprecated-apis` is false
for now.  Otherwise scripts parsing should use `grep -v '^# '` or
start using `-N none`.

Changelog-Added: JSON-RPC: `close` now sends notifications for slow closes (if `allow-deprecated-apis`=false)
Changelog-Deprecated: cli: scripts should filter out '^# ' or use `-N none`, as commands will start returning notifications soon
Fixes: #3925
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-23 13:53:16 +10:30
Rusty Russell
f395404a10 lightningd: infrastructure for internal notifications.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-23 13:53:16 +10:30
Rusty Russell
9f687d60d9 lightningd: forward notifications from plugins if enabled.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-23 13:53:16 +10:30
Rusty Russell
c732d8707a JSON-RPC: notifications command.
This lets callers enable notifications; we won't send any if they don't.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `notifications` command to enable notifications.
2020-10-23 13:53:16 +10:30
Rusty Russell
a95205c25c common/json_stream: add generic double-cr helper.
And make caller of json_stream_forward_change_id use it, since
we're going to reuse that.

Also call json_out_finished here, so next object doesn't have a ","
prepended.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-23 13:53:16 +10:30
Rusty Russell
becd4fe576 common: add routines for log level names.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-23 13:53:16 +10:30
Rusty Russell
0dec593aa8 lightningd: clean up close code now force is always true.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-23 13:53:16 +10:30
Rusty Russell
a8177e9013 Makefile: make check-includes check all the non-generated files.
Note that check-whitespace and check-bolt already do this, so we
can eliminate redundant lines in common/Makefile and bitcoin/Makefile.

We also include the plugin headers in ALL_C_HEADERS so they get
checked.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-22 12:14:34 +10:30
Rusty Russell
5e865ce42b Makefile: unify generated files definition.
We change gen_ to _gen, because filtering on gen_% doesn't work if they're
in subdirectories :(

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-22 12:14:34 +10:30
niftynei
d535a27104 df, bugfix: wait til after we've saved the channel to do this
this cleans up `cmd` and we're not done with it yet (we need it for
saving the channel updates to the database)
2020-10-21 09:04:32 +10:30
niftynei
c6d4bd676f dual-open,openchannel_update: include the index of the funding output
This allows us to do correct reporting via multiopenchannel :)
2020-10-21 09:04:32 +10:30
niftynei
daa55d1221 df: add notification for receiving peer's funding tx sigs
This will allow us to build complex, multi-peer transactions, with
easeTM!

Changelog-Added: EXPERIMENTAL, Plugins: `openchannel_peer_sigs` notification, which contains a peer's signatures for the funding transaction (`opt_dual_fund`)
2020-10-21 09:04:32 +10:30
niftynei
f9aab50ee8 dual-fund: rework where we send our tx-sigs message, allow peers in
Prior to this patch update, we expected a client to call
`openchannel_signed` before checking for peer's tx-sigs messages on the
wire.

When moving to a 'multifundchannel' approach, we'll need to be able to
collect sigs from our peers before sending our tx_sigs message. There's
no strict ordering on when tx-sigs messages are sent/received, so this
is fine.

To do this, we go ahead and start up channeld as soon as
commitment_sigs are secured, so that we process incoming tx-sigs from
our peers as soon as we get them.
2020-10-21 09:04:32 +10:30
niftynei
c6ad4f9b20 channel.psbt: make non-const
We update it in the next patch, which technically breaks this contract.
So we shouldn't have the contract of const on this in the first place
then.
2020-10-21 09:04:32 +10:30
niftynei
6d650064a0 df, nit: make this error message a little bit more informative 2020-10-21 06:22:18 +10:30
niftynei
a97e612442 df, bugfix: set the reserve correctly on the channel
We weren't passing it through to channeld. Gotta set it on the
uc->channel_config for it to all *just work* TM
2020-10-21 06:22:18 +10:30
niftynei
02c3f11405 df, bugfix: dont free the uc yet
we free it later, which is a problem if we also free it here.
2020-10-21 06:22:18 +10:30
niftynei
5c04ff1ad7 df: Pass the serial_id of the funding output to openchannel_init caller
This is handy/necessary for getting multifundchannel to work, as we need
to know what output to tell all the other peers about.

Changelog-Added: Experimental!! JSON-RPC: openchannel_init returns a field `funding_serial` that indicates the serial_id of the funding output in the provided PSBT
2020-10-21 06:22:18 +10:30
niftynei
4508584b21 dualfund: rearrange things so that the wire-dependent calls are separate
There's a few structs/wire calls that only exist under experimental features.
These were in a common file that was shared/used a bunch of places but
this causes problems. Here we move one of the problematic methods back
into `openingd`, as it's only used locally and then isolate the
references to the `witness_stack` in a new `common/psbt_internal` file.

This lets us remove the iff EXP_FEATURES inclusion switches in most of
the Makefiles.
2020-10-20 14:27:19 +10:30
niftynei
9d4afd5880 psbt: hoist up psbt_add_serials, so we can use it elsewhere
We're going to use this in multifundchannel.
2020-10-20 14:27:19 +10:30
niftynei
b4773203bb psbt-finalized: hoist method to common 2020-10-20 14:27:19 +10:30
niftynei
3674de9865 json: add channel_id helper 2020-10-20 14:27:19 +10:30
niftynei
4034d0c306 psbt: have the unknown map 'add' be a 'set' instead 2020-10-20 14:27:19 +10:30
niftynei
b696ec89a5 df-open: use channel_id for openchannel_update and openchannel_signed
Be as specific as possible is a good rule for things, I think
2020-10-20 12:50:31 +10:30
niftynei
085c590a51 dualopen: use separate wire for passing updated PSBTs back to dualopend
Rusty pointed out that having an empty channel_id is suboptimal; adding
another call is probably the right idea rather than re-using an existing
one.

Suggested-By: @rustyrussell
2020-10-20 12:50:31 +10:30
niftynei
41ebf71e26 psbt: new methods for generating serial_ids for an input/output
we need to do this elsewhere later, pull it out so we can use it
2020-10-20 12:50:31 +10:30
niftynei
818f152618 channel: save funding_psbt to database
Only populated until we receive the peer's funding_locked
2020-10-20 12:50:31 +10:30
niftynei
6c31eb9dd0 df-open: send the tx-sigs from channeld, check that we send it at the
...right time.

We re-send the tx_sigs on start/init/reconnect until we've gotten a
funding_locked from our peer. We also build it in channeld now, instead
of in dualopend, and don't pass in a message for them anymore
2020-10-20 12:50:31 +10:30
niftynei
865c9e9567 df-channeld: broadcast funding tx, once we get tx_sigs from peer
We broadcast the funding tx, as soon as we get the tx_sigs from the
peer, and check that it's valid
2020-10-20 12:50:31 +10:30
niftynei
aa1b8296c7 peer_control: move open_command up to where channeld can get it,
also include a method for finding a pending/available open_command
for a channel
2020-10-20 12:50:31 +10:30
niftynei
46641951fa dual-open: use tx_roles, not side, as switch
It's easier to reason about
2020-10-20 12:50:31 +10:30
niftynei
8858ae4f3d df-open: commands to update a PSBT or submit a signed PSBT
`openchannel_signed` and `openchannel_update` which allow a user to
continue a openchannel or kick off the completion of a openchannel.

`openchannel_update` should be called until it returns with
`commitments_secured`.
2020-10-20 12:50:31 +10:30
niftynei
537eeab208 df-open: add a 'open_commands' list to stash pending opens around in
`openchannel_signed` commands hang out across the openingd/channeld
boundary -- we don't return until we've successfully broadcast the
transaction (or timed out waiting for them to send a tx_sigs back).
2020-10-20 12:50:31 +10:30
niftynei
b2ec5a9f45 peer_channeld: pass over PSBT, remove second message
We need the PSBT to create the finalized tx from once the peer's
tx_signatures are received. Since we're passing the PSBT, we no longer
need the secondary message to be passed, as it was derived from the
PSBT.

Also removes now unused witness serialization code
2020-10-20 12:50:31 +10:30
niftynei
a7f29f30db df-open: pathway for getting a commit back from peer
Goes all the way back to where we save it to the database and return
whatever command kicked this off
2020-10-20 12:50:31 +10:30
niftynei
06c41a0547 dualfund: opener, openchannel_init command (1/3)
There are 3 commands for opening a channel with dualfunding.
`openchannel_init` is the first of these.

It initializes the open-channel dialog, and stops once we've run out of
updates (input/outputs) to send to the peer.
2020-10-20 12:50:31 +10:30
niftynei
9c1675fcb9 dual-fund: remove flag for option_anchor_outputs
It's assumed true
2020-10-20 12:50:31 +10:30
Rusty Russell
ec868d4acb lightningd: fix crash when we try to send fail_htlc msg to onchaind.
Great report from whitslack on this crash at startup:

```
2020-10-07T13:03:21.419Z **BROKEN** lightningd: FATAL SIGNAL 6 (version 0.9.1)
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: common/daemon.c:51 (crashdump) 0x559fb67bcc76
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: /var/tmp/portage/sys-libs/glibc-2.32-r2/work/glibc-2.32/signal/../sysdeps/unix/sysv/linux/x86_64/sigaction.c:0 ((null)) 0x7f61cdca8baf
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: ../sysdeps/unix/sysv/linux/raise.c:50 (__GI_raise) 0x7f61cdca8b31
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: /var/tmp/portage/sys-libs/glibc-2.32-r2/work/glibc-2.32/stdlib/abort.c:79 (__GI_abort) 0x7f61cdc92535
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: /var/tmp/portage/sys-libs/glibc-2.32-r2/work/glibc-2.32/assert/assert.c:92 (__assert_fail_base) 0x7f61cdc9241e
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: /var/tmp/portage/sys-libs/glibc-2.32-r2/work/glibc-2.32/assert/assert.c:101 (__GI___assert_fail) 0x7f61cdca1241
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/subd.c:750 (subd_send_msg) 0x559fb67a1c31
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/subd.c:745 (subd_send_msg) 0x559fb67a1c31
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/peer_htlcs.c:252 (local_fail_in_htlc) 0x559fb6798f77
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/peer_htlcs.c:1441 (onchain_failed_our_htlc) 0x559fb6798f77
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/onchain_control.c:339 (handle_missing_htlc_output) 0x559fb6786b9d
2020-10-07T13:03:21.419Z **BROKEN** lightningd: backtrace: lightningd/onchain_control.c:455 (onchain_msg) 0x559fb6786b9d
```

The problem is a channel with an onchaind can be in state FUNDING_STATE_SEEN,
because onchaind has started but not responded to init yet (which it does once it
has analyzed the commitment tx).

Channel B is onchain, and its onchaind fails the HTLC, and we try to send a msg
to channel A's onchaind as if it were channeld.

Explicitly check if it's channeld, rather than trying to see if it's onchaind.

Fixes: #4114
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: crash: assertion fail at restart when source and destination channels of an HTLC are both onchain.
2020-10-14 16:08:05 +10:30
Rusty Russell
4ba9ad66bc options: remove unused 'commit-fee-min/max' options.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-13 20:53:34 +02:00
Michael Schmoock
71381eb5be chore: reduce log buffer from 100MB to 10MB
The 100MB log buffer has been the biggest memory footprint for the daemon.
Keeping 10MB for emergency log dumps seems sufficient.
This has been mentioned in the last developer meeting.

Changelog-Changed: In-memory log buffer reduced from 100MB to 10MB
2020-09-26 10:38:44 +09:30
Rusty Russell
7260d9ea3d plugins: generate list of plugins more atomically, respect V=1 and --quiet.
I got a corrupt file, which looked like multiple concurrent attempts
to build it.  So instead, build it in one command, but also use
VERBOSE so we print correctly with V=1 (and --quiet).

Also move into plugins/ where it logically belongs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-24 13:19:59 +02:00
Rusty Russell
488b32b003 build: run update-mocks.
Some declarations are redundant now.  Removing them does nothing, but
it makes other PRs cleaner.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-24 13:19:59 +02:00
Michael Schmoock
980a9517c6 fix: openchannel_hook log BROKEN on dup close_to
Changelog-None
2020-09-24 11:22:47 +09:30
Rusty Russell
924cc04bd2 bolt11: have caller supply preferred chain.
This lets us distinguish testnet from signet invoices, since they
have the same prefix.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-24 09:24:14 +09:30
Rusty Russell
d5cb0d85b5 utils: use a cleaner pattern to capture wally allocations.
We force use of tal_wally_start/tal_wally_end around every wally
allocation, and with "end" make the caller choose where to reparent
everything.

This is particularly powerful where we allocate a tx or a psbt: we
want that tx or psbt to be the parent of the other allocations, so
this way we can reparent the tx or psbt, then reparent everything
else onto it.

Implementing psbt_finalize (which uses a behavior flag antipattern)
was tricky, so I ended up splitting that into 'psbt_finalize' and
'psbt_final_tx', which I think also makes the callers clearer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-23 14:45:53 +02:00
Rusty Russell
7db8680530 lightningd: don't hand around partially-assembled channel_info.
wallet_commit_channel would fill in the old_remote_per_commit and
fee_states, which is weird since the caller doesn't care.

Make the caller set all the channel_info fields, so wallet_commit_channel
is a simple consumer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-23 15:51:13 +09:30
Rusty Russell
58f6e316cc lightningd: separate fee_states out of struct channel_info.
It was always kind of weird in there anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-23 15:51:13 +09:30
niftynei
cf2a807d3c dual_open: don't release uc here, it gets freed elsewhere 2020-09-23 15:51:13 +09:30
niftynei
56b3711bbc dual_open: alignment nit 2020-09-23 15:51:13 +09:30
niftynei
ed9f661e07 dual_open: memleak problem with fee_states 2020-09-23 15:51:13 +09:30
niftynei
7d836a5945 dual_open_control: nit, reword comment 2020-09-23 15:51:13 +09:30
niftynei
aa6420cb58 dual_open: split logic for deserializing a changed vs signed PSBT
We need to run different checks on these, so split them out for finer
grained control
2020-09-23 15:51:13 +09:30
Christian Decker
1cb527d46a topo: Do not keep txids in memory indefinitely
I mistakenly assumed the block would be freed after processing completed. That
is not true since chaintopology keeps headers and stubs around for reorgs. So
we need to remove the precomputed txids along with the full_txs.
2020-09-23 13:52:49 +09:30
Rusty Russell
f37f2b6193 common/memleak: simplify and document API.
1. Rename memleak_enter_allocations to memleak_find_allocations.
2. Unify scanning for pointers into memleak_remove_region / memleak_remove_pointer.
3. Document the functions.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-23 13:52:49 +09:30
Rusty Russell
3c8049f32c bitcoin/psbt: psbt_input_add_unknown/psbt_output_add_unknown needs a tal ctx.
Since it allocates something, it needs a context (used in the next patch!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-23 13:52:49 +09:30
Rusty Russell
77b62d9e42 bitcoin/psbt: psbt_finalize needs a tal ctx.
Since it returns a wally_tx.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-23 13:52:49 +09:30
Rusty Russell
34a6294031 lightningd: really remove relative plugins dirs.
We promised this in 0.8.1!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: Plugin: Relative plugin paths are not relative to startup (deprecated v0.7.2.1)
2020-09-18 12:08:07 +09:30
Rusty Russell
288aa397fc lightningd: remove deprecated result for plugin stop.
Changelog-Removed: JSON API: `plugin stop` result with an empty ("") key (deprecated 0.8.1)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-18 12:08:07 +09:30
Rusty Russell
00115ee7f0 lightningd: remove deprecated rpc_command hook return.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: JSON API: The hook `rpc_command` returning `{"continue": true}` (deprecated 0.8.1)
2020-09-18 12:08:07 +09:30
Rusty Russell
8aa660e938 lightningd: remove deprecated return for db_write hook.
Changelog-Removed: JSON API: The hook `db_write` can no longer return `true` (deprecated in 0.8.1)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-18 12:08:07 +09:30
Rusty Russell
349f245f8c lightningd: remove deprecated per_hop_v0 in htlc hook.
Changelog-Removed: JSON API: `htlc_accepted` hook `per_hop_v0` object removed (deprecated 0.8.0)
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-18 12:08:07 +09:30
Rusty Russell
99ff474308 lightningd: remove deprecated plugin entries.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Removed: JSON API: `listconfigs` duplicated "plugin" paths (deprecated 0.8.0)
2020-09-18 12:08:07 +09:30
niftynei
b4989b9e15 listpeers: show correct amounts for starting funding allocation
With dual-funded channels, it's possible for the funding allocation of
the non-opener to not be zero
2020-09-17 10:15:29 +09:30
niftynei
9f006fdf03 memleak: 'generify' the memleak calling to code for openingd/dualopend
Switch on name of subd(aemon) as to whether to call dualopend or
openingd for memleak results
2020-09-17 10:12:08 +09:30
Rusty Russell
ea810b7011 lightningd: don't declare local vars stdin and stdout.
OpenBSD uses macros for these, and gets upset.

Fixes: #4044
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-14 03:57:17 +09:30
Rusty Russell
7f2b332021 connectd: implement connection timeout (60 seconds).
This is simple, and we now can multifundchannel to every node on testnet
(one simply hangs once we connect).

Changelog-Fixed: Protocol: We now hang up if peer doesn't respond to init message after 60 seconds.
2020-09-11 21:27:45 +09:30
Vincenzo Palazzo
f62d7bbe45 Added additional fix to code formatting and English spelling.
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2020-09-10 17:27:51 +09:30
Vincenzo Palazzo
9fbeb9bcdc delpay: code style changes and fixed docs
Changelog-Added: JSON-RPC: delpay a new method to delete the payment completed or failed.

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2020-09-10 17:27:51 +09:30
Michael Schmoock
160c564e5f fix: suppress duplicated channel_state_changed events 2020-09-10 10:24:06 +09:30
Michael Schmoock
d86855d1f7 feat: adds channel_state_changed notification
This notification will be raised whenever a channel state changes.
The payload includes the channel and peer identifiers and the
old and the new state.

Example payload:

```
{
    "channel_state_changed": {
        "peer_id": "03bc9337c7a28bb784d67742ebedd30a93bacdf7e4ca16436ef3798000242b2251",
        "channel_id": "a2d0851832f0e30a0cf778a826d72f077ca86b69f72677e0267f23f63a0599b4",
        "short_channel_id" : "561820x1020x1",
        "old_state": "CHANNELD_NORMAL",
        "new_state": "AWAITING_UNILATERAL"
    }
}
```

Changelog-Added: Plugins: channel_state_changed notification
2020-09-10 10:24:06 +09:30
Antoine Poinsot
b80ad95f1c channel_control: fix an use-after-free
As the cmd gets freed on a received error, the node id in which we iterate in `process_check_funding_broadcast`
may gets freed while we are using it.

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-09-10 10:23:35 +09:30
Moller40
4d672077e1 Avoid compile error on macos mojave
Fixing the following error by changing 'enum feerate' to int.

lightningd/bitcoind.c:183:29: error: result of comparison of constant
8 with expression of type 'enum feerate' is always true [-Werror,
-Wtautological-constant-out-of-range-compare]
        for (enum feerate f = 0; f < NUM_FEERATES; f++) {

Changelog-Fixed: compile error on macos
2020-09-09 20:43:27 +09:30
niftynei
aef5a5a0e5 psbt: psbt_has_serial_id -> psbt_find_serial_id
Cleans up some awkward spots in the code, makes the footprint a bit
neater

Suggested-By: @rustyrussell
2020-09-09 19:54:20 +09:30
niftynei
c50f377a85 psbt: pull out changeset logic into common, update API
Greatly simplify the changeset API. Instead of 'diff' we simply generate
the changes.

Also pulls up the 'next message' method, as at some point the
interactive tx protocol will be used for other things as well
(splices/closes etc)

Suggested-By: @rustyrussell
2020-09-09 19:54:20 +09:30
niftynei
5cd06227d7 build: exclude dualopend from non-experimental builds 2020-09-09 19:54:20 +09:30
niftynei
169b7817dc psbt: only compare a subset of psbt input/output fields
At some point, it's ok to add more extra info to a psbt and still not
have that be counted as 'diff'd.
2020-09-09 19:54:20 +09:30
niftynei
8d429ecd06 df: add needed info to any PSBT we produce
dual funding needs the max-witness-len and utxo fields set for every
input. we should add them when we create a 'fundpsbt', so that every
psbt that c-lightning generates is dual-funding ready
2020-09-09 19:54:20 +09:30
niftynei
9c89184c1f dualfund: add feature flag for dual-funding
turn off until we're ready to test both sides
2020-09-09 19:54:20 +09:30
niftynei
59525853ae dual-fund: accepter side implementation on lightningd
wherein we add the dual_open_control functions
2020-09-09 19:54:20 +09:30
niftynei
d6558deaa4 opening: pass two messages to channeld to send to peer
v2 of channel establishment, in the accpeter case, now sends 2 messages
to our peer after saving the information to disk (our commitment
signatures and our funding transaction signatures)
2020-09-09 19:54:20 +09:30
niftynei
0643945967 peer-control: send set of messages to start, not just one
The accepter has to send 2 messages over to channeld to send at start --
their commitment_signatures and tx_signatures
2020-09-09 19:54:20 +09:30
niftynei
cafdbbcd2d dualopend: new dualopend daemon, containing most of accepter side
oof. needs validation.
2020-09-09 19:54:20 +09:30
niftynei
6607f1d629 openingd: pull out common code, rename some things
We're going to reuse all this code for dualopend, which is coming soon.
2020-09-09 19:54:20 +09:30
niftynei
864f2f3e21 channel_id: save to database, dont derive from funding_txid
v2 channel open uses a different method to derive the channel_id, so now
we save it to the database so that we dont have to remember how to
derive it for each.

includes a migration for existing channels
2020-09-09 19:54:20 +09:30
Rusty Russell
91caf4cc30 options: handle wumbo and large-channels aliases properly.
Too trivial a fix to really list in Changelog, but I noticed that we
specified "wumbo" twice.  We should really just use the proper name
in listconfigs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-08 22:09:37 -05:00
ZmnSCPxj jxPCSnmZ
ee276bcb86 plugins/bcli.c: sendrawtransaction now has a required allowhighfees argument.
Changelog-Deprecated: plugin: `bcli` replacements should note that `sendrawtransaction` now has a second required Boolean argument, `allowhighfees`, which if `true`, means ignore any fee limits and just broadcast the transaction. Use `--deprecated-apis` to use older `bcli` replacement plugins that only support a single argument.
2020-09-09 12:38:19 +09:30
ZmnSCPxj jxPCSnmZ
abc585b7a4 onchaind/onchain_wire.csv: Allow suppression of auto-rebroadcasting for RBF txes created by onchaind. 2020-09-09 12:38:19 +09:30
ZmnSCPxj jxPCSnmZ
34bf0133f2 onchaind/onchaind_wire.csv: Propagate minimum relay fee to onchaind. 2020-09-09 12:38:19 +09:30
Christian Decker
cab52f1197 topology: Notify gossipd about spends after processing the block
We defer the notification to gossipd till the end of the spends. By itself not
a huge change, but it allows us to later migrate to doing updates blindly and
using the DB as our ground truth. It also allows us to simplify the
`wallet_outpoint_spend` semantics to not return two values in the next commit.
2020-09-09 09:24:29 +09:30
Rusty Russell
2be1f3fe1b lightningd: extract routehint selection code.
We're going to want this for bolt13 formation as well.

As a result of reworking the logic into "candidate selection" then
"route hint selection", we need to change the way round-robin works.
We use a simple incrementing index now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-08 19:47:11 +09:30
Rusty Russell
ca4730b5ff lightningd: use proper receivable calc for routehint selection.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-08 19:47:11 +09:30
Rusty Russell
84697edd6e lightningd: factor out spendable/receivable calcs, expose receivable.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-08 19:47:11 +09:30
Michael Schmoock
c2f5e8ce73 log: align level tags IO DEBUG INFO and UNUSUAL
This aligns all log TAGs except **BROKEN** which would require too many
spaces for all the other logs.

Changelog-None
2020-09-08 13:17:35 +09:30
Vincenzo Palazzo
c9687ab6ce Added proxy information to config information for plugins
Changelog-Added: Plugins: Proxy information now provided in init.configuration."

Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
2020-09-08 11:47:48 +09:30
Rusty Russell
ea819107eb common: remove funding_tx.
It's now only needed by devtools/mkfunding, so include a reduced one
there, and this also means we remove tx_spending_utxos().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-08 10:14:42 +09:30
Rusty Russell
83298c030a wallet: switch over to withdraw in module, remove lots of unused code.
This removes the reservation cleanup at startup, too, now they're all
using 'reserved_til'.

This changes test_withdraw, since it asserted that outputs were marked
spent as soon as we broadcast a transaction: now they're reserved until
it's mined.  Similarly, test_addfunds_from_block assumed we'd see funds
as soon as we broadcast the tx.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: `withdraw` now randomizes input and output order, not BIP69.
2020-09-08 10:14:42 +09:30
Rusty Russell
c34c055d82 Makefile: use completely separate spec-derived files for EXPERIMENTAL_FEATURES
This avoids overwriting the ones in git, and generally makes things neater.

We have convenience headers wire/peer_wire.h and wire/onion_wire.h to
avoid most #ifdefs: simply include those.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-08 09:42:00 +09:30
Michael Schmoock
a3e18a6aba openchannel: make openchannel hook chainable
This will make the `openchannel_hook` chainable. Logic is as follows:
 - The first plugin that rejects terminates the chain
 - If more than one plugin uses the `close_to` parameter, take the first
   value and log_unusual for the others.

Changelog-Added: openchannel_hook is now chainable
2020-09-08 06:14:29 +09:30
Christian Decker
d7ff74e8f6 json: Only start parsing once we're sure we have a message
PR #3957 improved performance considerably, however we still look over the
entire message for the message separator. If instead we just look in the
incrementally read data, we remove the quadratic behavior for large messages.

This is safe since we then loop over the messages which would drain any
message separator from the buffer before we attempt the next read.

Changelog-Fixed: bcli: Significant speedups for block synchronization
2020-09-02 13:21:32 +02:00
Christian Decker
f2797f303a block: Compute the txids only once
We're not going to mutate transactions in a block, so computing the txids
every time we need them is a waste, let's compute them upfront and use them
afterwards.
2020-09-02 13:21:32 +02:00
Rusty Russell
e4b5a1e0e8 dev: option to set what features we offer.
And use it to smoke-test !option_static_remotekey.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-02 10:52:49 +09:30
Rusty Russell
39e4796ae3 json_command: command_fail_badparam helper.
It's common to want to complain a token is not what we expected.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-02 09:46:37 +09:30
Rusty Russell
9b8f08a8a0 lightningd: return min-capacity-sat to its intended purpose.
And document exactly what it does: insist that an HTLC can pass of
this value (module assumptions of feerate).

Note that we remove the "is_opener" test from the capacity calculation
for anchor fees: it doesn't matter which side it is, someone has to pay
for anchor fees to it deducts from capacity.

This change breaks the test, which we rewrite.

Changelog-Changed: config: `min-capacity-sat` is now stricter about checking usable capacity of channels.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-01 12:27:39 -05:00
Rusty Russell
1746406e41 Makefile: normalize all the Makefiles
We create ALL_PROGRAMS, ALL_TEST_PROGRAMS, ALL_C_SOURCES and
ALL_C_HEADERS.  Then the toplevel Makefile knows which are
autogenerated (by wildcard), so it can have all the rules to clean
them or check the source as necessary.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-31 21:33:26 -05:00
Rusty Russell
8150d28575 Makefile: use generic rules to make spec-derived sources.
Now we use the same Makefile rules for all CSV->C generation.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-31 21:33:26 -05:00
Rusty Russell
dd8cd81e91 listpeers: add features array using BOLT9 names.
It's actually not possible to currently tell if you're using anchor_outputs
with a peer (since it depends on whether you both supported it at *channel open*).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-added: JSON-RPC: `listpeers` shows `features` list for each channel.
2020-08-28 14:46:09 -05:00
Rusty Russell
496c0dd1e6 common/random_select: central place for reservoir sampling.
Turns out we can make quite a simple API out of it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-28 10:56:50 +09:30
ZmnSCPxj jxPCSnmZ
ba3f38077e lightningd/invoice.c: Use round-robin channel selection.
Changelog-Changed: We now make MPP-aware routehints in invoices.
2020-08-26 09:29:46 +09:30
ZmnSCPxj jxPCSnmZ
a9a11265dd lightningd/lightningd.h: Maintain a round-robin list of channels. 2020-08-26 09:29:46 +09:30
Rusty Russell
3e52d4100d common: convert to new wire generation style.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-25 12:53:13 +09:30
Rusty Russell
398b4806b9 connectd: convert to new wire generation style.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-25 12:53:13 +09:30
Rusty Russell
fe8564555f closingd: convert to new wire generation style.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-25 12:53:13 +09:30
Rusty Russell
6d09c0eba9 openingd: convert wire to new scheme.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-25 12:53:13 +09:30
Rusty Russell
dffbf8de85 gossipd: convert wire to new scheme.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-25 12:53:13 +09:30
Rusty Russell
1702c7a69a hsmd: convert to new wire generation style.
Note that other directories were explicitly depending on the generated
file, instead of relying on their (already existing) dependency on 
$(LIGHTNINGD_HSM_CLIENT_OBJS), so we remove that.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-25 12:53:13 +09:30
Rusty Russell
8ae6740bb1 Makefile: create generic wiregen rules, use for channeld.
This means some files get renamed, and I took the opportunity to clarify
our naming (the *d* is important!)

1. channeld/channel_wire.csv -> channeld/channeld_wire.csv
2. channeld/gen_channel_wire.h -> channeld/channeld_wiregen.h
3. enum channel_wire_type -> enum channeld_wire
4. WIRE_CHANNEL_FUNDING_DEPTH -> WIRE_CHANNELD_FUNDING_DEPTH.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-25 12:53:13 +09:30
Rusty Russell
ef3fbab551 invoice: don't allow zero-value invoices.
You can't pay them anyway, and at least one person used 0 instead of "any".

Closes: #3808
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: JSON-RPC: `invoice` no longer accepts zero amounts (did you mean "any"?)
2020-08-25 12:22:45 +09:30
Rusty Russell
b09e519669 options: be more conservative with timeouts on mainnet.
As per https://github.com/lightningnetwork/lightning-rfc/pull/785

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: config: the default CLTV expiry is now 34 blocks, and final expiry 18 blocks as per new BOLT recommendations.
2020-08-24 20:55:47 +09:30
Rusty Russell
007daf6b9f doc: update bolt version
And sweep through and remove git qualifiers from many BOLT strings.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-24 20:55:47 +09:30
Rusty Russell
c284b5bbf4 lightningd: always do incremental parsing of JSON inputs.
time lightning-cli -R --network=regtest --lightning-dir /tmp/ltests-k8jhvtty/test_pay_stress_1/lightning-1/ listpays > /dev/null

Before:
	real	0m42.741s
	user	0m0.149s
	sys	0m0.016s

After:
	real	0m13.674s
	user	0m0.131s
	sys	0m0.024s

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: JSON-RPC: significant speedups for plugins which create large JSON replies (e.g. listpays on large nodes).
2020-08-21 09:52:33 +09:30
Rusty Russell
3ed03f9c8f common: make json_parse_input API retry friendly.
The jsmn parser is a beautiful piece of code.  In particular, you can parse
part of a string, then continue where you left off.

We don't take advantage of this, however, meaning for large JSON objects
we parse them multiple times before finally having enough to complete.

Expose the parser state and tokens through the API, so the caller can pass
them in repeatedly.  For the moment, every caller is allocates each time
(except the unit tests).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-21 09:52:33 +09:30
Rusty Russell
cbd0be5c0e EXPERIMENTAL_FEATURES: enable option_anchor_outputs.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-14 11:51:14 +09:30
Rusty Russell
a730cbdc96 onchaind: recognize (and ignore) anchor outputs.
Without this, onchaind fails to identify them.  So onchaind now needs
to know the funding pubkeys.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-14 11:51:14 +09:30
Rusty Russell
a5d0c14d4d option_anchor_outputs: wire into all the subdaemons.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-14 11:51:14 +09:30
Rusty Russell
e7423888ba initial_commit_tx, commit_tx: add anchor outputs if needed.
This also means we subtract 660 satoshis more everywhere we subtract
the base fee (except for mutual close, where the base fee is still
used).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-14 11:51:14 +09:30
Rusty Russell
d8d1014ca4 channeld: implement htlc sig checking changes for option_anchor_outputs.
This is best done by passing `struct bitcoin_signature` around instead
of raw signatures.  We still save raw sigs to the db, and of course the
wire protocol uses them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-14 11:51:14 +09:30
Rusty Russell
85e3b43176 channeld, openingd: take into account option_anchor_outputs for fees.
HTLC fees increase (larger weight), and the fee paid by the opener
has to include the anchor outputs (i.e. 660 sats).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-14 11:51:14 +09:30
Rusty Russell
edf60b4f9e openingd: remember if we negotiated option_anchor_outputs, to put in the db.
And hand it through to channeld just like option_static_remotekey.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-14 11:51:14 +09:30
Rusty Russell
cf25ca11b8 sendpsbt: annotate transactions in wallet.
This is what txsend does, only we have a psbt so we have
to change the db interface to take a wally_tx.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-13 12:37:02 -05:00
Christian Decker
639eaaf2b4 json-rpc: Do not return a null amount_msat to listpays
Suggested-by: Rusty Russell <@rustyrussell>
Changelog-Deprecated: JSON-RPC: `listsendpays` will no longer add `null` if we don't know the `amount_msat` for a payment.
2020-08-11 08:43:18 +09:30
Rusty Russell
7a7a849fc2 lightningd: allow plugin commands and options to mark themselves deprecated.
This lets us handle it the same way we handle builtin commands, and also
lets us warn if they use deprecated apis and allow-deprecated-apis=false.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Plugins: can now mark their options and commands deprecated.
2020-08-11 08:43:18 +09:30
Rusty Russell
151bc47583 JSON-RPC: getmanifest passes allow-deprecated-apis flag.
This allows plugins to choose how to present things in getmanifest.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: plugins: `getmanifest` may now include "allow-deprecated-apis" boolean flag.
Changelog-Deprecated: plugins: `getmanifest` without any parameters; plugins should accept any parameters for future use.
2020-08-11 08:43:18 +09:30
Rusty Russell
930e294801 help: better handing of deprecated commands
If allow-deprecated-apis=false, don't mention them at all (we already
disallow calling them in that case).  Otherwise, note that they're
deprecated in the help msg.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-11 08:43:18 +09:30
Rusty Russell
5e0b03fba9 common: hoist param_bitcoin_address where plugins can use it.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-11 08:39:24 +09:30
Vincenzo Palazzo
1521c29fcf listpays mod 1: add destination inside the response when bolt11 is null
Changelog-Added: JSON-RPC: `listpays` now lists the `destination` if it was provided (e.g., via the `pay` plugin or `keysend` plugin)
2020-08-09 16:03:03 +02:00
Rusty Russell
fa829f23db amount: add amount_msat_scale, amount_msat_ratio, amount_{msat,sat}_div
It's not all that rare to do these operations, and requiring annotations
for it is a little painful.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-06 09:36:47 +09:30
Rusty Russell
36d43b871f amount: add amount_msat and amount_sat initializers.
Generally, importing amounts needn't be checked, and it cuts down on
the warnings we get.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-06 09:36:47 +09:30
ZmnSCPxj jxPCSnmZ
1ded3fc52f lightningd/plugin.c: Add a --dev-builtin-plugins-unimportant for developers who want to mess around with the builtin plugins. 2020-08-04 13:27:51 -05:00
ZmnSCPxj jxPCSnmZ
4ca2e49812 lightningd/plugin.c: Make builtin plugins important.
Changelog-Changed: plugin: Builtin plugins are now marked as important, and if they crash, will cause C-lightning to stop as well.
2020-08-04 13:27:51 -05:00
ZmnSCPxj jxPCSnmZ
a847487bbe lightningd/plugin.c: Add important plugins, which if they terminate, lightningd also terminates.
Changelog-Added: New option `--important-plugin` loads a plugin is so important that if it dies, `lightningd` will exit rather than continue.  You can still `--disable-plugin` it, however, which trumps `--important-plugin` and it will not be started at all.
2020-08-04 13:27:51 -05:00
ZmnSCPxj jxPCSnmZ
50600dce95 lightningd/lightningd.c: Create API to exit lightningd with an exit code. 2020-08-04 13:27:51 -05:00