Commit Graph

8986 Commits

Author SHA1 Message Date
Michael Schmoock
6697ebcebb doc: update HACKING.md 2020-10-23 12:47:47 -05:00
Michael Schmoock
5b542195d3 doc: typo in lightning-setchannelfee.7
Changelog-None
2020-10-23 12:47:47 -05:00
Rusty Russell
1e5789d421 close: add notification for slow closes.
For compatibility, we only do this if `allow-deprecated-apis` is false
for now.  Otherwise scripts parsing should use `grep -v '^# '` or
start using `-N none`.

Changelog-Added: JSON-RPC: `close` now sends notifications for slow closes (if `allow-deprecated-apis`=false)
Changelog-Deprecated: cli: scripts should filter out '^# ' or use `-N none`, as commands will start returning notifications soon
Fixes: #3925
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-23 13:53:16 +10:30
Rusty Russell
f395404a10 lightningd: infrastructure for internal notifications.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-23 13:53:16 +10:30
Rusty Russell
44f6b8a746 pytest: add notifications to tests.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-23 13:53:16 +10:30
Rusty Russell
41290a436f lightning-cli: print notifications (with '# ' prefix) if received.
This can be suppressed with -N.

Note that we wull get an error with older lightningd, but we ignore it
anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: cli: print notifications and progress bars if commands provide them.
2020-10-23 13:53:16 +10:30
Rusty Russell
806f208295 pyln: handle (ignore) notifications, and add notify_msg to send them.
We also sanity check that response id matches our request.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: pyln: pyln.client handles and can send progress notifications.
2020-10-23 13:53:16 +10:30
Rusty Russell
572c849873 libplugin: ignore incoming notifications.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-23 13:53:16 +10:30
Rusty Russell
127811757f libplugin: support for sending notifications.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: libplugin: routines to send notification updates and progress.
2020-10-23 13:53:16 +10:30
Rusty Russell
9f687d60d9 lightningd: forward notifications from plugins if enabled.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-23 13:53:16 +10:30
Rusty Russell
c732d8707a JSON-RPC: notifications command.
This lets callers enable notifications; we won't send any if they don't.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: JSON-RPC: `notifications` command to enable notifications.
2020-10-23 13:53:16 +10:30
Rusty Russell
a95205c25c common/json_stream: add generic double-cr helper.
And make caller of json_stream_forward_change_id use it, since
we're going to reuse that.

Also call json_out_finished here, so next object doesn't have a ","
prepended.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-23 13:53:16 +10:30
Rusty Russell
d970addd4b ccan: update to latest version, get json_out_finished update.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-23 13:53:16 +10:30
Rusty Russell
becd4fe576 common: add routines for log level names.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-23 13:53:16 +10:30
Rusty Russell
0dec593aa8 lightningd: clean up close code now force is always true.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-23 13:53:16 +10:30
Sergi Delgado Segura
9de5f438c4 pyln: Adds type annotations to zbase32 functions 2020-10-22 13:56:57 +02:00
Sergi Delgado Segura
3779c0b6ee pyln: Moves old zbase32 test from primitives 2020-10-22 13:56:57 +02:00
Sergi Delgado Segura
f497b90ee8 pyln: Adds tests to zbase32 2020-10-22 13:56:57 +02:00
Sergi Delgado Segura
26f651f71f pyln: Adds one-liner docs to zbase32 functions 2020-10-22 13:56:57 +02:00
Sergi Delgado Segura
1da29305fc pyln: Improve zbase32 encoding / decoding
- Adds bitarray filling so mesages of any length can be encoded, instead of forcing the encoding to be of messages with length multiple of 5.
- Adds checks for encoding / decoding and raises expections if the inputs are not as expected.
- Flags functions that are supposed to be internal as "private".
2020-10-22 13:56:57 +02:00
Rusty Russell
375040a3d9 Makefile: fix dependencies for check-pyln-%
$(BIN_PROGRAMS) is defined afterwards, leading to the problem that
lightningd doens't get rebuilt (in fact, it was running my installed
lightningd instead!).

It also needs $(PLUGINS) and the subdaemons, so hoist them all.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-22 12:14:34 +10:30
Rusty Russell
a8177e9013 Makefile: make check-includes check all the non-generated files.
Note that check-whitespace and check-bolt already do this, so we
can eliminate redundant lines in common/Makefile and bitcoin/Makefile.

We also include the plugin headers in ALL_C_HEADERS so they get
checked.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-22 12:14:34 +10:30
Rusty Russell
5e865ce42b Makefile: unify generated files definition.
We change gen_ to _gen, because filtering on gen_% doesn't work if they're
in subdirectories :(

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-10-22 12:14:34 +10:30
Antoine Poinsot
49dcb90b4e doc: add a document about our fuzz testing integration
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot
85aa53c778 fuzz: add a runner Python script for fuzz targets
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot
b3594690d9 configure: allow to run with UB sanitizer
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot
c149a88bbe fuzz: add a fuzz target for initial_channel
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot
31e50c9c25 contrib: remove ASAN suppressions
My bad, i comitted them as part of the LSAN suppressions while this data
race could have and had been fixed.

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot
1aa94ccd9e fuzz: add a fuzz target for the close tx
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot
33c7eccb3b fuzz: add a fuzz target for channel_id derivation
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot
f0061d0606 fuzz: add a fuzz target for common/bip32
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot
a89a018475 fuzz: add a fuzz target for common/bigsize
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot
5d987f2dec fuzz: add a target for base32/64 encoding
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot
07bb931fed common: remove b64_decode
We don't use it, and it's buggy (will always return NULL)

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot
63b1ae54a1 fuzz: add a target for common/bech32
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot
ae4dc231c1 amount: correctly parse amount strings we generate
This:
	- Allows `.*btc` amounts (without post-decimal)
	- Avoids creating decimals when amount is 0 btc
	- Corrects our handling of the suffixes (memeqstr would
	  sometimes return false because of null-termination)

Changelog-Fixed: We are now able to parse any amount string (XXXmsat, XX.XXXbtc, ..) we create.

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot
2f90c45454 fuzz: add a fuzz target for amounts parsing
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
Antoine Poinsot
62b54d0125 build: introduce a fuzzing mode
This adds a new configuration, --enable-fuzzing (which is more than
welcome to be coupled with --enable-address-sanitizer), to pass the
fuzzer sanitizer argument when compiling objects. This allows libfuzzer
to actually be able "to fuzz" by detecting coverage and be smart when
mutating inputs.

As libfuzzer brings its own ~~fees~~ main(), we compile objects with
fsanitize=fuzzer-no-link, and special-case the linkage of the fuzz
targets.

A "lib" is added to abstract out the interface to the fuzzing tool used.
This allow us to use the same targets to fuzz using AFL, hongfuzz or w/e
by adding their entrypoints into libfuzz. (h/t to practicalswift who
introduced this for bitcoin-core, which i mimiced)

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-10-21 19:34:39 +02:00
niftynei
ede5f5be3c mfc: blackbox test for commitment vs funding tx feerates
Liquid is excluded because the mempool entry doesn't contain a 'weight'
field.
2020-10-21 11:24:16 +10:30
niftynei
26bc4f5239 tx,bugfix: correct signature length estimate
71-bytes for a signature already includes the sighash byte.

 2-bytes	30 44 (DER- prefix thing)
34-bytes	02 20 6e29c8df67fffdda1613cef1413eb1a9ef3627f1fc5e4d910837274eafcc7b2a (r)
34-bytes	02 20 4b8563d79b92fdd830a546862439f80b24132d09318af2c7220c791067067e29 (s)
 1-byte		01 (sighash)
==
71-bytes
2020-10-21 11:24:16 +10:30
niftynei
f8c4cc73ae mfc: use declared constants
reduce, reuse, recycle for a greener world
2020-10-21 11:24:16 +10:30
niftynei
e66937e012 mfc: Add a commitment_feerate optional parameter to multifundchannel
Technically there *are* two feerates that we need to know:
  - the feerate to use for the funding transaction, and
  - the feerate to tell our peer to use for our commitment txs/htlc txs

As written, `multifundchannel` uses the same feerate for both. This
optional parameter will allow us to differentiate between the two, which
will be exceedingly handy for anchor output worlds. ;)

FIXME: test this

Changelog-Added: JSON API: `multifundchannel` has a new optional argument, 'commitment_feerate', which can be used to differentiate between the funding feerate and the channel's initial commitment feerate
2020-10-21 11:24:16 +10:30
niftynei
ae825fff26 opening: use correct dust_limit for reserve floor
Fixes #4140

Reported-By: @PsySc0rpi0n
Changelog-Fixed: openingd now uses the correct dust limit for determining the allowable floor for a channel open (affects fundee only)
2020-10-21 11:18:31 +10:30
niftynei
1b3a9be416 df, channeld: cleanup how psbt signalling works
We used to send our tx_sigs before we got to channeld existing. We
changed how this worked so that multifundchannel could live, but failed
to clean up the logic of what "having a psbt around" means wrt channeld
and messaging our peer.

The general idea is that we want to send `tx_signatures` to our peer on
reconnect until they've sent us `funding_locked`.

Note that it's an error to
  - send `funding_locked` without having sent `tx_signatures`
  - send `tx_signatures` after sending `funding_locked`

We use the 'finalized' state of the peer's inputs/outputs to help signal
where we are in receiving their sigs -- but this doesn't work at all for
opens where the peer doesn't contribute inputs at all.

This isn't really a huge deal, but it does mean that if we receive a
peer's `tx_sigs` more than once (will happen for a reconnect before
`funding_locked`), then we'll issue a notification about receiving their
sigs multiple times. /shrug
2020-10-21 09:04:32 +10:30
niftynei
d535a27104 df, bugfix: wait til after we've saved the channel to do this
this cleans up `cmd` and we're not done with it yet (we need it for
saving the channel updates to the database)
2020-10-21 09:04:32 +10:30
niftynei
c6d4bd676f dual-open,openchannel_update: include the index of the funding output
This allows us to do correct reporting via multiopenchannel :)
2020-10-21 09:04:32 +10:30
niftynei
9d412718df psbt: save the index of the change on the 'parent'
Note that for removals, the index will be on the original; for
additions, the index will be on the new. Yes this is implicit.
2020-10-21 09:04:32 +10:30
niftynei
daa55d1221 df: add notification for receiving peer's funding tx sigs
This will allow us to build complex, multi-peer transactions, with
easeTM!

Changelog-Added: EXPERIMENTAL, Plugins: `openchannel_peer_sigs` notification, which contains a peer's signatures for the funding transaction (`opt_dual_fund`)
2020-10-21 09:04:32 +10:30
niftynei
f9aab50ee8 dual-fund: rework where we send our tx-sigs message, allow peers in
Prior to this patch update, we expected a client to call
`openchannel_signed` before checking for peer's tx-sigs messages on the
wire.

When moving to a 'multifundchannel' approach, we'll need to be able to
collect sigs from our peers before sending our tx_sigs message. There's
no strict ordering on when tx-sigs messages are sent/received, so this
is fine.

To do this, we go ahead and start up channeld as soon as
commitment_sigs are secured, so that we process incoming tx-sigs from
our peers as soon as we get them.
2020-10-21 09:04:32 +10:30
niftynei
c6ad4f9b20 channel.psbt: make non-const
We update it in the next patch, which technically breaks this contract.
So we shouldn't have the contract of const on this in the first place
then.
2020-10-21 09:04:32 +10:30