Commit Graph

697 Commits

Author SHA1 Message Date
Christian Decker
6dd14a2039 sphinx: Kill read_buffer with fire 🔥
Suggested-by: Rusty Russell <@rustyrussell>
Signed-off-by: Christian Decker <@cdecker>
2020-03-12 10:25:01 +10:30
Christian Decker
e79cda8c9a sphinx: Treat compressed onions as a standalone struct
Expands the interface to play with onions a bit more. Potentially a bit
slower due to allocations, but that's a small price to pay. It also allows us
to avoid serializing a compressed onion to `u8*` if we process it right away.
2020-03-12 10:25:01 +10:30
Christian Decker
fd37c5b672 sphinx: Expose the shared secret creation function 2020-03-12 10:25:01 +10:30
Christian Decker
49a3321d7e sphinx: Add functions to decompress
Also implements a way to decompress an onion using the devtools/onion tool

Changelog-Added: devtools: The `onion` tool can now generate, compress and decompress onions for rendez-vous routing
2020-03-12 10:25:01 +10:30
Christian Decker
59b6159e56 sphinx: Functions to enable RV mode and serialize compressed onions
We will later use these to generate RV compressed onions and to opt into the
rendezvous style generation.
2020-03-12 10:25:01 +10:30
Christian Decker
96dc0238ba sphinx: Working onion wrapping with filler cancellation 2020-03-12 10:25:01 +10:30
Rusty Russell
c92e782e22 wire: add fromwire_tal_arrn() helper.
Does the allocation and copying; this is useful because we can
avoid being fooled into doing giant allocations.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-03-09 16:04:56 +01:00
Christian Decker
959687bf6d onion: Pass the position and type of the failing TLV type out
We'll need this when returning an error to the sender.
2020-03-04 22:57:24 +01:00
Christian Decker
0ba6aba7d1 onion: Do not leak parsed TLV if it is invalid
We were freeing the payload but not the TLV from which we extracted the
payload.
2020-03-04 22:57:24 +01:00
ZmnSCPxj jxPCSnmZ
d9b2482415 lightningd/hsm_control.c: Implement getsharedsecret.
ChangeLog-Added: New `getsharedsecret` command, which lets you compute a shared secret with this node knowing only a public point. This implements the BOLT standard of hashing the ECDH point, and is incompatible with ECIES.
2020-02-28 14:45:50 +10:30
Rusty Russell
c51c6f9133 sphinx: use crypto_stream_chacha20_xor to generate stream and xor at once.
Slightly more efficient.  We still generate an overlong stream in a couple
of other places though.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-02-27 14:16:16 +10:30
Rusty Russell
2aad3ffcf8 common: tal_dup_talarr() helper.
This is a common thing to do, so create a macro.

Unfortunately, it still needs the type arg, because the paramter may
be const, and the return cannot be, and C doesn't have a general
"(-const)" cast.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-02-27 14:16:16 +10:30
Rusty Russell
684ed4231f common/wireaddr: don't include lightningd/lightningd.
common should not include specific per-daemon files.  Turns out this
caused a lot of indirect includes to be exposed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-02-27 14:16:16 +10:30
Rusty Russell
73ad9b5c0a common: avoid locale dependent strtod(3)
Replace `json_to_double()` (which uses `strtod(3)`) with our own
floating-point parsing function `json_to_millionths()` that
specifically expects to receive such a number that can fit in a
64 bit integer after being multiplied by 1 million.

The main piece of the code in this patch comes from
https://github.com/ElementsProject/lightning/pull/3535#discussion_r381041419

Changelog-None
2020-02-27 09:07:04 +10:30
Vasil Dimov
89ceb273f5 wire: remove towire_double()
Before this patch we used to send `double`s over the wire by just
copying them. This is not portable because the internal represenation
of a `double` is implementation specific.

Instead of this, multiply any floating-point numbers that come from
the outside (e.g. JSONs) by 1 million and round them to integers when
handling them.

* Introduce a new param_millionths() that expects a floating-point
  number and returns it multipled by 1000000 as an integer.

* Replace param_double() and param_percent() with param_millionths()

* Previously the riskfactor would be allowed to be negative, which must
  have been unintentional. This patch changes that to require a
  non-negative number.

Changelog-None
2020-02-27 09:07:04 +10:30
Vasil Dimov
6b7db1ea7c common: remove unused json_add_double()
Changelog-None
2020-02-27 09:07:04 +10:30
Rusty Russell
ed839bfda0 channeld: get the onionreply back from lightningd for failed htlcs.
Instead of making it ourselves, lightningd does it.  Now we only have
two cases of failed htlcs: completely malformed (BADONION), and with
an already-wrapped onion reply to send.

This makes channeld's job much simpler.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-02-25 11:12:12 +10:30
darosior
cd15cec2f3 txprepare: don't crash if we are passed unconfirmed utxos
Changelog-added: `txprepare` doesn't crash lightningd anymore if you pass unconfirmed utxos
2020-02-21 17:52:02 +01:00
darosior
70a79e3998 plugins/bcli: a new plugin for gathering Bitcoin data
Most is taken from lightningd/bitcoind and adapted. This currently
exposes 5 commands:
- `getchaininfo`, currently called at startup to check the network and
  whether we are on IBD.
- `getrawblockbyheight`, which basically does the `getblockhash` +
  `getblock` trick.
- `getfeerate`
- `sendrawtransaction`
- `getutxout`, used to gather infos about an output and currently used by
  `getfilteredblock` in `lightningd/bitcoind`.
2020-02-12 11:45:07 +10:30
Rusty Russell
30580731a6 Minor fixups on PR #3477
Feedback from @niftynei and me; nothing major, but avoids
another round-trip.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-02-11 13:53:31 +10:30
Christian Decker
9b976da3bc features: Add featurebits_or helper to combine two featurebitstrings
We will be doing this when collecting featurebits from the plugins, so make
this a reusable function.
2020-02-11 13:53:31 +10:30
Christian Decker
9521549c49 elements: Fix missing witness script entry for fee outputs
Fixes #3487
2020-02-08 14:26:12 +01:00
Rusty Russell
f376a9c24c common: use tabs everywhere.
Didn't generally fixup inside comments and the bech32 code: reformatting that
is just anti-social.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-02-05 14:19:12 +01:00
Rusty Russell
5d2fdfe66b common: add check that pico-valued invoices are round numbers.
Otherwise you can ask for a sub-millisatoshi amount, which is dumb and
violates the spec.

See-also: https://github.com/lightningnetwork/lightning-rfc/pull/736
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: We now reject invoices which ask for sub-millisatoshi amounts
2020-02-05 14:19:12 +01:00
Rusty Russell
3e9d4de02e common: add testing for proposed bolt11 additional tests.
See-also: https://github.com/lightningnetwork/lightning-rfc/pull/736
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-02-05 14:19:12 +01:00
Rusty Russell
77c867d95f common: fix bolt11 parsing of unknown f versions.
As tested in next test.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-02-05 14:19:12 +01:00
Rusty Russell
d8e933aac7 common: add pico-BTC parsing test from proposed BOLT.
See-also: https://github.com/lightningnetwork/lightning-rfc/pull/699
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-02-05 14:19:12 +01:00
Rusty Russell
7028493d01 common: implement route comparison in run-bolt11.
This next test will have route hints.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-02-05 14:19:12 +01:00
darosior
98e8aac75f libplugin: use ccan/io for response to plugin commands
This pass to json_stream helpers for commands outputs, but keeps
compatibility with existing plugins which use jout as of now, by not
starting/closing the "result"/"error" objects.
2020-02-04 13:24:32 +10:30
darosior
e6f56765ca common/json: add a json helper for the new errcode type 2020-02-04 13:24:32 +10:30
darosior
3510c29e5d common: move json_stream helpers to common/json
Now that we have json_stream in common/, we can move all the related
helpers from lightningd/json to common/json. This way everyone can
benefit of them (including libplugin, the plugins themselves,
potentially lightning-cli), not lightningd alone!

Note that the Makefile of the common/test/ had to be modified, because
the new helpers make use of common/wireaddr... Which turns out to
\#include <lightingd/lightningd.h> ! So we couldnt just include the .c
and add mocks if we redefined some structs (hello run-param).
2020-02-04 13:24:32 +10:30
darosior
d35387f489 common: move json_stream from lightningd/ to common/
It's not lightningd-specific and we are going to need it for libplugin. The only
drawback is the log_io removal in json_stream_output_write()..
2020-02-04 13:24:32 +10:30
Ken Sedgwick
5c8f881a75
hsmd: Added fields to hsm_sign_remote_commitment_tx to allow complete validation.
Changelog-Added: hsmd: Added fields to hsm_sign_remote_commitment_tx to allow complete validation by signing daemon.
2020-02-04 10:40:43 +10:30
darosior
273029f244 wallet: set nLockTime to the tip for withdrawal transactions
This sets the nLockTime to the tip (and accordingly each input's nSequence to
0xfffffffe) for withdrawal transactions.

Even if the anti fee-sniping argument might not be valid until some time yet,
this makes our regular wallet transactions far less distinguishable from
bitcoind's ones since it now defaults to using native Segwit transactions
(like us). Moreover other wallets are likely to implement this (if they
haven't already).

Changelog-Added: wallet: withdrawal transactions now sets nlocktime to the current tip.
2020-02-03 00:45:27 +00:00
Wladimir J. van der Laan
e4c6fd89b7 Add missing extern qualifiers for gcc 10
GCC 10 defaults to `-fno-common`. no longer automatically sharing
global variable definitions, which makes it important to define
them in only one place (otherwise there will be duplicate definition
errors). Add `extern` qualifiers where (I think) is the best place for
them.
2020-02-02 12:59:17 +10:30
Rusty Russell
8055389b90 common: remove version qualifiers from BOLT11 now changes are merged.
We also update since the merged version sets feature bit 9 (as it's
supposed to now that we tied that to payment_secret).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-01-31 06:07:56 +00:00
Rusty Russell
3cf8443285 Makefile: update bolt markers which have been since merged.
We tag them with specific versions when they're experimental,
but do a poor job of cleaning them up (and thus ensuring they're
checked!) afterwards.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-01-31 06:07:56 +00:00
Rusty Russell
f373cd45be Makefile: update BOLTVERSION to latest master.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-01-31 06:07:56 +00:00
Rusty Russell
ff1e1dd487 Makefile: update BOLTVERSION to flatten features.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-01-31 06:07:56 +00:00
Vasil Dimov
55173a56b7 Use dedicated type for error codes
Before this patch we used `int` for error codes. The problem with
`int` is that we try to pass it to/from wire and the size of `int` is
not defined by the standard. So a sender with 4-byte `int` would write
4 bytes to the wire and a receiver with 2-byte `int` (for example) would
read just 2 bytes from the wire.

To resolve this:

* Introduce an error code type with a known size:
  `typedef s32 errcode_t`.

* Change all error code macros to constants of type `errcode_t`.
  Constants also play better with gdb - it would visualize the name of
  the constant instead of the numeric value.

* Change all functions that take error codes to take the new type
  `errcode_t` instead of `int`.

* Introduce towire / fromwire functions to send / receive the newly added
  type `errcode_t` and use it instead of `towire_int()`.

In addition:

* Remove the now unneeded `towire_int()`.

* Replace a hardcoded error code `-2` with a new constant
  `INVOICE_EXPIRED_DURING_WAIT` (903).

Changelog-Changed: The waitinvoice command would now return error code 903 to designate that the invoice expired during wait, instead of the previous -2
2020-01-31 06:02:47 +00:00
Rusty Russell
c11212bb52 pytest: test that we handle WIRE_UPDATE_FAIL_MALFORMED_HTLC correctly.
We could use sendonion to do this, but it actually takes a different path through
pay, and I wanted to test all of it, so I made a new dev flag.

We currently get upset with the response:

	lightningd/pay.c:556: payment_failed: Assertion `!hout->failcode' failed.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-01-29 21:15:25 +01:00
Christian Decker
9fd84169bb common: Add an assertion for custommsgs in gossip handler
This is mainly meant as a marker so that we can later remove the code if we
decide to make the handling of custommsgs a non-developer option. It marks the
place that we would otherwise handle what in dev-mode is a custommsg.
2020-01-28 23:50:52 +01:00
ZmnSCPxj jxPCSnmZ
67590fc6be lightningd/invoice.c: Add timeout parameter to waitanyinvoice.
Fixes: #3192

Changelog-Added: `waitanyinvoice` now supports a `timeout` parameter, which when set will cause the command to fail when the timeout is reached; can set this to 0 to fail immediately if no new invoice has been paid yet.
2020-01-28 14:07:52 +01:00
lisa neigut
d36af2c340 txprepare: make output finding map for withdraw_tx variable len
The number of outputs got updated, but the map used to calculate the
change output's location did not (still assumes only one output). This
patch fixes this to make the output map a variable size.

Changelog-Fixed: JSON API: `txprepare` no longer crashes when more than two outputs are specified
2020-01-27 22:59:41 +01:00
Christian Decker
9038364c63 sphinx: Actually use the pad stream to generate the packet
We flipped two buffers and were not actually using the chacha20 stream.
2020-01-27 22:48:42 +01:00
Rusty Russell
262e4c840f sphinx: use struct secret for shared secret.
Generally I prefer structures over u8, since the size is enforced at
runtime; and in several places we were doing conversions as the code
using Sphinx does treat struct secret as type of the secret.

Note that passing an array is the same as passing the address, so
changing from 'u8 secret[32]' to 'struct secret secret' means various
'secret' parameters change to '&secret'.  Technically, '&secret' also
would have worked before, since '&' is a noop on array, but that's
always seemed a bit weird.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-01-24 10:01:44 +10:30
Rusty Russell
1099f6a5e1 common: use struct onionreply.
This makes it clear we're dealing with a message which is a wrapped error
reply (needing unwrap_onionreply), not an already-wrapped one.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-01-23 16:17:42 +10:30
Rusty Russell
aa6aad0131 common: add struct onionreply
I really want a type which means "I am a wrapped onion reply" as separate
from "I am a normal wire msg".  Currently both user u8 *, and I got very
confused trying to figure out where each one was an unwrapped error msg,
or where it still needed (un)wrapping.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-01-23 16:17:42 +10:30
Rusty Russell
aae5148206 common: remove onionreply type from sphinx.c
It's only mildly used, and I really to use onionreply in a more
generic sense in the next patch.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-01-23 16:17:42 +10:30
ZmnSCPxj jxPCSnmZ
44e8256338 common/json.c: Implement json_to_u32. 2020-01-21 22:23:21 +01:00