Commit Graph

427 Commits

Author SHA1 Message Date
Rusty Russell
a7c5a1f1d2 lightningd: implement receiving warnings.
This takes from the draft spec at https://github.com/lightningnetwork/lightning-rfc/pull/834

Note that if this draft does not get included, the peer will simply
ignore the warning message (we always close the connection afterwards
anyway).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: Protocol: we now report the new (draft) warning message.
2021-02-04 12:02:52 +10:30
Rusty Russell
06a54606a3 check-includes: allow redundant "config.h"
We should actually be including this (as it may define _GNU_SOURCE
etc) before any system headers.  But where we include <assert.h> we
often didn't, because check-includes would complain that the headers
included it too.

Weaken that check, and include config.h in C files before assert.h.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-02-04 12:02:36 +10:30
Rusty Russell
001b5d6416 offers: make it a runtime option.
The fetchinvoice and offers plugins disable themselves if the option
isn't enabled (it's enabled by default on EXPERIMENTAL_FEATURES).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: `experimental-offers` enables fetch, payment and creation of (early draft) offers.
2021-01-13 14:45:36 +01:00
Rusty Russell
a4bc3e5673 common/bolt12: include correct bolt12 wiregen header.
Don't include exp directly, use an ifdef in common/bolt12
(like we do for peer and onion wiregen files).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-13 14:45:36 +01:00
Rusty Russell
6dab04ebd5 tools/generate-wire.py: include digits in #ifndef idempotent header.
Otherwise check complains that it's '#ifndef LIGHTNING_WIRE_BOLT_WIREGEN_H':

wire/bolt12_wiregen.h seems to be missing the expected include guard:
  #ifndef LIGHTNING_WIRE_BOLT12_WIREGEN_H
  #define LIGHTNING_WIRE_BOLT12_WIREGEN_H
  ...
  #endif /* LIGHTNING_WIRE_BOLT12_WIREGEN_H */

make: *** [Makefile:458: check-includes] Error 1

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-13 14:45:36 +01:00
Rusty Russell
5a483ddd8b wire: Apply onion mesages spec patch even when not experimental.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-13 14:45:36 +01:00
Rusty Russell
c3200b07de wire: add bolt12_wire.csv
For some reason, an old version was checked in.  We don't currently
use it (since bolt12 support depends on EXPERIMENTAL_FEATURES, which
uses bolt12_exp_wire.csv, which is generated and not checked in).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-13 14:45:36 +01:00
Rusty Russell
7af63eb0a7 wire/Makefile: refactor to allow non-experimental spec patches.
And rename the current patches to all be _exp_*.patch.  These
are applied after the basic patches.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>


Header from folded patch 'fix.patch':
2021-01-13 14:45:36 +01:00
niftynei
2f3acb91fc df: actually send a temp-id to the peer in open_channel2 WIP
We actually need a channel_id in openchannel so we can properly
attribute any errors that we get back
2021-01-10 13:44:04 +01:00
niftynei
924aaf180e temp_channel_id: move to common
We actually do need this for dualopend's, specifically for returning
errors to open_channel2
2021-01-10 13:44:04 +01:00
Rusty Russell
23af9d4972 onion_message: support variable-length onion messages.
Updated to the BOLT, and a few tweaks, and we can send giant onion_messages.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-09 12:45:31 +01:00
Rusty Russell
9d57612415 gossipd, channeld: handle onion messages in gossipd so we don't need a channel.
The previous onion_message code required a confirmed, not-shutting-down
channel, not just a connection.  That's overkill; plus before widespread
adoption we will want to connect directly as a last resort.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-01-09 14:34:35 +10:30
Rusty Russell
43b71de897 lightningd: low-level createinvoicerequest API (EXPERIMENTAL_FEATURES)
This is similar to the createinvoice API, except we don't need to save
invoice requests in the database.  We may, however, have to look up
payment_key for recurring invoice requests, and sign the message with
the payment_key.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-23 14:24:39 +01:00
Rusty Russell
353e4cf3c2 wire/Makefile: make generated wire files depend on Makefile.
That way they rebuild when we change flags in Makefile.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-23 14:24:39 +01:00
Rusty Russell
3776af4a35 common/sphinx: make TOTAL_PACKET_SIZE a macro.
This paves the way for using it on different-sized onions.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-11 15:51:24 +01:00
Rusty Russell
2cd93597cc wire.c: import bolt 12 offers CSV.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-09 16:40:13 +10:30
Rusty Russell
a11edebb7c utf8: handle UTF-8 arrays.
BOLT 12 introduces this as a new fundamental type, which lets us easily
validate them.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-09 16:40:13 +10:30
Rusty Russell
ff8830876d wire/tlvstream: add tlv_make_fields helper to populate ->fields array.
This is vital for calculating merkle trees; I previously used
towire+fromwire to get this!

Requires generation change so we can magic the ARRAY_SIZE var (the C
pre-processor can't uppercase things).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-04 20:16:54 -06:00
Rusty Russell
5c167d16ab tools/generate-wire.py: use helpers.
This was terrible cut & paste.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-04 20:16:54 -06:00
Rusty Russell
dc83e64003 tools/generate-wire: don't use void * pointers for tlv fromwire.
And fix up the one place which got it wrong.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-04 20:16:54 -06:00
Rusty Russell
ca2bd98082 unittest: use common_setup / common_shutdown almost everywhere.
Avoids much cut & paste.  Some tests don't need any of it, but most
want at least some of this infrastructure.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-12-02 12:55:09 +10:30
YOSHIDA Masanori
9f893e3f6d wire: remove declaration of towire_tlvs since its implementation is already removed
Signed-off-by: YOSHIDA Masanori <masanori.yoshida@gmail.com>
Changelog-None
2020-11-02 10:59:05 -06:00
niftynei
97fd18f0b5 df: incorporate a few spec changes -- serial_id is now 64-bits
And we pass 3-params for feerate so it's a 'pick a range' conversation.
2020-10-27 19:52:05 -05:00
niftynei
82c0b48215 wires: towire/fromwire for wally_tx
We're eventually moving away from 'bitcoin_tx
2020-10-20 12:50:31 +10:30
Matt Whitlock
eab14768a8 update SHA256STAMPs using sorted dependencies 2020-09-17 10:23:40 +09:30
Michael Schmoock
dd60332d91 chore: adds experimental wiregen to gitignore
Changelog-None
2020-09-09 20:46:46 +09:30
niftynei
b44e36b99e spec-update: get rid of max-witness-len
We can use a fixed value and close the channel if they don't cover their
amount; this wasn't really helping with anything other than setting a
floor for an expected feerate
2020-09-09 19:54:20 +09:30
niftynei
05a6a11860 dual-fund: add interactive dual-funding wires
imported from ??? TODO: FIXME
niftynei/lightning-rfc:nifty/interactive-dual-funding
2020-09-09 19:54:20 +09:30
Rusty Russell
0e805427dc tools/generate-wire.py: strip trailing whitespace on lines, fix bolt quotes.
There's a lot of it, and it means we can't `make check-source` on
these files.

Also bring bolt quotes up-to-date.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-09 16:23:58 +09:30
Rusty Russell
c67f5b301e wire: fix up regeneration
We didn't rename the _csv to .csv, which patterns expect if you're
regenerating (e.g. if you switch on --enable-experimental-features
then switch it off again!)

Also remove unused experimental targets.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-09 16:23:58 +09:30
Rusty Russell
c34c055d82 Makefile: use completely separate spec-derived files for EXPERIMENTAL_FEATURES
This avoids overwriting the ones in git, and generally makes things neater.

We have convenience headers wire/peer_wire.h and wire/onion_wire.h to
avoid most #ifdefs: simply include those.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-08 09:42:00 +09:30
Rusty Russell
d8e8426b52 Makefile: remove EXPERIMENTAL_FEATURES marker from generated files
We're going to make experimental versions of these completely separate files.

Also remove the dependency on the Makefile itself: it simply causes
unnecessary churn.  We can always force-rebuild when we change a rule.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-08 09:42:00 +09:30
Rusty Russell
82f2f43425 Generated files: more merge fallout.
We changed the generation templates!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-09-02 11:10:21 +09:30
Rusty Russell
3c6af3efb4 Makefile: commit and preserve all the wiregenerated files.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Build: we no longer require extra Python modules to build.
2020-08-31 21:33:26 -05:00
Rusty Russell
1746406e41 Makefile: normalize all the Makefiles
We create ALL_PROGRAMS, ALL_TEST_PROGRAMS, ALL_C_SOURCES and
ALL_C_HEADERS.  Then the toplevel Makefile knows which are
autogenerated (by wildcard), so it can have all the rules to clean
them or check the source as necessary.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-31 21:33:26 -05:00
Rusty Russell
8150d28575 Makefile: use generic rules to make spec-derived sources.
Now we use the same Makefile rules for all CSV->C generation.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-08-31 21:33:26 -05:00
Rusty Russell
055cfd17a8 wire: locale-independent patch order for EXPERIMENTAL_FEATURES patches
Also, remove fuzz caused by varint->bigsize change.

For some reason my build machine sorts patches into another order, and fails
to patch:

	patching file wire/gen_onion_wire_csv.104951
	Hunk #1 succeeded at 52 with fuzz 1 (offset -19 lines).
	patching file wire/gen_onion_wire_csv.104951
	Hunk #1 FAILED at 8.
	1 out of 1 hunk FAILED -- saving rejects to file wire/gen_onion_wire_csv.104951.rej
	make: *** [wire/Makefile:60: wire/gen_onion_wire_csv] Error 1

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-07-17 13:54:21 +02:00
Christian Decker
f6745682c3 tlvstream: Allow overwriting an already set value
This is necessary in the next commit to override the total_msat that is being
delivered to the destination.
2020-07-15 11:32:58 +02:00
Christian Decker
514c4044c8 tlv: Allow passing a raw pointer and a length to tlvstream_set_raw
Allows us to do fewer allocations, since the argument doesn't have to be tal
allocated itself.

Suggested-by: Rusty Russell <@rustyrussell>
2020-07-13 14:19:16 +02:00
niftynei
77946bd9fe fromwire: return NULL if array empty
libwally's API requires us to pass in NULL pointers if the array size is
zero, so we update our array from wire-er to comply with this
requirement

[ Added fix to avoid tal_resize() of NULL -- RR ]
2020-07-01 19:50:02 -05:00
Christian Decker
cb9debc229 tlv: Add generic getters and setters for tlvstream
Trying to rework the TLV streams to have a more homogenous interface to work
with. This is by no means a complete implementation, just the groundwork that
is going to be used by the wire code generator to generate the specific
accessors, but it's enough so we can manipulate TLV streams in the onion and
later just switch to the generated ones.
2020-07-01 12:19:02 +02:00
Rusty Russell
93d04d08d0 wire: update to latest version of the spec.
The main change here is that the previously-optional open/accept
fields and reestablish fields are now compulsory (everyone was
including them anyway).  In fact, the open/accept is a TLV
because it was actually the same format.

For more details, see lightning-rfc/f068dd0d8dfa5ae75feedd99f269e23be4777381

Changelog-Removed: protocol: support for optioned form of reestablish messages now compulsory.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-06-23 18:49:15 +02:00
niftynei
24ecb3e2b9 Remove trailing line in fromwire.c 2020-05-21 18:45:07 +09:30
Rusty Russell
b920d4d21b wire: make fromwire_fail return non-const, use it more
It returns NULL, so you can simply `return fromwire_fail(...)`
if you want to return NULL in this case.  Use that more.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-20 11:36:13 +02:00
Rusty Russell
cfb320c972 wire: move remaining bitcoin functions out to bitcoin/ files.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-18 14:51:12 +02:00
Rusty Russell
197d1bcef2 wire: move towire/fromwire_short_channel_id out to bitcoin/short_channel_id.c
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-18 14:51:12 +02:00
Rusty Russell
13356b75bf wire: move towire/fromwire_amount from wire/ into common/amount.c
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-18 14:51:12 +02:00
Rusty Russell
4dbfce6057 wire: move towire/fromwire_preimage out to bitcoin/preimage.c.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-18 14:51:12 +02:00
Rusty Russell
27220646c3 common/wire: move bitcoin-specific marshalling functions into bitcoin files.
We did this originally because these types are referred to in the bolts, and we
had no way of injecting the correct include lines into those.  Now we do, so
there's less excuse for this.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-18 14:51:12 +02:00
Rusty Russell
f77d70d546 common/json: move rest of bitcoin/lightning-specific json functions to json_helpers.
This dramatically reduces the linking requirements of lightning-cli.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-18 14:51:12 +02:00
Rusty Russell
fda5f0b427 common/channel_id: move channel_id into its own file.
The definition was in wire/wire.h, and helper functions in fromwire.c!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-18 14:51:12 +02:00
Rusty Russell
f02c464c2e wire/Makefile: list wire/tlvstream.c in WIRE_SRC.
Otherwise, it doesn't depend on anything (it's compiled becasue
plugins/keysend explicitly lists wire/tlvstream.o as a dependency).

This made me miss a compile break.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-18 14:51:12 +02:00
lisa neigut
15e4e922c9 wire: serialize the amounts for a bitcoin tx over the wire
we'll need this for calculating fees etc in onchaind
2020-05-12 15:46:58 +09:30
Rusty Russell
b0c9059602 tools/generate-wire: no more lonely messages!
When we have only a single member in a TLV (e.g. an optional u64),
wrapping it in a struct is awkward.  This changes it to directly
access those fields.

This is not only more elegant (60 fewer lines), it would also be
more cache friendly.  That's right: cache hot singles!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-05-06 14:56:09 -05:00
lisa neigut
261a46449c wire: suppress printout from patch command
We were using patch's '--silent' flag, but that broke on the busybox
implementation of `patch`, since they don't support it. Instead, we use
the universally supported "pipe to /dev/null" approach

Suggested-By: @rustyrussell
Changelog-None
2020-05-04 10:21:07 +09:30
Christian Decker
1b32cc1c73 wire: Add a function to serialize a raw set of TLV fields
The generated wrappers will ignore the raw fields and will only consider the
shortcut fields. This function takes the raw fields and serializes them
instead.
2020-04-16 18:03:35 +09:30
Rusty Russell
b29d1ed3ff channeld: support HTLCs with blinding (EXPERIMENTAL_FEATURES)
Note that it's channeld which calculates the shared secret, too.  This
minimizes the work that lightningd has to do, at cost of passing this
through.

We also don't yet save the blinding field(s) to the database.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-14 12:51:18 +09:30
Rusty Russell
ae80787d1e EXPERIMENTAL: new error code for use when HTLC is blinded.
Everyone should return this for any HTLC error when they're handed a blinding
alongside the HTLC.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-14 12:51:18 +09:30
Rusty Russell
f33b39075b EXPERIMENTAL: onion TLV field for enctlv for blinding.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-14 12:51:18 +09:30
Rusty Russell
990a3ec92e EXPERIMENTAL: protocol TLV field for update_add_htlc to support blinding.
All NULL for now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-14 12:51:18 +09:30
Rusty Russell
f76ab93df6 EXPERMENTAL_FEATURES: Import onion message types.
This tracks https://github.com/lightningnetwork/lightning-rfc/pull/759

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-04-02 14:32:38 +10:30
Rusty Russell
c92e782e22 wire: add fromwire_tal_arrn() helper.
Does the allocation and copying; this is useful because we can
avoid being fooled into doing giant allocations.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-03-09 16:04:56 +01:00
Rusty Russell
2aad3ffcf8 common: tal_dup_talarr() helper.
This is a common thing to do, so create a macro.

Unfortunately, it still needs the type arg, because the paramter may
be const, and the return cannot be, and C doesn't have a general
"(-const)" cast.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2020-02-27 14:16:16 +10:30
Vasil Dimov
89ceb273f5 wire: remove towire_double()
Before this patch we used to send `double`s over the wire by just
copying them. This is not portable because the internal represenation
of a `double` is implementation specific.

Instead of this, multiply any floating-point numbers that come from
the outside (e.g. JSONs) by 1 million and round them to integers when
handling them.

* Introduce a new param_millionths() that expects a floating-point
  number and returns it multipled by 1000000 as an integer.

* Replace param_double() and param_percent() with param_millionths()

* Previously the riskfactor would be allowed to be negative, which must
  have been unintentional. This patch changes that to require a
  non-negative number.

Changelog-None
2020-02-27 09:07:04 +10:30
Ken Sedgwick
5c8f881a75
hsmd: Added fields to hsm_sign_remote_commitment_tx to allow complete validation.
Changelog-Added: hsmd: Added fields to hsm_sign_remote_commitment_tx to allow complete validation by signing daemon.
2020-02-04 10:40:43 +10:30
Wladimir J. van der Laan
e4c6fd89b7 Add missing extern qualifiers for gcc 10
GCC 10 defaults to `-fno-common`. no longer automatically sharing
global variable definitions, which makes it important to define
them in only one place (otherwise there will be duplicate definition
errors). Add `extern` qualifiers where (I think) is the best place for
them.
2020-02-02 12:59:17 +10:30
Vasil Dimov
55173a56b7 Use dedicated type for error codes
Before this patch we used `int` for error codes. The problem with
`int` is that we try to pass it to/from wire and the size of `int` is
not defined by the standard. So a sender with 4-byte `int` would write
4 bytes to the wire and a receiver with 2-byte `int` (for example) would
read just 2 bytes from the wire.

To resolve this:

* Introduce an error code type with a known size:
  `typedef s32 errcode_t`.

* Change all error code macros to constants of type `errcode_t`.
  Constants also play better with gdb - it would visualize the name of
  the constant instead of the numeric value.

* Change all functions that take error codes to take the new type
  `errcode_t` instead of `int`.

* Introduce towire / fromwire functions to send / receive the newly added
  type `errcode_t` and use it instead of `towire_int()`.

In addition:

* Remove the now unneeded `towire_int()`.

* Replace a hardcoded error code `-2` with a new constant
  `INVOICE_EXPIRED_DURING_WAIT` (903).

Changelog-Changed: The waitinvoice command would now return error code 903 to designate that the invoice expired during wait, instead of the previous -2
2020-01-31 06:02:47 +00:00
Christian Decker
a8fa55d275 wire: Add common messages that are independent of daemons
These messages may be exchanged between the master and any daemon. For now
these are just the daemons that a peer may be attached to at any time since
the first example of this is the custommsg infrastructure.
2020-01-28 23:50:52 +01:00
Vasil Dimov
fb7c006187 wire: add towire_int() and use it in connectd
Add towire_int() and fromwire_int() functions to "(de)serialize"
"int". This will only work as long as both the caller of towire_int()
and the caller of fromwire_int() use the same in-memory representation
of signed integers and have the same sizeof(int).

Changelog-None
2020-01-21 16:59:18 +01:00
lisa neigut
e1ef902db4 wire-make: remove --silent flag from patch command
not supported on all architectures
2020-01-18 19:37:46 +08:00
Rusty Russell
b315a29202 wire: perform corruption test on init_msg.
Fixes: #3301
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-13 16:36:40 +01:00
Rusty Russell
839909d2cf Protocol: make var_onion, payment_secret and basic_mpp non-EXPERIMENTAL.
Thanks to @t-bast, who made this possible by interop testing with Eclair!

Changelog-Added: Protocol: can now send and receive TLV-style onion messages.
Changelog-Added: Protocol: can now send and receive BOLT11 payment_secrets.
Changelog-Added: Protocol: can now receive basic multi-part payments.
Changelog-Added: RPC: low-level commands sendpay and waitsendpay can now be used to manually send multi-part payments.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-12-12 22:16:03 +01:00
Christian Decker
db92c2ac5e tlv: Remove unused TLV deserialization function 2019-12-03 00:37:15 +00:00
Christian Decker
626675c83c tlv: Migrate tlv serialization to typesafe function 2019-12-03 00:37:15 +00:00
Christian Decker
5a78671d9f wire: Remove unused fromwire_tlvs
We are now using the typesafe variant everywhere.
2019-12-03 00:37:15 +00:00
Christian Decker
838001af4c tlv: Migrate run-tlvstream to typesafe helpers 2019-12-03 00:37:15 +00:00
darosior
3322048774 connectd: add network to init message
Changelog-Added: protocol: We now signal the network we are running on at init.
2019-11-29 21:17:08 +01:00
Rusty Russell
ebac3d2a85 spec: update to experimental BOLTs with secret/total_amount.
Also pulls in a new onion error (mpp_timeout).  We change our
route_step_decode_end() to always return the total_msat and optional
secret.

We check total_amount (to prohibit mpp), but we do nothing with
secret for now other than hand it to the htlc_accepted hook.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-11-24 23:33:17 +00:00
Christian Decker
2255024ead tlv: Add raw fields so we can store unknown fields as well 2019-11-22 04:40:25 +00:00
Christian Decker
7283efa5b5 elements: Add amount_asset to support more than just plain satoshis
Currently the only source for amount_asset is the value getter on a tx output,
and we don't hand it too far around (mainly ignoring it if it isn't the
chain's main currency). Eventually we could bubble them up to the wallet, use
them to select outputs or actually support assets in the channels.

Since we don't hand them around too widely I thought it was ok for them to be
pass-by-value rather than having to allocate them and pass them around by
reference. They're just 41 bytes currently so the overhead should be ok.

Signed-off-by: Christian Decker <@cdecker>
2019-10-03 04:32:57 +00:00
Christian Decker
c38afc5512 tx: Switch to amount_sat for fee computations
Suggested-by: Rusty Russell <@rustyrussell>
2019-10-03 04:32:57 +00:00
Christian Decker
d5f0c08a88 elements: Remove global is_elements variable in favor of chainparams
No need to keep duplicate globals.
2019-10-03 04:32:57 +00:00
Christian Decker
ef7a63d8f8 elements: Move from a global is_elements to a global chainparams
We now have a pointer to chainparams, that fails valgrind if we do anything
chain-specific before setting it.

Suggested-by: Rusty Russell <@rustyrussell>
2019-10-03 04:32:57 +00:00
Rusty Russell
15612d269a Make option_static_remotekey non-EXPERIMENTAL now it's in spec.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-30 01:50:44 +00:00
Rusty Russell
27790832a5 gossipd: gossip_queries_ex is not longer experimental.
The master spec has some typos which make it not parse, so I created
a PR and generated the CSV from that:

https://github.com/lightningnetwork/lightning-rfc/pull/673

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-22 01:17:11 +00:00
Rusty Russell
6eb838ddda wire: update csv to current bolt.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-22 01:17:11 +00:00
Rusty Russell
0da074cad3 BOLT: update bolt quotes now that TLVs are not experimental.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-22 01:17:11 +00:00
lisa neigut
aef5b30b81 wire: update how experimental csv patches are managed
We used to append new patches to a single file. This caused
some problems and is a lot harder to cleanup later.

This patch moves the experimental patches to their own, individual
patch files, that are named for the current BOLTVERSION, which they're
taken from.

Also moves the current patchfile over to a 'gossipqueries' one,
as it already exists.
2019-09-16 05:09:15 +00:00
Rusty Russell
226e2aee48 option_static_remotekey: update to latest draft.
531c8d7d9b

In this one, we always send my_current_per_commitment_point, though it's
ignored.  And we have our official feature numbers.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-10 16:18:25 -05:00
Rusty Russell
ded8eb7b31 wire/extracted_peer_experimental_csv: option_static_remotekey
Aka. BOLTVERSION=930a9b44076a8f25a8626b31b3d5a55c0888308c from
https://github.com/lightningnetwork/lightning-rfc/pull/642

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-10 16:18:25 -05:00
trueptolemy
cdcafdaf74 API: txprepare now support mutiple outputs 2019-09-05 16:05:36 -05:00
Rusty Russell
02b9b7f6e6 tests: update mocks for --enable-experimental-features builds.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-29 15:51:36 +02:00
Rusty Russell
189b2f1313 BOLT: update CSV to latest bolt version.
This removes the WIRE_FINAL_EXPIRY_TOO_SOON which leaked too much info,
and adds the blockheight to WIRE_INCORRECT_OR_UNKNOWN_PAYMENT_DETAILS.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-29 09:01:48 +02:00
Rusty Russell
855dff704c gossipd: test crc32 routines using test vectors from PR.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-27 12:35:25 +02:00
Rusty Russell
f6cf4bf62a spec: remove encoding byte from checksums.
Make the TLV element a simple array.  This is a bit neater, in fact, and
makes the test vectors in that 557 PR work.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-10 02:48:34 +00:00
Rusty Russell
d2030539e1 EXPERIMENTAL: pull in PR 557 (with minor fixes): range query support.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-10 02:48:34 +00:00
Rusty Russell
8774070a31 bolt: Update to latest bolt, including TLV onion format.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 17:32:48 +02:00
Rusty Russell
fafd5ed4b0 wire/test/run-tlvstream.c: use generated defs for test vectors.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 17:32:48 +02:00
Rusty Russell
00ca52ea07 bolt: update to aa33af0c4d7ae0180c04ef98e61af49c1f876a36
This introduces the TLV test vectors.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 17:32:48 +02:00
Rusty Russell
b17b4c3eb6 wire: include definition of struct node_id.
Next BOLT update puts it in a TLV, so we need the definition.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 17:32:48 +02:00
Rusty Russell
9b88fd4c60 bolt: update to 950b2f5481c2a4b57ef1102e2374543e81c4aa88
Just a simple field renaming which only alters comments,
though I updated variable names too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 17:32:48 +02:00
Rusty Russell
296cfe8d1b wire/tlvstream: suppress gcc -O3 warning about prev_type.
Use a pointer, so it's explicit and gcc is happy.  We avoid the
allocation by pointing it to another stack var.

./wire/tlvstream.c:81:22: error: ‘prev_type’ may be used uninitialized in this function [-Werror=maybe-uninitialized]

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-02 15:56:15 +02:00
Rusty Russell
cc70b9c4ec wire: use common/bigsize routines
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-31 23:25:59 +00:00
Rusty Russell
2241e25cb4 test: move run-bigsize into common to match common/bigsize.
And make it use bigsize accessors directly, not via fromwire/towire.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-31 23:25:59 +00:00
Rusty Russell
3fa375881a bigsize: make it a proper first-class type.
It doesn't belong in bitcoin, and should not be confused with varint_t.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-31 23:25:59 +00:00
Rusty Russell
c580225c38 wire: handle bigsize/varint fields.
They're currently called varint, but there's a proposal to call them all
bigsize.  Allow both for now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-27 21:19:57 -05:00
Rusty Russell
2e68b88be8 tools: fix tlv generation
We need to hand -s to both header and body generation, or neither:

wire/gen_peer_wire.c:53:13: error: static declaration of ‘towire_channel_update_timestamps’ follows non-static declaration
In file included from wire/gen_peer_wire.c:5:
./wire/gen_peer_wire.h:78:6: note: previous declaration of ‘towire_channel_update_timestamps’ was here

We also need it for printwire, otherwise we get static unused functions for subtypes:

devtools/gen_print_wire.c:155:13: error: ‘printwire_channel_update_checksums’ defined but not used [-Werror=unused-function]
 static void printwire_channel_update_checksums(const char *fieldname, const u8 **cursor, size_t *plen)
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
devtools/gen_print_wire.c:133:13: error: ‘printwire_channel_update_timestamps’ defined but not used [-Werror=unused-function]
 static void printwire_channel_update_timestamps(const char *fieldname, const u8 **cursor, size_t *plen)
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-27 21:19:57 -05:00
lisa neigut
64b43dd060 wire-gen: allow NULL for a tlv record in towire
Allowing signaling of no TLV with NULL
2019-07-27 05:18:27 +00:00
Rusty Russell
d551b125cc Makefile: make extract-experimental-bolt-csv check out the BOLTVERSION.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-24 06:35:57 +00:00
Rusty Russell
837530514a Makefile: make wire generated CSV depend on config.
Required if we change EXPERIMENTAL_FEATURES for example.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-24 06:35:57 +00:00
lisa neigut
281b4c241e bolt-gen: fixup the devtool/decodemsg printing facility
Fixup TLV handling in the bolt printing utility, `devtools/decodemsg`
2019-07-24 02:52:53 +00:00
Rusty Russell
e737fe72d6 wire: add bigsize parsing tests.
Based on:
	https://github.com/lightningnetwork/lightning-rfc/pull/640

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-18 06:51:35 +00:00
Rusty Russell
3477034657 tests: add test for tlvstream (from BOLT 1 test vectors).
https://github.com/lightningnetwork/lightning-rfc/pull/631

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-18 06:51:35 +00:00
Rusty Russell
7cf0006c78 wire/tlvstream: routines to marshal/unmarshal TLV streams.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-18 06:51:35 +00:00
Rusty Russell
95d9377cfe wire: tlv base type handling.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-18 06:51:35 +00:00
Rusty Russell
54790c17ea wire: rename var_int to bigsize, and insist on minimal.
The new TLV spec uses BigSize, like Bitcoin's CompactInt but
*little-endian*.  So change our name for clarity, and insist that
decoding be minimal as the spec requires.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-07-18 06:51:35 +00:00
lisa neigut
85e325cb1f bolt: update to lightning-rfc:6f6ea63233c new fundamental types
RFC tweaks to the types used to specify message fields
2019-07-16 06:10:58 +00:00
lisa neigut
5c07afac7d bolt: update to BOLT spec changes (extract format + type specifications)
updates the bolt version to 6639cef095a2ecc7b8f0c48c6e7f2f906fbfbc58.

this requires us to use the new bolt parser at generate-bolt.py
and updates to all of the type specifications (ie. from u8 -> byte)
2019-07-16 06:10:58 +00:00
lisa neigut
7046d0220c makefiles: move all unit tests under make check-units
Isolate unit tests under their own make directive.
2019-06-30 16:41:30 +09:30
Rusty Russell
728bb4e662 common/gossip_store: handle timestamp filtering.
This means we intercept the peer's gossip_timestamp_filter request
in the per-peer subdaemon itself.  The rest of the semantics are fairly
simple however.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-04 01:29:39 +00:00
Rusty Russell
5b7776cf13 wire/test: remove unused padding code.
Turns out the peer part of the spec no longer uses padding (it's used only
in the onion), and GCC-9 with -O3 warns we're padding NULL to memcpy.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-06-03 00:07:11 +00:00
lisa neigut
e3bac6c165 wire-gen: patch in the wire/gen_*_experimental_wire files
use the `wire/gen_*experimental_wire` patch file and apply it to
existing wire message set.

empty for now, but will be generated shortly.
2019-05-02 01:20:41 +00:00
lisa neigut
39153f3122 wire-gen: use staging file for generated wire csvs
for now we straight copy the `extracted_peer_wire_csv` over into the
file that is used to generate the .c/.h files; in the future
we'll use this destination file as a way to modify the
`gen_peer_wire_csv`s from a patch.
2019-05-02 01:20:41 +00:00
Rusty Russell
e51a13975f Makefile: don't generate CSVs unless explicitly told to.
ANd rename them: every other "gen" file can be discarded and rebuilt.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-05-02 01:20:41 +00:00
Rusty Russell
0b484b111e gossipd: make more compact getchannels entries.
We can save significant space by combining both sides: so much that we
can reduce the WIRE_LEN_LIMIT to something sane again.

MCP results from 5 runs, min-max(mean +/- stddev):
	store_load_msec:34467-36764(35517.8+/-7.7e+02)
	vsz_kb:2637488
	store_rewrite_sec:35.310000-36.580000(35.816+/-0.44)
	listnodes_sec:1.140000-2.780000(1.596+/-0.6)
	listchannels_sec:55.390000-58.110000(56.998+/-0.99)
	routing_sec:30.330000-30.920000(30.642+/-0.19)
	peer_write_all_sec:50.640000-53.360000(51.822+/-0.91)

MCP notable changes from previous patch (>1 stddev):
	-store_rewrite_sec:34.720000-35.130000(34.94+/-0.14)
	+store_rewrite_sec:35.310000-36.580000(35.816+/-0.44)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-04-09 12:37:16 -07:00
Rusty Russell
91849dddc4 wire: use struct node_id for node ids.
Don't turn them to/from pubkeys implicitly.  This means nodeids in the store
don't get converted, but bitcoin keys still do.

MCP results from 5 runs, min-max(mean +/- stddev):
	store_load_msec:33934-35251(34531.4+/-5e+02)
	vsz_kb:2637488
	store_rewrite_sec:34.720000-35.130000(34.94+/-0.14)
	listnodes_sec:1.020000-1.290000(1.146+/-0.086)
	listchannels_sec:51.110000-58.240000(54.826+/-2.5)
	routing_sec:30.000000-33.320000(30.726+/-1.3)
	peer_write_all_sec:50.370000-52.970000(51.646+/-1.1)

MCP notable changes from previous patch (>1 stddev):
	-store_load_msec:46184-47474(46673.4+/-4.5e+02)
	+store_load_msec:33934-35251(34531.4+/-5e+02)
	-vsz_kb:2638880
	+vsz_kb:2637488
	-store_rewrite_sec:46.750000-48.280000(47.512+/-0.51)
	+store_rewrite_sec:34.720000-35.130000(34.94+/-0.14)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-04-09 12:37:16 -07:00
Rusty Russell
a2fa699e0e Use node_id everywhere for nodes.
I tried to just do gossipd, but it was uncontainable, so this ended up being
a complete sweep.

We didn't get much space saving in gossipd, even though we should save
24 bytes per node.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-04-09 12:37:16 -07:00
Rusty Russell
b4455d517c common/node_id: new type.
Node ids are pubkeys, but we only use them as pubkeys for routing and checking
gossip messages.  So we're packing and unpacking them constantly, and wasting
some space and time.

This introduces a new type, explicitly the SEC1 compressed encoding
(33 bytes).  We ensure its validity when we load from the db, or get it
from JSON.  We still use 'struct pubkey' for peer messages, which checks
validity.

Results from 5 runs, min-max(mean +/- stddev):
	store_load_msec,vsz_kb,store_rewrite_sec,listnodes_sec,listchannels_sec,routing_sec,peer_write_all_sec
	39475-39572(39518+/-36),2880732,41.150000-41.390000(41.298+/-0.085),2.260000-2.550000(2.336+/-0.11),44.390000-65.150000(58.648+/-7.5),32.740000-33.020000(32.89+/-0.093),44.130000-45.090000(44.566+/-0.32)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-04-09 12:37:16 -07:00
Rusty Russell
837a095d68 pubkey: rename PUBKEY_DER_LEN to PUBKEY_CMPR_LEN.
Pubkeys are not not actually DER encoding, but Pieter Wuille corrected
me: it's SEC 1 documented encoding.

Results from 5 runs, min-max(mean +/- stddev):
	store_load_msec,vsz_kb,store_rewrite_sec,listnodes_sec,listchannels_sec,routing_sec,peer_write_all_sec
	38922-39297(39180.6+/-1.3e+02),2880728,41.040000-41.160000(41.106+/-0.05),2.270000-2.530000(2.338+/-0.097),44.570000-53.980000(49.696+/-3),32.840000-33.080000(32.95+/-0.095),43.060000-44.950000(43.696+/-0.72)

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-04-09 12:37:16 -07:00
Rusty Russell
6f59d4deb5 gossipd: temporarily allow giant messages
We push a huge msg for listchannels with the million-channels project.
We need to fix that, but this works around it so we can benchmark.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-04-08 04:41:43 +00:00
lisa neigut
fc0c97d06d wire: remove redundant size check from fromwire_var_int 2019-04-08 00:37:29 +00:00
lisa neigut
74ae9f09ac wire: add var_int parsing functions
so we can put and pull bitcoin 'var_int' length types from the
wire.

for more info on variable integers, see http://learnmeabitcoin.com/glossary/varint
2019-04-03 03:15:42 +00:00
Rusty Russell
6765423393 Documentation: Update to BOLT v1.0.
Mainly typo fixes, but we removed the INCORRECT_PAYMENT_AMOUNT error
altogether.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-04-01 13:22:05 +02:00
trueptolemy
92c08cd861 Fix the version of bip32 private_key generation
We set the version BIP32_VER_TEST_PRIVATE for testnet/regtest
BIP32 privkey generation with libwally-core, and set
BIP32_VER_MAIN_PRIVATE for mainnet.
For litecoin, we also set it like bitcoin else.
2019-03-18 02:47:04 +00:00
Rusty Russell
38e7d19dd5 Makefile: check for direct amount_sat/amount_msat access.
We need to do it in various places, but we shouldn't do it lightly:
the primitives are there to help us get overflow handling correct.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 08:01:37 +00:00
Rusty Russell
28f5da7b2f tools/generate-wire: use amount_msat / amount_sat for peer protocol.
Basically we tell it that every field ending in '_msat' is a struct
amount_msat, and 'satoshis' is an amount_sat.  The exceptions are
channel_update's fee_base_msat which is a u32, and
final_incorrect_htlc_amount's incoming_htlc_amt which is also a
'struct amount_msat'.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 08:01:37 +00:00
Rusty Russell
7fad7bccba common/amount: new types struct amount_msat and struct amount_sat.
They're generally used pass-by-copy (unusual for C structs, but
convenient they're basically u64) and all possibly problematic
operations return WARN_UNUSED_RESULT bool to make you handle the
over/underflow cases.

The new #include in json.h means we bolt11.c sees the amount.h definition
of MSAT_PER_BTC, so delete its local version.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 00:44:57 +00:00
Rusty Russell
269dbe585c wire: move short_channel formatting functions into bitcoin/short_channel_id
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-02-21 00:44:57 +00:00
Christian Decker
bccb8db974 wire: derive_channel_id should accept const struct *bitcoin_txid
It's not modifying anything in the txid itself, just mashing it up
with the txout index.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-02-20 15:37:59 +01:00
Rusty Russell
9f1f79587e short_channel_id_dir: new primitive for one direction of short_channel_id
Currently only used by gossipd for channel elimination.

Also print them in canonical form (/[01]), so tests need to be
changed.

Suggested-by: @cdecker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-01-15 12:01:38 +01:00
Rusty Russell
c3e96e058e bolt: Updated the BOLT specification to unify UNKNOWN_PAYMENT_HASH & INCORRECT_PAYMENT_AMOUNT
This is based on Christian's change, but removes all trace of the old codes.

I've proposed another spec change which removes this code altogether:
	https://github.com/lightningnetwork/lightning-rfc/pull/544

Signed-off-by: Christian Decker <decker.christian@gmail.com>
Reported-by: Rusty Russell <@rustyrussell>
2019-01-15 02:19:56 +00:00
Rusty Russell
dffe2f516a signature: wrap almost all signatures in struct bitcoin_signature.
This is prep work for when we sign htlc txs with
SIGHASH_SINGLE|SIGHASH_ANYONECANPAY.

We still deal with raw signatures for the htlc txs at the moment, since
we send them like that across the wire, and changing that was simply too
painful (for the moment?).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-12-06 23:11:51 +01:00
Rusty Russell
40943c9867 Update for latest bolt version: bca814e270dcbee2fea51c0a26ca99efef261f2b
The only change is that the final_incorrect_htlc_amount field is now 64
bit.  Since no implementation yet parses that field, we just updated it
quietly in the spec.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-10-28 23:51:05 +00:00
lisa neigut
1b6bd3fded wire: add test for parsing optional version of channel_update 2018-10-09 23:22:52 +00:00
lisa neigut
b9331e5ac8 gossipd: parse and respect optional htlc_maximum_msat
If another channel has set the optional `htlc_maximum_msat` field,
we should correctly parse that field and respect it when drawing up
routes for payments.
2018-10-09 23:22:52 +00:00
lisa neigut
b1ceaf9910 gossipd: Update BOLT-split flags in channel_update
BOLT 7's been updated to split the flags field in `channel_update`
into two: `channel_flags` and `message_flags`. This changeset does the
minimal necessary to get to building with the new flags.
2018-09-21 00:24:12 +00:00
Christian Decker
2d7e603ac1 chainparams: Move the BOLT2 quote to the chainparams where we set it 2018-09-14 21:18:11 +02:00
Rusty Russell
10167eedd2 wire/gen_peer_wire: Update to BOLTVERSION.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-13 17:38:13 -07:00
Rusty Russell
e8a7a7addb wire/Makefile: generate CSVs from specs based on BOLTVERSION.
Not whatever happens to be lying around!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-13 17:38:13 -07:00
Rusty Russell
fc02af12d0 wire: add onion_defs.h to bolt-check, update quote.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-09-13 17:38:13 -07:00
Rusty Russell
5cf34d6618 Remove tal_len, use tal_count() or tal_bytelen().
tal_count() is used where there's a type, even if it's char or u8, and
tal_bytelen() is going to replace tal_len() for clarity: it's only needed
where a pointer is void.

We shim tal_bytelen() for now.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-30 11:31:17 +02:00
Rusty Russell
a52d522525 gossipd: handle ping messages for remote peers too.
This simplifies our ping handling: make gossipd always do it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-25 02:13:52 +00:00
Rusty Russell
e217bc1220 per-commit-secret is a struct secret, not a sha256.
Well, it's generated by shachain, so technically it is a sha256, but
that's an internal detail.  It's a secret.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-17 12:32:00 +02:00
Rusty Russell
fed5a117e7 Update ccan/structeq.
structeq() is too dangerous: if a structure has padding, it can fail
silently.

The new ccan/structeq instead provides a macro to define foo_eq(),
which does the right thing in case of padding (which none of our
structures currently have anyway).

Upgrade ccan, and use it everywhere.  Except run-peer-wire.c, which
is only testing code and can use raw memcmp(): valgrind will tell us
if padding exists.

Interestingly, we still declared short_channel_id_eq, even though
we didn't define it any more!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-07-04 23:57:00 +02:00
Rusty Russell
ce4eef6943 wire/*: fix up BOLT references.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-18 12:31:09 +02:00
Rusty Russell
6c6da45f53 wire: Update to lastest BOLT draft.
This includes the gossip query messages.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-06-06 03:25:56 +00:00
Rusty Russell
7bfb9978de wire: remove --check-alignment arg from spec parsing tool.
It's no longer functional, and is being removed upstream.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-05-15 23:39:09 +00:00
practicalswift
abf510740d Force the use of the POSIX C locale for all commands and their subprocesses 2018-04-27 14:02:59 +02:00
practicalswift
6269a4c55d Remove unused functions not covered by unit tests 2018-03-26 23:35:56 +00:00
practicalswift
98f49c0837 Remove include in file foo.c that is already included in foo.h 2018-03-25 23:54:21 +00:00
Rusty Russell
0a6e3d1e13 utils: remove tal_tmpctx altogether, use global.
In particular, we now only free tmpctx at the end of main().

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-16 00:16:10 +00:00
Rusty Russell
46cc7c281e features: more general accessor functions.
As we add more features, the current code is insufficient.

1. Keep an array of single feature bits, for easy switching on and off.
2. Create feature_offered() which checks for both compulsory and optional
   variants.
3. Invert requires_unsupported_features() and unsupported_features()
   which tend to be double-negative, all_supported_features() and
   features_supported().
4. Move single feature definition from wire/peer_wire.h to common/features.h.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-14 02:19:37 +00:00
Rusty Russell
61f048bbf1 gossip: rename is_gossip_msg to is_msg_for_gossipd.
We're going to expand the range of messages which go through gossipd
when we support queries.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-13 16:34:55 +01:00
Rusty Russell
c5d41a23d7 short_channel_id: just use structeq.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-01 23:33:56 +01:00
Rusty Russell
042d5d13f5 short_channel_id: don't use bitfields.
I leave all the now-unnecessary accessors in place to avoid churn, but
the use of bitfields has been more pain than help.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-03-01 23:33:56 +01:00
Rusty Russell
cf3f19524e gossip: formalize passing of siphash_seed.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-26 06:35:02 +00:00
ZmnSCPxj
cfcdcf3993 wire: Implement fromwire/towire_double. 2018-02-26 02:36:27 +00:00
Rusty Russell
e92b710406 tools/generate-wire.py: remove length argument from fromwire_ routines.
We always hand in "NULL" (which means use tal_len on the msg), except
for two places which do that manually for no good reason.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-20 22:36:21 +01:00
Rusty Russell
b7db06b577 tools/generate-wire.py: wirestring type for handing strings.
A convenient alias for char *, though we don't allow control characters
so our logs can't be fooled with embedded \n.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-08 19:07:12 +01:00
Rusty Russell
9a6c36a568 bitcoin/tx: remove unused pull_bitcoin_tx_onto.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-08 19:07:12 +01:00
Rusty Russell
674a2c7554 tools/generate_wire.py: make bitcoin_tx a varlen structs
Now it allocates naturally.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-02-08 19:07:12 +01:00
practicalswift
9f47c0431a Fix typos 2018-01-28 13:53:39 +01:00
Rusty Russell
904a3e4ae3 MAX_FUNDING_SATOSHI: clean up def to be max, fix name, share with openingd.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-15 12:45:12 +01:00
Rusty Russell
2bc2502b23 wire: move extract_channel_id to here.
It can be useful for other daemons.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-12 09:43:01 +01:00
Rusty Russell
27e0ea2358 wire: remove special zero-key handling.
We used to use a hack for gossip_resolve_channel_reply, where we'd send
a NULL key on failure.  It's now been neatened to use a counter, so we
don't need this.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-11 21:39:13 +01:00
Christian Decker
248879e203 wire: Move bitcoin_tx serialization from htlc_wire to wire
Removes the need to keep a second transaction around and marking it as
`noleak`, just to make sure that dependencies are not free'd along
with the original tx.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-08 17:19:25 +01:00
practicalswift
7b80e5b66c Avoid undefined behaviour in eq_var(p1, p2, field)
memcmp((p1)->field, (p2)->field, ...) results in undefined behaviour
if (p1)->field or (p2)->field is NULL. This holds also when
tal_count((p1)->field) * sizeof(*(p1)->field) == 0.
2018-01-08 09:38:25 +00:00
Rusty Russell
810abb6b21 bitcoin: create new wrapper type bitcoin_blkid, log backward endianness.
It's just a sha256_double, but importantly when we convert it to a
string (in type_to_string, which is used in logging) we use
bitcoin_blkid_to_hex() so it's reversed as people expect.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-21 11:05:38 +00:00
Rusty Russell
0237e0b28c bitcoin: create new wrapper type bitcoin_txid, log backward endianness.
It's just a sha256_double, but importantly when we convert it to a
string (in type_to_string, which is used in logging) we use
bitcoin_txid_to_hex() so it's reversed as people expect.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-21 11:05:38 +00:00
Rusty Russell
477a529856 pay: make sure we don't think payment in progress if it immediately fails.
If send_htlc_out() fails, it doesn't initialize pc->out; that can
make us think it's still in progress.

Reported-by: Jonas Nick
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-19 14:40:26 +01:00
Christian Decker
61852b4603 cleanup: Use check_act* in handshake and remove unused static inline
This was tripping up `clang`.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-12 02:31:03 +00:00
Rusty Russell
9de3827199 channeld: don't use ccan/io, go sync.
We revert to a simple select() loop.  This makes things simpler, and fixes
the problem where we want to exit but we've partially read a peer packet.

We still queue up outgoing peer packets for non-blocking send: if we
went full sync there, we'd risk deadlock if both sides wrote a huge
number of packets and neither was reading.

This also greatly simplifies the next patches, where we want to make
our first get/response from gossipd.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-02 12:54:12 +01:00
Rusty Russell
40315bfb91 test: fix dependencies.
Test objects must be added to $(ALL_OBJS) so they correctly depend on
CCAN headers etc.

Also, each test in a subdir must depend on headers and src in the parent
directory, as it will often #include them directly.

Reported-by: Christian Decker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-22 19:40:46 +01:00
Rusty Russell
46f2e17905 openingd: update to BOLT with htlckey.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-16 17:40:57 +01:00
Rusty Russell
44e45348f2 option-data-loss-protect: fix generate-wire.py and update.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-16 15:26:41 +01:00
Rusty Russell
860a76b1c9 lightningd: update to add wire_expiry_too_far.
From recently-merged BOLT update.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-01 09:46:05 +01:00
Rusty Russell
a02ca46b03 secp256k1_ecdsa_recoverable_signature: add support.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-01 01:09:23 +00:00
Rusty Russell
78cd25d620 ipaddr: rename to wireaddr.
In future it will have TOR support, so the name will be awkward.

We collect the to/fromwire functions in common/wireaddr.c, and the
parsing functions in lightningd/netaddress.c.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-26 21:01:09 +00:00
Rusty Russell
7e022b522c gossipd: don't try to handle padding inside fromwire_ipaddr.
It makes it impossible to embed an ipaddr in another structure, since we
always try to skip over any zeroes, which may swallow a following field.

Do the skip specially for the case where we're parsing routing messages:
we never use padding for our own internal messages anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-24 16:12:22 +02:00
Rusty Russell
f32e0b35ad Revert "Send/receive cltv_expiry_delta in open/accept channel."
This reverts commit 18e3f9820f.
2017-10-11 11:54:50 +02:00
Rusty Russell
18e3f9820f Send/receive cltv_expiry_delta in open/accept channel.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-10-10 20:17:37 +02:00
Christian Decker
f35e29607d wire: Fix include order in wire_io.h
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-09-29 14:40:34 +02:00
Rusty Russell
29b83aed2a wire: restore BE endian to wire headers for internal messages.
We don't anticipate daemons across machines, but you never know.

Suggested-by: Christian Decker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 14:40:34 +02:00
Rusty Russell
3d316518fd wire: use 26-bit lengths for inter-daemon messaging.
Fixes: #289
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 14:40:34 +02:00
Rusty Russell
32631b4278 generate-wire.py: add --bolt arg, use size->type hacks only when that's specified.
For our own internal comms CSVs, we should always name explicit types.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-29 14:40:34 +02:00
Rusty Russell
a8c60ed289 towire: remove useless double-invert in towire_bool.
GCC optimizes it out anyway: I sent an uninitialized var and it sent 8!
The receiver checks the value is 0 or 1 anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-28 13:07:05 +09:30
Rusty Russell
ee9e300da0 gossip: fix address descriptor handling.
1. The code to skip over padding didn't take into account max.
2. It also didn't use symbolic names.
3. We are not supposed to fail on unknown addresses, just stop parsing.
4. We don't use the read_ip/write_ip code, so get rid of it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-09-03 02:01:54 +02:00
Rusty Russell
bbed5e3411 Rename subdaemons, move them into top level.
We leave the *build* results in lightningd/ for ease of in-place testing though.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00
Rusty Russell
739b163f8b Makefiles: simplify dependencies.
Gather all binaries and objects and make the depend on external
requirements and common headers.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00
Rusty Russell
14277f93f2 tests: fix up whitespace.
The next makefile cleanup adds them to the whitespace check.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-08-29 17:54:14 +02:00