Commit Graph

240 Commits

Author SHA1 Message Date
Rusty Russell
888745be16 dev_disconnect: remove @ marker.
Once connectd is doing this, we can't close as soon as we send,
and in fact we can't do 'fail write' either.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-30 09:50:40 +10:30
Rusty Russell
5111f39d2a connectd: clean up lightningd connection handling.
They all returned the same thing anyway.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-30 09:50:40 +10:30
Simon Vrouwe
605fda7214 common: cleanup unsused parameter in timer_expired() 2021-12-14 09:33:10 +10:30
Michael Schmoock
c2d2cc1274 connectd: fix empty error message
1. Adds the missing DNS error massages so they can be handled by
   connect_control.
2. Prepends a 'All addresses failed' to code 401 message, so we
   always have at least some error message to the user.

Changelog-None
2021-12-08 13:52:24 +01:00
Rusty Russell
d22fd59997 gossipd: remove gossip_msg.[ch]
This was a remnant from when we used to get routing from gossipd.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-06 10:05:39 +10:30
Rusty Russell
d9968bbc0c bitcoin: remove unused functions, or make static.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-06 10:05:39 +10:30
Rusty Russell
4ffda340d3 check: make sure all files outside contrib/ include "config.h" first.
And turn "" includes into full-path (which makes it easier to put
config.h first, and finds some cases check-includes.sh missed
previously).

config.h sets _GNU_SOURCE which really needs to be done before any
'#includes': we mainly got away with it with glibc, but other platforms
like Alpine may have stricter requirements.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-06 10:05:39 +10:30
Rusty Russell
5284ee4dae connectd: don't advertize websocket support if we have no other (non-Tor) addrs.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-05 16:43:09 +01:00
Rusty Russell
ff556fefc6 connectd: fix websocket binding when we're doing both IPv4 and IPv6 on same port.
We would fail connectd when listening on the IPv6 version failed; instead we should
allow that.

Changelog-Experimental: experimental-websocket-port fixed to work with default addresses.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-12-05 16:43:09 +01:00
Michael Schmoock
a3ea9fdc87 chore: use EXPERIMENTAL for BOLT7 DNS #911
Changelog-EXPERIMENTAL: Ability to announce DNS addresses
2021-11-30 09:38:17 +10:30
Michael Schmoock
cb20d6747e connectd: resolve ADDR_TYPE_DNS
This will resolve ADDR_TYPE_DNS wireaddr by expanding connect->addrs with one
new wireaddr ADDR_INTERNAL_WIREADDR per DNS result and calling recursion
2021-11-30 09:38:17 +10:30
Michael Schmoock
01e8a523e9 bolt7: allow announcement of ADDR_TYPE_DNS 2021-11-30 09:38:17 +10:30
Michael Schmoock
25bd09716f wireaddr: adds helper is_ipaddr, is_toraddr and is_dnsaddr 2021-11-30 09:38:17 +10:30
Rusty Russell
6c9b752751 memleak: make notleak use the tal name instead of a boutique struct.
This lets us mark it directly.

Get rid of long-unused "notleaks" member of struct lightningd too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-11-27 10:38:13 +10:30
Rusty Russell
68043c2e8c common: clean up autodata in common_shutdown().
valgrind locally complains about the allocations in autodata leaking:

```
==138200== 16 bytes in 1 blocks are still reachable in loss record 1 of 2
==138200==    at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==138200==    by 0x10D41A: autodata_register_ (autodata.c:20)
==138200==    by 0x10E7B8: register_autotype_type_to_string (type_to_string.h:79)
==138200==    by 0x10F5CA: register_one_type_to_string0 (block.c:259)
==138200==    by 0x19734C: __libc_csu_init (in /home/rusty/devel/cvs/lightning/common/test/run-route-specific)
==138200==    by 0x4A3D03F: (below main) (libc-start.c:264)
==138200== 
==138200== 176 bytes in 1 blocks are still reachable in loss record 2 of 2
==138200==    at 0x483DFAF: realloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==138200==    by 0x10D472: autodata_register_ (autodata.c:26)
==138200==    by 0x122D37: register_autotype_type_to_string (type_to_string.h:79)
==138200==    by 0x122F1F: register_one_type_to_string0 (node_id.c:50)
==138200==    by 0x19734C: __libc_csu_init (in /home/rusty/devel/cvs/lightning/common/test/run-route-specific)
==138200==    by 0x4A3D03F: (below main) (libc-start.c:264)
==138200== 
make: *** [Makefile:638: unittest/common/test/run-route-specific] Error 7
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-11-17 10:51:48 +10:30
Rusty Russell
b2c762969c wireaddr: clean up tor parsing.
blob[] is really a string from the commandline; leave it as a char.

And parsing is much simpler than this code makes it seem!

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-11-14 18:49:46 +01:00
Rusty Russell
2f247c7bfb torv2: remove support for advertizing and connecting.
October was the date Torv2 is no longer supported by the Tor Project;
it will probably not work at all by next release, so we should remove
it now even though it's not quite the 6 months we prefer for
deprecation cycles.

I still see 110 nodes advertizing Torv2 (vs 10,292 Torv3); we still
parse and display it, we just don't advertize or connect to it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-11-14 18:49:46 +01:00
Rusty Russell
80a47f1111 lightning_websocketd: simple proxy for websockets.
WebSocket is a bit weird:
1. It starts like an HTTP connection, but they send special headers.
2. We reply with special headers, one of which involves SHA1 of one of theirs.
3. We are then in WebSocket mode, where each frame starts with a 2-20 byte
   header.

We relay data in a simplistic way: if either side sends something, we
read it and relay it synchronously.  That avoids any gratuitous
buffering.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-22 11:56:30 +02:00
Rusty Russell
f78184ce46 connectd: listen on ports for which we should spawn a proxy.
If the port is set, we spawn it (lightning_websocketd) on any
connection to that port.  That means websocketd is a per-peer daemon,
but it means every other daemon uses the connection normally (it's
just actually talking to websocketd instead of the client directly).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-10-22 11:56:30 +02:00
Rusty Russell
b013b3ab0c patch websocket-address-support.patch 2021-10-22 11:56:30 +02:00
W. J. van der Laan
dd93f5dd29 Add missing headers for compiling on FreeBSD 13
- `netinet/in.h`: struct sockaddr_in, struct sockaddr_in6
- `unistd.h`: close()
2021-09-24 12:40:09 +09:30
Rusty Russell
79e09b92ef Makefile: remove generated files.
By popular merge-hell demand.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Build: Python is now required to build, as generated files are no longer checked into the repository.
2021-09-22 15:25:14 +02:00
Rusty Russell
24536c5561 common/autodata: use instead of ccan/autodata
This means it needs to be linked ~everywhere.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-21 18:04:43 +02:00
Rusty Russell
7401b26824 cleanup: remove unneeded includes in C files.
Before:
 Ten builds, laptop -j5, no ccache:

```
real	0m36.686000-38.956000(38.608+/-0.65)s
user	2m32.864000-42.253000(40.7545+/-2.7)s
sys	0m16.618000-18.316000(17.8531+/-0.48)s
```

 Ten builds, laptop -j5, ccache (warm):

```
real	0m8.212000-8.577000(8.39989+/-0.13)s
user	0m12.731000-13.212000(12.9751+/-0.17)s
sys	0m3.697000-3.902000(3.83722+/-0.064)s
```

After:
 Ten builds, laptop -j5, no ccache: 8% faster

```
real	0m33.802000-35.773000(35.468+/-0.54)s
user	2m19.073000-27.754000(26.2542+/-2.3)s
sys	0m15.784000-17.173000(16.7165+/-0.37)s
```

 Ten builds, laptop -j5, ccache (warm): 1% faster

```
real	0m8.200000-8.485000(8.30138+/-0.097)s
user	0m12.485000-13.100000(12.7344+/-0.19)s
sys	0m3.702000-3.889000(3.78787+/-0.056)s
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-17 09:43:22 +09:30
Rusty Russell
ea30c34d82 cleanup: remove unneeded includes in header files.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-17 09:43:22 +09:30
Rusty Russell
00a0d09340 tools/check-includes.sh: test that c files include their .h files.
This is best-practice (to ensure prototypes match up), but there were a
few places we didn't (at least, directly).  Make it a requirement,
either of form "foo.h" or <dir/foo.h>.

The noise is the change to our print templates.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-17 09:43:22 +09:30
Rusty Russell
183fe107e8 lightningd: use channel_type, pass to-and-from channeld.
Instead of explicit option_static_remotekey and option_anchor_outputs flags.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-13 15:53:23 +02:00
Rusty Russell
cb22015b2a common/channel_type: wrapper for generated 'struct channel_type'.
We make it a first-class citizen internally, even though we won't use
it over the wire (at least, non-experimental builds).  This scheme
follows the latest draft, in which features are flagged compulsory.

We also add several helper functions.

Since uses the *even* bits (as per latest spec), not the *odd* bits,
we have some other fixups.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-13 15:53:23 +02:00
Rusty Russell
1b8551923d tools: handle generating print templates for CSV without tlvs, messages
We want to use this to handle the simple description for channel_type.

It also needs to handle variable-size types (just like subtypes).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-13 15:53:23 +02:00
Rusty Russell
a5fee67b91 common/memleak: take over dump_memleak(), allow print pointer.
This will let plugins use it.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-09-08 19:11:47 +02:00
Michael Schmoock
24ea498350 cleanup: rename use_proxy_always to always_use_proxy to match cfg
This renames all occurences of use_proxy_always to always_use_proxy
to keep it inline with config values. This was a bit confusing.

Only significant change is that the payload in the plugins init
requests also contained the old name. No plugin currently seems to make
use of this variable yet. The old name 'use_proxy_always' is added when
deprecated APIs is enabled.

Changelog-Deprecated: Plugins: Renames plugin init 'use_proxy_always' to 'always_use_proxy'
2021-08-23 14:43:40 +09:30
Michael Schmoock
0550ffa9db cleanup: nits 2021-08-23 14:43:40 +09:30
Michael Schmoock
f31c5b7662 connectd: prefer non-TOR connections and filter wireaddr hint duplicate
This does two things:
 - It moves non-tor addresses upfront so it prefers normal connection
   which are less laggy and more reliable.
 - It prevents connectd from trying the same wire_addr twice when the
   addr_hint was given and gossip also added the same address.

Changelog-Changed: connectd: try non-TOR connections first
Changelog-Fixed: connectd: do not try address hint twice
2021-08-23 14:43:40 +09:30
Rusty Russell
ceb40dea38 lightningd: don't turn zero-length tlv fields into NULL.
Fixes: #4667
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-07-21 13:27:27 -04:00
niftynei
265f960cfe liquidity-ads: persist channel blockheight states to disk
Adds new tables to database, backfills, basically copies the fee_rates
state machine for channeld.
2021-07-20 13:28:38 -04:00
Rusty Russell
9eb531868f lightningd: make invoices insist on payment_secret.
In fact, we make it compulsory, which means if you don't understand it
you'll hang up on us!

Add some logging for that in future.

Changelog-Changed: Protocol: All new invoices require a payment_secret (i.e. modern TLV format onion)
Changelog-Changed: Protocol: We can no longer connect to peers which don't support `payment_secret`.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-07-14 14:38:00 -05:00
Christian Decker
b4ead97517 tlv: Allow passing some extra types to accept when parsing the stream 2021-06-26 10:55:13 +09:30
Rusty Russell
62e1e2467c dev_disconnect: new option to stop using connection, but don't close.
This allows us to ensure a packet is read by the other end, but we
don't read anything else from them or write anything to them.

Using '+' is similar, but because it closes the connection, the peer
might notice before receiving the packet (such as if it does a write).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-25 13:23:28 +09:30
Rusty Russell
976a79094b connectd: fix advertizing for Tor ports.
We were accidentally using the port that the tor service was
connecting to, not the /torport the user said to use.

Fixes: #4597
Reported-by: @openoms
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: Config: `addr` autotor and statictor /torport arguments now advertized correctly.
2021-06-17 11:46:48 +09:30
Rusty Russell
4cc3eaf79c common/bech32: update to bech32m reference.
I did this by copying the updated bech32 code, and then re-patching in
our minor changes:

1. Headers modded (we need size_t)
2. Explicit length for bech32_encode/decode (not 90).
3. Exposing and bech32_ prefix for convert_bits, charset, charset_rev.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-11 13:54:20 +09:30
Rusty Russell
2fea448498 gen/impl_template: fix generation of singleton varsize elements.
And as Lisa requested, add testcases.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-04 16:13:08 +09:30
Rusty Russell
e2f225e4cd lightningd: use connectd to send the final error message, not openingd/dualopend
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-03 16:16:34 +09:30
Rusty Russell
7cffea10ce connectd: new command to make us send message and close.
Currently we abuse openingd and dualopend to do this, but connectd already
has the ability to talk to peers, so it's more efficient.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-06-03 16:16:34 +09:30
Rusty Russell
25b5e1e099 update-mocks: make sure we cover all test programs.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-22 17:53:04 +09:30
Rusty Russell
e960b1015d !fixup
Log this so I can try to find dual fund connnectd race.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-05-11 11:25:16 +09:30
Rusty Russell
b0d6996ed6 lightningd: get connection direction from connectd.
This matters: if we connected, the address is probably usable for future connections.
But if they connected, the port is probably not (but the IP address may be).

Changelog-Added: JSON-RPC: `connect` returns "direction" ("in": they iniatated, or "out": we initiated)
Changelog-Added: plugins: `peer_connected` hook and `connect` notifications have "direction" field.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-03-26 13:22:33 +10:30
Rusty Russell
6dae0118f9 connectd: clearly differentiate incoming and outgoing paths.
This should make it clearer where the problem seen in
https://github.com/ElementsProject/lightning/issues/4297 is.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-03-25 11:31:58 +10:30
Jan Sarenik
1b02d15695 typo: information is an uncountable mass noun
See https://en.wikipedia.org/wiki/Information

In libplugin.c also the word "details" was added (without removing
the 'information').

Changelog-None
2021-03-16 10:45:40 +10:30
Rusty Russell
6b11cc8b8c common: disallow NULL channel_id to peer_failed_err.
No more sending "all-channel" errors; in particular, gossipd now only
sends warnings (which make us hang up), not errors, and peer_connected
rejections are warnings (and disconnect), not errors.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Changed: Plugins: `peer_connected` rejections now send a warning, not an error, to the peer.
2021-02-04 12:02:52 +10:30
Rusty Russell
06a54606a3 check-includes: allow redundant "config.h"
We should actually be including this (as it may define _GNU_SOURCE
etc) before any system headers.  But where we include <assert.h> we
often didn't, because check-includes would complain that the headers
included it too.

Weaken that check, and include config.h in C files before assert.h.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2021-02-04 12:02:36 +10:30