Commit Graph

103 Commits

Author SHA1 Message Date
Christian Decker
007c349dec gossip: Implement handler for enabling and disabling channels
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-26 01:09:44 +00:00
Christian Decker
c6ae4b521f gossip: Add message to enable and disable a channel
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-26 01:09:44 +00:00
Rusty Russell
cf54f23947 JSONRPC: add id argument to listpeers.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-19 22:23:45 +00:00
Rusty Russell
06c4f6ddca JSONRPC: add optional short_channel_id argument to listchannels
In order to just list one (though it may return two entries, one for each
channel direction!).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-19 22:23:45 +00:00
Rusty Russell
e1e7f289fb JSONRPC listnodes: return timestamp, alias and color.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-19 22:23:45 +00:00
Rusty Russell
784ec96841 gossip: save alias from node_announcement
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-19 22:23:45 +00:00
Rusty Russell
ced572b973 JSONRPC: getnodes: rename to listnodes.
Like listinvoice, and add optional 'id' parameter to ask about a
specific node.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-19 22:23:45 +00:00
Christian Decker
f27013c338 gossip: Only skip updates if we don't have channels yet 2018-01-14 23:40:04 +00:00
Christian Decker
8f16f6ca09 gossip: Forget nodes that have no associated channels anymore
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-14 23:40:04 +00:00
Christian Decker
3a42e52bcd gossip: Fix a memcmp with unset memory in broadcast queue
`tal_fmt` overallocates the returned string under some circumstances,
meaning that the trailer of the formatted string is unset, but still
considered in `tal_len`. The solution then is to truncate the
formatted string to the real string length. Only necessary here, since
we mix strings and `tal_len`.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-14 23:40:04 +00:00
Christian Decker
ca6c6feaad gossip: Remove channels older than 2*channel-update-interval
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-14 23:40:04 +00:00
Christian Decker
574dc8cd1f gossip: Send keepalive channel_updates
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-14 23:40:04 +00:00
Christian Decker
4470612016 gossip: Network pruning loop every channel-update-interval/2 seconds
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-14 23:40:04 +00:00
Christian Decker
7fc8e3a4e2 gossip: Pass --channel-update-interval to gossipd
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-14 23:40:04 +00:00
ZmnSCPxj
a2877232af gossipd: Check features in node_announcement and channel_announcement.
Fixes: #548
2018-01-13 11:29:42 +01:00
Christian Decker
160285f3ea gossip: Do not clobber updates attached to a channel announcement
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-12 21:55:50 +01:00
Rusty Russell
6b7b7f1007 test_lightningd.py: fix timing error in test_forward_different_fees_and_cltv
We need to make sure all the updates are known to gossip.  Since
one is the local update, we change that message to look the same.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-12 09:46:56 +01:00
Rusty Russell
d4c8210a9e gossipd: don't hang if we try to connect to already-connected peer.
Closes: #287
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-11 23:47:47 +01:00
Rusty Russell
10b40c524d gossipd: attach updates to pending transactions in preference.
Otherwise, we otherwise end up with out-of-order updates
(ie. preceeding announcements).

I assume that is because of the locally-inserted connections.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-11 23:13:23 +01:00
Rusty Russell
96ed75d8b2 gossipd: handle channel_update while we're still checking channel_announce.
We only need to keep one.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-11 23:13:23 +01:00
Rusty Russell
db30411ba5 gossipd: look up short_channel_id when we see a channel_announcement.
This is done it two parts, since we have to ask the main daemon to do
the lookup for us.

If this becomes a bottleneck, we can have a separate daemon, or even
an RPC pipe to bitcoind ourselves.

Fixes: #403
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-11 23:13:23 +01:00
Rusty Russell
e22dd5ad71 lightningd: implement gossip_get_txout/gossip_get_txout_reply.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-11 23:13:23 +01:00
Rusty Russell
61be62ddb7 gossipd: ensure node_announcement timestamps always increment.
Covers a possible source of subtle bugs.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-11 23:13:23 +01:00
Rusty Russell
0f97b8cf36 subdaemon.c: subdaemon_setup() routine for all daemons.
Our handling of SIGPIPE was incoherent and inconsistent, and we had much
cut & paste between the daemons.  They should *ALL* ignore SIGPIPE, and
much of the rest of the boilerplate can be shared, so should be.

Reported-by: @ZmnSCPxj
Fixes: #528
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-08 18:16:39 +01:00
Rusty Russell
a8de8a3140 json_getchannels: add public flag.
Fixes: #509
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-08 18:09:31 +01:00
Christian Decker
202868b677 gossipd: Fix copy-paste error when adding a channel locally
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-02 18:37:39 +01:00
practicalswift
d340ee8abb Remove redundant call to get_connection_by_scid(...) 2018-01-02 18:37:39 +01:00
practicalswift
3d39312212 Fix typos 2018-01-02 15:09:36 +01:00
Christian Decker
fff7efaf97 gossipd: Remove annoying trace 2018-01-02 01:35:59 +00:00
Christian Decker
3923109f51 gossipd: Replace lookup by short_channel_id by endpoint lookup
Sometimes we could get into a situation in which we knew the channel
but couldn't find it via the short_channel_id. That'd result in a
replacement which triggered an assert.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-02 01:35:59 +00:00
Christian Decker
3af2dcba6a gossip: Fix a typo in the add_channel_direction logic
Signed-off-by: Christian Decker <decker.christian@gmail.com>
Reported-by: William Casarin @jb55
2018-01-02 01:20:34 +00:00
practicalswift
d50fb131b6 Avoid passing uninitialized value dummy with uninitialized field addrlen to {initiator,responder}_handshake_ 2017-12-30 12:28:24 +01:00
Rusty Russell
047a2ea043 gossip: don't use assert around code with side effects.
The use of status_failed() requires a stubs update, which fails
with unnamed parameters, so tweak the status.h header as well.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-22 15:47:17 +01:00
Christian Decker
0db821e2cf routing: Fix the announcement detection for channel_announcements
If we side-load a channel, using local-add or the removed JSON-RPC
call, then we could end up in a situation in which a channel is
present, but has no associated channel_announcement. The presence of
the channel_announcement was used to identify new channels, so this
could lead to channels always being considered new. This then caused
the announcements being added to the queue always, resulting in
channel_updates preceeding the announcement.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-22 15:47:17 +01:00
Christian Decker
83caf1fdab routing: Add tracking of eviction for broadcasts
We should never be evicting channel_announcements because a) they were
deeply buried and should not change the short_channel_id/tag, b) they
are static.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-22 15:47:17 +01:00
Christian Decker
1b28220b64 Revert "broadcast: don't reorder channel_announce when we get the real one."
This reverts commit f293ff0a6a.
2017-12-22 15:47:17 +01:00
Rusty Russell
887e9dcc44 travis: reenable check-source (without BOLT text).
We've been slipping, so fix up minor issues too so it compiles.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-21 14:33:27 +01:00
Rusty Russell
810abb6b21 bitcoin: create new wrapper type bitcoin_blkid, log backward endianness.
It's just a sha256_double, but importantly when we convert it to a
string (in type_to_string, which is used in logging) we use
bitcoin_blkid_to_hex() so it's reversed as people expect.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-21 11:05:38 +00:00
Rusty Russell
f293ff0a6a broadcast: don't reorder channel_announce when we get the real one.
If channel_announce is rebroadcast, it should replace the existing one
in-place.  We currently only do this if we start from the unsigned one
and replace it with the signed one when we hit 6 confirms.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-21 09:13:28 +01:00
Rusty Russell
bb601a1eeb gossipd/test/run-bench-find_route: don't abort if we try to route to ourselves
This would fail, and we'd free an uninitialized pointer.

Also, add us to .gitignore and clear up a comment.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 13:57:28 +01:00
Rusty Russell
32cee8620b routing: re-expose half_add_connection.
I made it static in one PR (745c7b223a),
Christian used it in another.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 08:19:28 +00:00
Christian Decker
da0792e69c gossip: Add local channels when channeld tells us to
This adds the channel from us to the remote node and activates it with
our local parameters.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-20 06:14:53 +00:00
Christian Decker
2988e290cf gossip: Added internal gossip message to add a local channel
Couldn't find a good place to put these messages, we probably want to
do the same capability based request routing that we did for the HSM,
but for now this just defines the message in the master messages file.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-20 06:14:53 +00:00
Rusty Russell
bd27eba6f8 bench: reduce defaults for travis run.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-18 22:03:31 +01:00
Rusty Russell
a7eee0b669 routing: precalc per-block risk factor.
Saves a little by doing up-front calculation.

# 1M nodes:
$ /gossipd/test/run-bench-find_route 1000000 1 > /tmp/out
=> 42863 msec

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-18 22:03:31 +01:00
Rusty Russell
2824ff2b30 routing: remove possibility of overflow.
This check is expensive, so just restrict msatoshi going in, as well
as turn off channels charging more than 24x fee.

# 1M nodes:
$ /gossipd/test/run-bench-find_route 1000000 1 > /tmp/out
=> 44164 msec

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-18 22:03:31 +01:00
Rusty Russell
589cb673ce routing: remove negative fee support.
We can't get them; channel_update doesn't support it.

# 1M nodes:
$ /gossipd/test/run-bench-find_route 1000000 1 > /tmp/out
=> 47677 msec

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-18 22:03:31 +01:00
Rusty Russell
21cc904b03 gossipd/test/run-bench-find_route.c: add perfme support.
Compile this, and link from perfme-start and perfme-stop in your path:

/* Simple wrapper to allow a program to perf itself. 
 * Copyright Rusty Russell, Blockstream 2015.
 *
 * This program is free software: you can redistribute it and/or modify
 * it under the terms of the GNU General Public License as published by
 * the Free Software Foundation, either version 3 of the License, or
 * (at your option) any later version.
 *
 * This program is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 * GNU General Public License for more details.
 *
 * See <http://www.gnu.org/licenses/>.
 */
#include <ccan/err/err.h>
#include <ccan/str/str.h>
#include <sys/types.h>
#include <unistd.h>
#include <signal.h>
#include <errno.h>
#include <stdio.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <stdlib.h>

#define PERFME_PREFIX "/tmp/perfme."
#define MAX_ENV_ARGS 20

static void write_noerr(int fd)
{
	int e = errno;
	if (write(fd, "", 1) != 1)
		/* Complain about warn_unused_result fascist bullshit */ ;
	errno = e;
}

/* Child.  Setup pid, run perf. */
static void exec_perf(int pfd[2], const char *perfpid, const char *perfout,
		      pid_t parent)
{
	char pid[STR_MAX_CHARS(pid_t)];
	int i, fd;
	char *cmd, *args[MAX_ENV_ARGS + 5];
	
	fd = open(perfpid, O_CREAT|O_EXCL|O_WRONLY, 0400);
	if (fd < 0) {
		write_noerr(pfd[1]);
		err(1, "opening %s", perfpid);
	}

	sprintf(pid, "%u", getpid());
	if (write(fd, pid, strlen(pid)) != strlen(pid)) {
		write_noerr(pfd[1]);
		err(1, "writing to %s", perfpid);
	}
	close(fd);

	sprintf(pid, "%u", parent);
	cmd = getenv("PERFME");
	if (!cmd)
		cmd = "perf record --call-graph dwarf -q";
	cmd = strdup(cmd);
	for (i = 0; i < MAX_ENV_ARGS; i++) {
		args[i] = strtok(i == 0 ? cmd : NULL, " ");
		if (!args[i])
			break;
	}
	if (i == 0 || i == MAX_ENV_ARGS)
		errx(1, "Too %s args in $PERFME: '%s'",
		     i ? "many" : "few", getenv("PERFME"));

	args[i++] = "-p";
	args[i++] = pid;
	args[i++] = "-o";
	args[i++] = (char *)perfout;
	args[i++] = NULL;

	execvp(args[0], args);
	write_noerr(pfd[1]);
	err(1, "Execing %s", args[0]);
}

int main(int argc, char *argv[])
{
	pid_t parent = argv[1] ? atoi(argv[1]) : getppid();
	char perfout[sizeof(PERFME_PREFIX) + STR_MAX_CHARS(parent)];	
	char perfpid[sizeof(perfout) + sizeof(".pid")];

	err_set_progname(argv[0]);

	sprintf(perfpid, PERFME_PREFIX "%u.pid", parent);
	if (strends(argv[0], "perfme-stop")) {
		char pid[STR_MAX_CHARS(pid_t)];
		int r, fd = open(perfpid, O_RDONLY);
		if (fd < 0)
			err(1, "Opening %s", perfpid);
		r = read(fd, pid, sizeof(pid) - 1);
		if (r < 0)
			err(1, "Reading %s", perfpid);
		pid[r] = 0;
		if (unlink(perfpid) != 0)
			warn("Unlinking %s", perfpid);
		if (atoi(pid) <= 0)
			errx(1, "Invalid pid '%s' from %s", pid, perfpid);
		if (kill(atoi(pid), SIGTERM) != 0)
			err(1, "Stopping %s", pid);
		exit(0);
	} else if (strends(argv[0], "perfme-start")) {
		int pfd[2];

		sprintf(perfout, PERFME_PREFIX "%u", parent);

		/* Use pipe to detect successful exec. */
		if (pipe(pfd) != 0)
			err(1, "Creating pipe");
		
		switch (fork()) {
		case 0:
			close(pfd[0]);
			fcntl(pfd[1], F_SETFD,
			      fcntl(pfd[1], F_GETFD)|FD_CLOEXEC);

			exec_perf(pfd, perfpid, perfout, parent);
		case -1:
			err(1, "Forking");
		default:
			/* Parent.  Wait for child. */
			close(pfd[1]);
			if (read(pfd[0], perfpid, 1) == 1)
				exit(1);
			fprintf(stderr, "Perf recording into %s\n", perfout);
			sleep(1);
			exit(0);
		}
	}
	errx(1, "Unknown name: am I perfme-start or perfme-stop?");
}

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-18 22:03:31 +01:00
Rusty Russell
ee06c71efc gossipd: make routing far less chatty.
# 100,000 nodes
$ ./gossipd/test/run-bench-find_route 100000 1 > /tmp/out
=> 5357 msec

# 1M nodes:
$ /gossipd/test/run-bench-find_route 1000000 1 > /tmp/out
=> 55064 msec

# 1M nodes with CDEBUGFLAGS='-std=gnu11 -g -fstack-protector -O3 -flto' and manual linking with '-flto':
$ /gossipd/test/run-bench-find_route 1000000 1 > /tmp/out
=> 33501 msec

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-18 22:03:31 +01:00
Rusty Russell
4a54884d4b gossipd: routing benchmark.
Initial run (100,000 nodes):

$ ./gossipd/test/run-bench-find_route 100000 1 > /tmp/out
=> 15646 msec

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-18 22:03:31 +01:00