Commit Graph

80 Commits

Author SHA1 Message Date
Rusty Russell
0f97b8cf36 subdaemon.c: subdaemon_setup() routine for all daemons.
Our handling of SIGPIPE was incoherent and inconsistent, and we had much
cut & paste between the daemons.  They should *ALL* ignore SIGPIPE, and
much of the rest of the boilerplate can be shared, so should be.

Reported-by: @ZmnSCPxj
Fixes: #528
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-08 18:16:39 +01:00
Rusty Russell
a8de8a3140 json_getchannels: add public flag.
Fixes: #509
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2018-01-08 18:09:31 +01:00
Christian Decker
202868b677 gossipd: Fix copy-paste error when adding a channel locally
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-02 18:37:39 +01:00
practicalswift
d340ee8abb Remove redundant call to get_connection_by_scid(...) 2018-01-02 18:37:39 +01:00
practicalswift
3d39312212 Fix typos 2018-01-02 15:09:36 +01:00
Christian Decker
fff7efaf97 gossipd: Remove annoying trace 2018-01-02 01:35:59 +00:00
Christian Decker
3923109f51 gossipd: Replace lookup by short_channel_id by endpoint lookup
Sometimes we could get into a situation in which we knew the channel
but couldn't find it via the short_channel_id. That'd result in a
replacement which triggered an assert.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2018-01-02 01:35:59 +00:00
Christian Decker
3af2dcba6a gossip: Fix a typo in the add_channel_direction logic
Signed-off-by: Christian Decker <decker.christian@gmail.com>
Reported-by: William Casarin @jb55
2018-01-02 01:20:34 +00:00
practicalswift
d50fb131b6 Avoid passing uninitialized value dummy with uninitialized field addrlen to {initiator,responder}_handshake_ 2017-12-30 12:28:24 +01:00
Rusty Russell
047a2ea043 gossip: don't use assert around code with side effects.
The use of status_failed() requires a stubs update, which fails
with unnamed parameters, so tweak the status.h header as well.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-22 15:47:17 +01:00
Christian Decker
0db821e2cf routing: Fix the announcement detection for channel_announcements
If we side-load a channel, using local-add or the removed JSON-RPC
call, then we could end up in a situation in which a channel is
present, but has no associated channel_announcement. The presence of
the channel_announcement was used to identify new channels, so this
could lead to channels always being considered new. This then caused
the announcements being added to the queue always, resulting in
channel_updates preceeding the announcement.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-22 15:47:17 +01:00
Christian Decker
83caf1fdab routing: Add tracking of eviction for broadcasts
We should never be evicting channel_announcements because a) they were
deeply buried and should not change the short_channel_id/tag, b) they
are static.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-22 15:47:17 +01:00
Christian Decker
1b28220b64 Revert "broadcast: don't reorder channel_announce when we get the real one."
This reverts commit f293ff0a6a.
2017-12-22 15:47:17 +01:00
Rusty Russell
887e9dcc44 travis: reenable check-source (without BOLT text).
We've been slipping, so fix up minor issues too so it compiles.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-21 14:33:27 +01:00
Rusty Russell
810abb6b21 bitcoin: create new wrapper type bitcoin_blkid, log backward endianness.
It's just a sha256_double, but importantly when we convert it to a
string (in type_to_string, which is used in logging) we use
bitcoin_blkid_to_hex() so it's reversed as people expect.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-21 11:05:38 +00:00
Rusty Russell
f293ff0a6a broadcast: don't reorder channel_announce when we get the real one.
If channel_announce is rebroadcast, it should replace the existing one
in-place.  We currently only do this if we start from the unsigned one
and replace it with the signed one when we hit 6 confirms.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-21 09:13:28 +01:00
Rusty Russell
bb601a1eeb gossipd/test/run-bench-find_route: don't abort if we try to route to ourselves
This would fail, and we'd free an uninitialized pointer.

Also, add us to .gitignore and clear up a comment.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 13:57:28 +01:00
Rusty Russell
32cee8620b routing: re-expose half_add_connection.
I made it static in one PR (745c7b223a),
Christian used it in another.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-20 08:19:28 +00:00
Christian Decker
da0792e69c gossip: Add local channels when channeld tells us to
This adds the channel from us to the remote node and activates it with
our local parameters.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-20 06:14:53 +00:00
Christian Decker
2988e290cf gossip: Added internal gossip message to add a local channel
Couldn't find a good place to put these messages, we probably want to
do the same capability based request routing that we did for the HSM,
but for now this just defines the message in the master messages file.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-20 06:14:53 +00:00
Rusty Russell
bd27eba6f8 bench: reduce defaults for travis run.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-18 22:03:31 +01:00
Rusty Russell
a7eee0b669 routing: precalc per-block risk factor.
Saves a little by doing up-front calculation.

# 1M nodes:
$ /gossipd/test/run-bench-find_route 1000000 1 > /tmp/out
=> 42863 msec

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-18 22:03:31 +01:00
Rusty Russell
2824ff2b30 routing: remove possibility of overflow.
This check is expensive, so just restrict msatoshi going in, as well
as turn off channels charging more than 24x fee.

# 1M nodes:
$ /gossipd/test/run-bench-find_route 1000000 1 > /tmp/out
=> 44164 msec

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-18 22:03:31 +01:00
Rusty Russell
589cb673ce routing: remove negative fee support.
We can't get them; channel_update doesn't support it.

# 1M nodes:
$ /gossipd/test/run-bench-find_route 1000000 1 > /tmp/out
=> 47677 msec

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-18 22:03:31 +01:00
Rusty Russell
21cc904b03 gossipd/test/run-bench-find_route.c: add perfme support.
Compile this, and link from perfme-start and perfme-stop in your path:

/* Simple wrapper to allow a program to perf itself. 
 * Copyright Rusty Russell, Blockstream 2015.
 *
 * This program is free software: you can redistribute it and/or modify
 * it under the terms of the GNU General Public License as published by
 * the Free Software Foundation, either version 3 of the License, or
 * (at your option) any later version.
 *
 * This program is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 * GNU General Public License for more details.
 *
 * See <http://www.gnu.org/licenses/>.
 */
#include <ccan/err/err.h>
#include <ccan/str/str.h>
#include <sys/types.h>
#include <unistd.h>
#include <signal.h>
#include <errno.h>
#include <stdio.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <stdlib.h>

#define PERFME_PREFIX "/tmp/perfme."
#define MAX_ENV_ARGS 20

static void write_noerr(int fd)
{
	int e = errno;
	if (write(fd, "", 1) != 1)
		/* Complain about warn_unused_result fascist bullshit */ ;
	errno = e;
}

/* Child.  Setup pid, run perf. */
static void exec_perf(int pfd[2], const char *perfpid, const char *perfout,
		      pid_t parent)
{
	char pid[STR_MAX_CHARS(pid_t)];
	int i, fd;
	char *cmd, *args[MAX_ENV_ARGS + 5];
	
	fd = open(perfpid, O_CREAT|O_EXCL|O_WRONLY, 0400);
	if (fd < 0) {
		write_noerr(pfd[1]);
		err(1, "opening %s", perfpid);
	}

	sprintf(pid, "%u", getpid());
	if (write(fd, pid, strlen(pid)) != strlen(pid)) {
		write_noerr(pfd[1]);
		err(1, "writing to %s", perfpid);
	}
	close(fd);

	sprintf(pid, "%u", parent);
	cmd = getenv("PERFME");
	if (!cmd)
		cmd = "perf record --call-graph dwarf -q";
	cmd = strdup(cmd);
	for (i = 0; i < MAX_ENV_ARGS; i++) {
		args[i] = strtok(i == 0 ? cmd : NULL, " ");
		if (!args[i])
			break;
	}
	if (i == 0 || i == MAX_ENV_ARGS)
		errx(1, "Too %s args in $PERFME: '%s'",
		     i ? "many" : "few", getenv("PERFME"));

	args[i++] = "-p";
	args[i++] = pid;
	args[i++] = "-o";
	args[i++] = (char *)perfout;
	args[i++] = NULL;

	execvp(args[0], args);
	write_noerr(pfd[1]);
	err(1, "Execing %s", args[0]);
}

int main(int argc, char *argv[])
{
	pid_t parent = argv[1] ? atoi(argv[1]) : getppid();
	char perfout[sizeof(PERFME_PREFIX) + STR_MAX_CHARS(parent)];	
	char perfpid[sizeof(perfout) + sizeof(".pid")];

	err_set_progname(argv[0]);

	sprintf(perfpid, PERFME_PREFIX "%u.pid", parent);
	if (strends(argv[0], "perfme-stop")) {
		char pid[STR_MAX_CHARS(pid_t)];
		int r, fd = open(perfpid, O_RDONLY);
		if (fd < 0)
			err(1, "Opening %s", perfpid);
		r = read(fd, pid, sizeof(pid) - 1);
		if (r < 0)
			err(1, "Reading %s", perfpid);
		pid[r] = 0;
		if (unlink(perfpid) != 0)
			warn("Unlinking %s", perfpid);
		if (atoi(pid) <= 0)
			errx(1, "Invalid pid '%s' from %s", pid, perfpid);
		if (kill(atoi(pid), SIGTERM) != 0)
			err(1, "Stopping %s", pid);
		exit(0);
	} else if (strends(argv[0], "perfme-start")) {
		int pfd[2];

		sprintf(perfout, PERFME_PREFIX "%u", parent);

		/* Use pipe to detect successful exec. */
		if (pipe(pfd) != 0)
			err(1, "Creating pipe");
		
		switch (fork()) {
		case 0:
			close(pfd[0]);
			fcntl(pfd[1], F_SETFD,
			      fcntl(pfd[1], F_GETFD)|FD_CLOEXEC);

			exec_perf(pfd, perfpid, perfout, parent);
		case -1:
			err(1, "Forking");
		default:
			/* Parent.  Wait for child. */
			close(pfd[1]);
			if (read(pfd[0], perfpid, 1) == 1)
				exit(1);
			fprintf(stderr, "Perf recording into %s\n", perfout);
			sleep(1);
			exit(0);
		}
	}
	errx(1, "Unknown name: am I perfme-start or perfme-stop?");
}

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-18 22:03:31 +01:00
Rusty Russell
ee06c71efc gossipd: make routing far less chatty.
# 100,000 nodes
$ ./gossipd/test/run-bench-find_route 100000 1 > /tmp/out
=> 5357 msec

# 1M nodes:
$ /gossipd/test/run-bench-find_route 1000000 1 > /tmp/out
=> 55064 msec

# 1M nodes with CDEBUGFLAGS='-std=gnu11 -g -fstack-protector -O3 -flto' and manual linking with '-flto':
$ /gossipd/test/run-bench-find_route 1000000 1 > /tmp/out
=> 33501 msec

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-18 22:03:31 +01:00
Rusty Russell
4a54884d4b gossipd: routing benchmark.
Initial run (100,000 nodes):

$ ./gossipd/test/run-bench-find_route 100000 1 > /tmp/out
=> 15646 msec

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-18 22:03:31 +01:00
Rusty Russell
745c7b223a gossipd/routing: make local functions static.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-18 22:03:31 +01:00
Christian Decker
a8a6d1d669 channel: Directly send announcements and updates to gossipd
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-17 02:44:20 +00:00
Rusty Russell
94711969f9 gossipd: hand out gossip_index to other daemons.
When gossipd sends a message, have a gossip_index.  When it gets back a
peer, the current gossip_index is included, so it can know exactly where
it's up to.

Most of this is mechanical plumbing through openingd, channeld and closingd,
even though openingd and closingd don't (currently) read gossip, so their
gossip_index will be unchanged.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-17 02:44:20 +00:00
Rusty Russell
6afc7dcc09 gossipd: don't increment broadcast_index until *after* message sent.
If the peer is moved from remote to local, this may be lost; it's more
secure to increment after we've sent the broadcast.
2017-12-17 02:44:20 +00:00
Rusty Russell
67aa95c194 gossipd: hand back peer, don't hand a new peer.
All peers come from gossipd, and maintain an fd to talk to it.  Sometimes
we hand the peer back, but to avoid a race, we always recreated it.

The race was that a daemon closed the gossip_fd, which made gossipd
forget the peer, then master handed the peer back to gossipd.  We stop
the race by never closing the gossipfd, but hand it back to gossipd
for closing.

Now gossipd has to accept two fds, but the handling of peers is far
clearer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-17 02:44:20 +00:00
Rusty Russell
8832925122 gossipd: don't hand length to route code, it's implied.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-17 02:44:20 +00:00
Rusty Russell
36746bf65a Makefile: make gossipd objects depend correctly on its own headers.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-17 02:44:20 +00:00
Rusty Russell
bcfbc24308 gossipd: split peer structure to clearly separate local and remote fields.
We should also go through and use consistent nomenclature on functions which
are used with a local peer ("lpeer_xxx"?) and those with a remote peer
("rpeer_xxx"?) but this is minimal.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-17 02:44:20 +00:00
Christian Decker
a0724f45af routing: Do not set an empty channel_announcement if none is given
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-17 02:44:20 +00:00
Christian Decker
4a51de44c0 gossip: Forward when we don't have a valid node_announcement yet
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-17 02:44:20 +00:00
Christian Decker
580a2612fd routing: Add local and sigfail to trace when receiving cannounce
Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-17 02:44:20 +00:00
Christian Decker
ba45ccffd8 routing: Return boolean from handle_channel_announcement
This will later be used to determine whether or not we should announce
ourselves as a node.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-17 02:44:20 +00:00
Christian Decker
b5f737e563 gossip: Passing alias, color and wireaddrs through to gossipd
First step towards `gossipd` managing the `node_announcement`.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-17 02:44:20 +00:00
Christian Decker
19c030ea1f routing: Make routing_state aware of its own ID
This is used to identify our own announcements.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-17 02:44:20 +00:00
Christian Decker
8d6281175d gossip: Remove HSM_FD from handshake 2017-12-17 02:44:20 +00:00
Christian Decker
61852b4603 cleanup: Use check_act* in handshake and remove unused static inline
This was tripping up `clang`.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2017-12-12 02:31:03 +00:00
practicalswift
61c47c09d0 Fix typos 2017-12-08 13:07:20 +01:00
Christian Decker
2d9cd4759e hsm: Make the master simply a client with special capabilities 2017-12-03 17:06:13 +01:00
Rusty Russell
40516941af channeld: fill in channel_update field in errors by asking gossipd.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-02 12:54:12 +01:00
Rusty Russell
933c1794a1 gossipd: gossip_get_update / gossip_get_update_reply from channeld.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-12-02 12:54:12 +01:00
Rusty Russell
af7e6158af Makefile: clean needs to do more, distclean should remove everything.
I checked this with git status --ignored after a full build and 'make distclean'.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-24 13:29:03 +01:00
Rusty Russell
40315bfb91 test: fix dependencies.
Test objects must be added to $(ALL_OBJS) so they correctly depend on
CCAN headers etc.

Also, each test in a subdir must depend on headers and src in the parent
directory, as it will often #include them directly.

Reported-by: Christian Decker
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-22 19:40:46 +01:00
Rusty Russell
a9002eac52 Update to latest BOLT.
And nail "make check-source" to that specific version (which is a commit id,
not a branch name, so needs a different syntax for git).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2017-11-01 01:09:23 +00:00