since PR #3867 utxos are unreserved by height, destroy_utxos and
related functions are not used anymore so clean them up also
However free(ld->jsonrpc) still needs to happen before free(ld) because its
destructors need list_head pointers from ld
because:
- shutdown_subdaemons can trigger db write, comments in that function say so at least
- resurrecting the main event loop with subdaemons still running is counter productive
in shutting down activity (such as htlc's, hook_calls etc.)
- custom behavior injected by plugins via hooks should be consistent, see test
in previous commmit
IDEA:
in shutdown_plugins, when starting new io_loop:
- A plugin that is still running can return a jsonrpc_request response, this triggers
response_cb, which cannot be handled because subdaemons are gone -> so any response_cb should be blocked/aborted
- jsonrpc is still there, so users (such as plugins) can make new jsonrpc_request's which
cannot be handled because subdaemons are gone -> so new rpc_request should also be blocked
- But we do want to send/receive notifications and log messages (handled in jsonrpc as jsonrpc_notification)
as these do not trigger subdaemon calls or db_write's
Log messages and notifications do not have "id" field, where jsonrpc_request *do* have an "id" field
PLAN (hypothesis):
- hack into plugin_read_json_one OR plugin_response_handle to filter-out json with
an "id" field, this should
block/abandon any jsonrpc_request responses (and new jsonrpc_requests for plugins?)
Q. Can internal (so not via plugin) jsonrpc_requests called in the main io_loop return/revive in
the shutdown io_loop?
A. No. All code under lightningd/ returning command_still_pending depends on either a subdaemon, timer or
plugin. In shutdown loop the subdaemons are dead, timer struct cleared and plugins will be taken
care of (in next commits).
fixup: we can only io_break the main io_loop once
Incase the error handling happening after the quted line is non-critical:
```
return tal_fmt(NULL, "Unable to parse address '%s': %s", arg, err_msg);
```
we should not expand the proposed_listen_announce array without adding
a proposed_wireaddr. So we move the expand of proposed_listen_announce
to the location where we also expand the proposed_wireaddr.
Changelog-None
This include the following commits:
- review 1/2: move from tab to space, and remove the exp. prop from doc;
- review 2/2: remove experimental features;
Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
It runs 6 nodes: under valgrind this ends up consuming 5.3 GB RSS. By
stopping nodes between, we peak about 1G RSS.
Measured using:
(while true; do echo $(for i in 4 5 6; do ps uh | tr -s ' ' | cut -d\ -f$i | tally; done); sleep 5; done)&
(Which measures my other processes as well, but that's only about 100M).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This happened in my tal_dump(), and I couldn't see how we ended up
with object having more than one "backtrace". Adding asserts that we
never added a second backtrace didn't trigger.
Finally I wondered if we were tal_steal() backtraces, and sure enough
we do that blinding in one place: libwally wrapping. So fix that.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Do proper refcounting on log prefixes; previously we kept them around,
which is fine, but the extra notleak() and backtrace in developer mode
could get quite heavy (I ended up with 1G of backtraces!). This is
mainly due to creating one on every JSONRPC command, and running
clboss.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: lightningd: remove slow memory leak in DEVELOPER builds.
I noticed that `wallet/db_postgres.c` never actually calls `db_changes_add`.
PostgreSQL arguably has a better replication system (a PostgreSQL cluster)
than what `db_write` hook can offer, so rather than make `db_write` work on
PostgreSQL, just document that it does not actually work there.
ChangeLog-none
Because db->conn is a void *, changing it (from a direct pointer to
a pointer to a pair of pointers) did not break compile if one place hadn't
been update.
The result was a confusing failure: sqlite3 complaining about API misuse,
since the db->conn pointer was not a valid db handle any more.
This is one case where avoiding a void * is hard: we might not even
have the postgresql types, since it might not be installed. But a union
would have been superior here.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
```
cc lightningd/subd.c
lightningd/subd.c:216:7: error: expected identifier or '('
int stdout = STDOUT_FILENO, stderr = STDERR_FILENO;
^
/usr/include/stdio.h:198:17: note: expanded from macro 'stdout'
^
lightningd/subd.c:216:7: error: expected ')'
/usr/include/stdio.h:198:17: note: expanded from macro 'stdout'
^
lightningd/subd.c:216:7: note: to match this '('
/usr/include/stdio.h:198:16: note: expanded from macro 'stdout'
^
lightningd/subd.c:224:12: error: cannot take the address of an rvalue of type 'FILE *' (aka 'struct __sFILE *')
fds[1] = &stdout;
^~~~~~~
lightningd/subd.c:225:12: error: cannot take the address of an rvalue of type 'FILE *' (aka 'struct __sFILE *')
fds[2] = &stderr;
^~~~~~~
4 errors generated.
gmake: *** [Makefile:279: lightningd/subd.o] Error 1
```
Changelog-None: introduced since last release.
Fixes: #4914
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> ```
ChangeLog-Added: With the `sqlite3://` scheme for `--wallet` option, you can now specify a second file path for real-time database backup by separating it from the main file path with a `:` character.
1. db_col_text becomes db_col_strdup, which is what is usually wanted.
2. db_col_short_channel_id becomes db_col_short_channel_id_str, to emphasize
that it stores in string form. Modern versions should store u64.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This simplistically maps names to numbers, eg:
SELECT foo, bar FROM tbl;
'foo' -> 0
'bar' -> 1
If a statement is too complex for our simple parsing, we treat it as a
single field (which currently it always is).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
valgrind locally complains about the allocations in autodata leaking:
```
==138200== 16 bytes in 1 blocks are still reachable in loss record 1 of 2
==138200== at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==138200== by 0x10D41A: autodata_register_ (autodata.c:20)
==138200== by 0x10E7B8: register_autotype_type_to_string (type_to_string.h:79)
==138200== by 0x10F5CA: register_one_type_to_string0 (block.c:259)
==138200== by 0x19734C: __libc_csu_init (in /home/rusty/devel/cvs/lightning/common/test/run-route-specific)
==138200== by 0x4A3D03F: (below main) (libc-start.c:264)
==138200==
==138200== 176 bytes in 1 blocks are still reachable in loss record 2 of 2
==138200== at 0x483DFAF: realloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==138200== by 0x10D472: autodata_register_ (autodata.c:26)
==138200== by 0x122D37: register_autotype_type_to_string (type_to_string.h:79)
==138200== by 0x122F1F: register_one_type_to_string0 (node_id.c:50)
==138200== by 0x19734C: __libc_csu_init (in /home/rusty/devel/cvs/lightning/common/test/run-route-specific)
==138200== by 0x4A3D03F: (below main) (libc-start.c:264)
==138200==
make: *** [Makefile:638: unittest/common/test/run-route-specific] Error 7
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
If we forget a channel, we can get upset when we get an update about it:
```
2021-11-04T00:35:43.8242370Z lightningd-3: 2021-11-04T00:29:22.073Z DEBUG gossipd: Pruning channel 103x1x1 from network view (ages 61 and 22s)
...
2021-11-04T00:35:43.8263502Z lightningd-3: 2021-11-04T00:29:22.509Z DEBUG 022d223620a359a47ff7f7ac447c85c46c923da53389221a0054c11c1e3ca31d59-gossipd: Bad gossip order: WIRE_CHANNEL_UPDATE before announcement 103x1x1/0
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
