Turns out we needed more comprehensive testing; we ended up with three
separate tests. To avoid changing test_channel_drainage as we fix
spendable_msat, I substituted raw numbers there.
The first is a variation of the existing tests, testing we can't
exceed spendable_msat, and we can pay it, both ways.
The second is with a larger amount, which triggers a different problem.
The final is with a giant channel, which tests our 2^32-1 msat cap.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is where payment tests should go. Also mark it xfail for the moment,
and remove developer-only tag (propagating gossip is only 60 seconds, which
is OK).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
There were several gossip breakages in master; bumping version means
upgrades get a clean store (not just those upgrading from stable version).
Fixes: #2719
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This was incorrectly handled before, hence the wrapper which checks
correctness of the arguments.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
@pm47 gave a great bug report showing c-lightning sending the same
UPDATE_FEE over and over, with the final surprise result being that we
blamed the peer for sending us multiple empty commits!
The spam is caused by us checking "are we at the desired feerate?" but
then if we can't afford the desired feerate, setting the feerate we
can afford, even though it's a duplicate. Doing the feerate cap before
we test if it's what we have already eliminates this.
But the empty commits was harder to find: it's caused by a heuristic in
channel_rcvd_revoke_and_ack:
```
/* For funder, ack also means time to apply new feerate locally. */
if (channel->funder == LOCAL &&
(channel->view[LOCAL].feerate_per_kw
!= channel->view[REMOTE].feerate_per_kw)) {
status_trace("Applying feerate %u to LOCAL (was %u)",
channel->view[REMOTE].feerate_per_kw,
channel->view[LOCAL].feerate_per_kw);
channel->view[LOCAL].feerate_per_kw
= channel->view[REMOTE].feerate_per_kw;
channel->changes_pending[LOCAL] = true;
}
```
We assume we never send duplicates, so we detect an otherwise-empty
change using the difference in feerates. If we don't set this flag,
we will get upset if we receive a commitment_signed since we consider
there to be no changes to commit.
This is actually hard to test: the previous commit adds a test which
spams update_fee and doesn't trigger this bug, because both sides
use the same "there's nothing outstanding" logic.
Fixes: #2701
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is important for things we automatically watched because it spends a
watch txo, but only onchaind knows the details about what the TX really is and
how it should be handled.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
This takes the guesswork out of `drop_to_chain` and allows us to annotate the
last_tx consistently.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
Since we add the transactions while processing the blockchain, and before we
have enough context to annotate them correctly, i.e., in the txwatches, we add
them first and then annotate them aposteriori.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
Mainly used to differentiate channel-related transactions from on-chain wallet
transactions. Will be used to filter `listtransaction` results and bundle
transactions that belong to the same channel.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
I was bumping against some blocksync performance issues with 12k+ keys, 24k+
scriptpubkeys being checked against, and migrating that list to a hashset is
an easy fix to shave off 99% of the time to process a block.
Signed-off-by: Christian Decker <decker.christian@gmail.com>
I noticed that DEVELOPER was being reset to 0 by ./configure --reconfigure;
that's because we set the defaults first, then --reconfigure would only
override *unset* vars. We now set up defaults last.
We unify all our "find the default" functions, to neaten them; in
particular find_pytest and our open-coded valgrind testing function.
We can figure out whether valgrind works using /bin/true instead of waiting
until configurator is built.
We're also more careful with ${FOO-default} vs ${FOO:-default}; the former
does not replace if FOO is set to the empty string, which is possible for
some of our settings (COPTFLAGS, CDEBUGFLAGS in particular).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
- mock_rpc function now returns full JSON-RPC response, is much cleaner
- Since reached_announce_depth counting is fixed when starting
channeld, we don't need the 7th block to tell depth anymore.
Remote node may (incorrectly) not send announcement_signatures when
reconnecting, so we we use a copy and can still re-announce.
Also checks that we still send our announcement_signatures when reconnecting.
Fixes a corner case when reconnecting (which restarts channeld) at depth=6
where we didn't correctly send/respond with announce_signatures.
NOTE: A complete restart of node may initialize channeld with unupdated height
because of an unfinished rescan. But when rescan is finished, funding tx_watch is
fired (at least once), which then tells channeld the latest depth.
Broken by 909913c265, but since Travis
skips this test ("temporarily", according to the commit msg in January!)
it wasn't caught.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
1. Create a plugin: ./lightning/tests/plugins/pretend_badlog.py
This plugin subscribes 'warning' notification and log the payload of
'warning';
2. Add a new test: tests/test_plugin.py::test_warning_notification
This test runs the plugin-pretend_badlog.py and check if 'warning'
notification can be normal triggered and subscribed.
- Related Changes for `warning` notification
Add a `bool` type parameter in `log_()` and `lov()`, this `bool` flag
indicates if we should call `warning` notifier.
1) The process of copying `log_book` of every peer to the `log_book` of
`ld` is usually included in `log_()` and `lov()`, and it may lead to
repeated `warning` notification. So a `bool`, which explicitly indicates
if the `warning` notification is disabled during this call, is necessary
.
2) The `LOG_INFO` and `LOG_DEBUG` level don't need to call
warning, so set that `bool` paramater as `FALSE` for these log level and
only set it as `TRUE` for `LOG_UNUAUSL`/`LOG_BROKEN`. As for `LOG_IO`,
it use `log_io()` to log, so we needn't think about notifier for it.
This notification bases on `LOG_BROKEN` and `LOG_UNUSUAL` level log.
--Introduction
A notification for topic `warning` is sent every time a new `BROKEN`/
`UNUSUAL` level(in plugins, we use `error`/`warn`) log generated, which
means an unusual/borken thing happens, such as channel failed,
message resolving failed...
```json
{
"warning": {
"level": "warn",
"time": "1559743608.565342521",
"source": "lightningd(17652): 0821f80652fb840239df8dc99205792bba2e559a05469915804c08420230e23c7c chan #7854:",
"log": "Peer permanent failure in CHANNELD_NORMAL: lightning_channeld: sent ERROR bad reestablish dataloss msg"
}
}
```
1. `level` is `warn` or `error`:
`warn` means something seems bad happened and it's under control, but
we'd better check it;
`error` means something extremely bad is out of control, and it may lead
to crash;
2. `time` is the second since epoch;
3. `source`, in fact, is the `prefix` of the log_entry. It means where
the event happened, it may have the following forms:
`<node_id> chan #<db_id_of_channel>:`, `lightningd(<lightningd_pid>):`,
`plugin-<plugin_name>:`, `<daemon_name>(<daemon_pid>):`, `jsonrpc:`,
`jcon fd <error_fd_to_jsonrpc>:`, `plugin-manager`;
4. `log` is the context of the original log entry.
--Note:
1. The main code uses `UNUSUAL`/`BROKEN`, and plugin module uses `warn`
/`error`, considering the consistency with plugin, warning choose `warn`
/`error`. But users who use c-lightning with plugins may want to
`getlog` with specified level when receive warning. It's the duty for
plugin dev to turn `warn`/`error` into `UNUSUAL`/`BROKEN` and present it
to the users, or pass it directly to `getlog`;
2. About time, `json_log()` in `log` module uses the Relative Time, from
the time when `log_book` inited to the time when this event happend.
But I consider the `UNUSUAL`/`BROKEN` event is rare, and it is very
likely to happen after running for a long time, so for users, they will
pay more attention to Absolute Time.
-- Related Change
1. Remove the definitions of `log`, `log_book`, `log_entry` from `log.c`
to `log.h`, then they can be used in warning declaration and definition.
2. Remove `void json_add_time(struct json_stream *result, const char
*fieldname, struct timespec ts)` from `log.c` to `json.c`, and add
related declaration in `json.h`. Now the notification function in
`notification.c` can call it.
2. Add a pointer to `struct lightningd` in `struct log_book`. This may
affect the independence of the `log` module, but storing a pointer to
`ld` is more direct;
We currently set TIMEOUT and PYTEST_PAR in our Travis instance directly,
which is a bit unhelpful. In particular, parallelism should be increased
when DEVELOPER isn't set since we spend 60 seconds waiting for gossip in
many cases.
If this doesn't stop timeouts, I'll have to mark more tests SLOW_MACHINE :(
331.24s call tests/test_connection.py::test_opening_tiny_channel
245.45s call tests/test_gossip.py::test_gossip_store_load_complex
215.35s call tests/test_invoices.py::test_invoice_routeboost
194.04s call tests/test_plugin.py::test_htlc_accepted_hook_forward_restart
182.62s call tests/test_plugin.py::test_htlc_accepted_hook_fail
182.02s call tests/test_plugin.py::test_htlc_accepted_hook_resolve
182.01s call tests/test_pay.py::test_pay_limits
159.10s call tests/test_gossip.py::test_gossip_notices_close
153.94s call tests/test_connection.py::test_peerinfo
121.62s call tests/test_invoices.py::test_invoice_preimage
121.46s call tests/test_gossip.py::test_gossipwith
120.76s call tests/test_pay.py::test_setchannelfee_all
120.61s call tests/test_connection.py::test_second_channel
120.42s call tests/test_closing.py::test_closing_id
120.39s call tests/test_gossip.py::test_gossip_addresses
120.39s call tests/test_gossip.py::test_gossip_weirdalias
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We reserve inputs when we're going to send a transaction, but we don't
unreserve them if we crash. This is most graphically demonstrated by
the txprepare case, which makes it easier to trigger.
Instead, we should query bitcoind to see whether the tx made it out or
not, as we would do manually with dev-rescan-outputs.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We fail this at the moment, since we rely on shutdown to do the cleanups
for us.
(Also had to fix the unclean shutdown path: the caller checks the rc unless
mayfail is set, and of course it's not zero since we just SIGTERM'd it).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This allows you to prepare a tx, then release or discard it later.
Shares almost all the code with json_withdraw (which is now technically
superfluous).
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
