Commit Graph

6569 Commits

Author SHA1 Message Date
Christian Decker
f08fd9eaab wallet: Remove printf-like db_select variant
We need to have full DB queries that can be extracted at compile time later in
order to be able to rewrite them in other SQL dialects. In addition we had a
bit of unnecessary code-duplication in db_select and db_select_prepare. Now
the former uses the latter internally.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-05 23:41:05 +00:00
Christian Decker
b856bb2f54 wallet: Expand select query field lists from macros
These will interfere with our query extraction process later on, and they were
really separating definition from use anyway, so let's expand these field lists.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-05 23:41:05 +00:00
trueptolemy
b5600b5403 CHANGELOG: And the entry of the change of txprepare 2019-09-05 16:05:36 -05:00
trueptolemy
d813dff3b4 doc: Update the document of txprepare 2019-09-05 16:05:36 -05:00
trueptolemy
74055eada5 pytest: Test txprepare with multiple outputs 2019-09-05 16:05:36 -05:00
trueptolemy
42d5d753db cleanup: Remove unused structure struct withdrawal 2019-09-05 16:05:36 -05:00
trueptolemy
cdcafdaf74 API: txprepare now support mutiple outputs 2019-09-05 16:05:36 -05:00
trueptolemy
0ab3817715 common: Fix a memleak in withdraw_tx 2019-09-05 16:05:36 -05:00
trueptolemy
b660531216 common: withdraw_tx() now use the array of struct bitcoin_tx_output as parameter 2019-09-05 16:05:36 -05:00
trueptolemy
6570c743c4 bitcoin: Add the interface to add multiple outputs to bitcoin transaction 2019-09-05 16:05:36 -05:00
darosior
4672795e1a lightningd: don't join the config_dir to the pid_file path 2019-09-05 19:10:33 +02:00
lisa neigut
10ed2ebdb4 wallet: track our change outputs
Add change scripts to our txfilter so that we mark them as confirmed
(and can subsequently spend them)
2019-09-05 15:48:49 +02:00
lisa neigut
01e0ba8232 test: add check that withdraw's change is confirmed
We're not tracking change outputs for withdraws, so they're not
confirming. This test exposes the problem.
2019-09-05 15:48:49 +02:00
lisa neigut
4030ccf94c fix memleak: remove duplicated new_channel creation
We call this same exact code inside `funder_finalize_channel_setup`;
which results in a memleak since we don't free the previous before
clobbering it.
2019-09-05 01:32:48 +00:00
lisa neigut
4d2f91f7ae test-fixtures: use helper for checking errors
log files were being deleted on memleak errors, since
we weren't marking the node has having an error.

this helper function is designed to exactly handle this, so
we use the helper function and modify it to print any additional
error messages that are handed back from killall.
2019-09-05 01:32:48 +00:00
lisa neigut
df1d92a7a2 test-fixtures: return error set, don't throw exception
Throwing an exception while killing all nodes meant that
we aren't cleaning up all the nodes properly. Instead,
collect the errors, and return them back to the upper level,
where we report them and terminate as expected.
2019-09-05 01:32:48 +00:00
lisa neigut
63c80e7aa9 test-fixtures: move 'broken' check to after memleak
Memleaks appear in the logs as 'broken', so the broken log
check captures them as well. This moves broken to after memleak
so we get more informative error messages.
2019-09-05 01:32:48 +00:00
lisa neigut
5663ecc599 listfunds: add 'blockheight' for confirmed transactions
Needed to calculate the value of all inputs for 'all' in
externalized fundchannel
2019-09-04 14:08:08 +02:00
Michael Schmoock
76f27f47dc doc: HACKING - point out how to run the pylightning tests 2019-09-03 19:02:04 +02:00
Michael Schmoock
bc5add399b pylightning: adds a millisatoshi test class 2019-09-03 19:02:04 +02:00
Michael Schmoock
c0069d8944 pylightning: adds Millisatoshi __radd__ method for sum() 2019-09-03 19:02:04 +02:00
darosior
0e2ec27576 doc/requirements.txt: update 'mrkd' dependency 2019-09-02 16:31:36 +02:00
Christian Decker
7fafd934cf pytest: Remove --duration argument
We're not going to optimize travis any longer so let's no fill logs with
useless measurements.
2019-09-02 16:31:36 +02:00
Christian Decker
fd1e8a5999 pytest: Fix directory deletion on passed tests
We were checking the test request against the searched for string. This fixes
it by actually looking at the outcome instead and should clean up correctly
if tests do not fail.

Signed-off-by: Christian Decker <decker.christian@gmail.com>
2019-09-02 16:31:36 +02:00
Christian Decker
245dff26e0 pytest: Updated requirements.txt to latest versions
This is an issue that was raised in #2665: some of the dependencies where
causing warnings to be added to the logs about deprecated dependencies. Since
I did not get these warnings I just blanket updated all the dependencies in
the hopes of getting the warnings to resolve.

Signed-off-by: Christian Decker <@cdecker>
2019-09-02 16:31:36 +02:00
Christian Decker
c3254e6639 docs: Update manpages to be recognized correctly by mrkd
mrkd started enforcing the `name -- short description` style of top-level
headings somewhere, and was thus failing to build the man-pages. I swapped
the title and with the existing short description to make it work
again. `mrkd` will automatically infer the section from the filename so no
need to put it in the title as well.

In addition I removed the "last updated" lines at the bottom since they are
out of date at best, and misleading at the worst. If we want to keep them, I'd
suggest generating them from the commit that last touched them.
2019-09-02 16:31:36 +02:00
Rusty Russell
049529542a lightningd: delay reprocessing of incoming htlcs at startup until plugins ready.
Fixes: #2923
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-01 16:55:32 +02:00
Rusty Russell
f0ab4d72da pytest: add test that htlc_accepted hook only gets called after init.
We fail this.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-09-01 16:55:32 +02:00
nicolas.dorier
49496ab3ba [Docker] Use stretch-slim instead of Alpine 2019-08-30 16:17:51 +02:00
darosior
8f8e955421 Update installation instructions and Dockerfile
Get rid of the 'asciidoc' dependency, make developers also install 'doc/requirements.txt'
2019-08-30 00:34:11 +00:00
darosior
092f20aeeb README: Replace broken links to manpages 2019-08-30 00:34:11 +00:00
Rusty Russell
3ef5160fa7 travis: add --enable-experimental-features (build-only) test.
Also, add a 32-bit build-only test, and move all the build-only tests
to the top.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-29 15:51:36 +02:00
Rusty Russell
02b9b7f6e6 tests: update mocks for --enable-experimental-features builds.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-29 15:51:36 +02:00
Christian Decker
b77270fa46 docs: Add the title of the manpage to TOC and sort alphabetically
Fixes #2987
2019-08-29 14:56:24 +02:00
Rusty Russell
5e3e2f4e17 run-secret_eq_consttime: try harder.
This will never be reliable under high load, without making it unable
to detect real errors.  But the test is useful because if we don't
have this test we'll never notice if we break the const-timedness of
our implementation.

So, move the calloc out of the test loop (which seems to make it more
reliable), and then after we've run it, check the 1-minute load
average.  Too high, we don't complain about results.  It's not
perfect, but it's better.

Running 100 times (-O3) serially gave 100 successes with the following results:
	Constant: Within 5% 562-926(832.89+/-73)/1000 times
        Non-constant: More than 5% slower 860-990(956.35+/-26)/1000 times

More importantly, if we swap the const and non-const tests, we get
the expected 100 failures:
	Non-constant: Within 5% 14-79(41.17+/-14)/1000 times
	Constant: More than 5% slower 44-231(111.89+/-33)/1000 times

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-29 09:01:48 +02:00
Rusty Russell
6901732ee0 lightningd: create --list-features-only which lists what features we support.
This allows the lightning-rfc protocol tests to automatically query what
features we support.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-29 09:01:48 +02:00
Rusty Russell
17edcfa24d features: don't assume we'll always only advertize optional features.
Generalize things a bit so OPTIONAL_FEATURE() and COMPULSORY_FEATURE()
work with either odd or even features, then explicitly use OPTIONAL_FEATURE
in our internal feature array.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-29 09:01:48 +02:00
Rusty Russell
a38131f349 EXPERIMENTAL: advertize the extended_queries feature.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-29 09:01:48 +02:00
Rusty Russell
d943d8abbc lightningd: expose full onion error when we have it.
Mainly useful for testing.  In particular, we don't save it to the db.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-29 09:01:48 +02:00
Rusty Russell
189b2f1313 BOLT: update CSV to latest bolt version.
This removes the WIRE_FINAL_EXPIRY_TOO_SOON which leaked too much info,
and adds the blockheight to WIRE_INCORRECT_OR_UNKNOWN_PAYMENT_DETAILS.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-29 09:01:48 +02:00
Rusty Russell
2600a6ed2e channeld: get current block height when an HTLC fails.
We need it to put in the error code for
WIRE_INCORRECT_OR_UNKNOWN_PAYMENT_DETAILS.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-29 09:01:48 +02:00
Rusty Russell
6349222ea2 Spec: Update to latest BOLT, include our first global feature definition.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-29 09:01:48 +02:00
Rusty Russell
6310a183af pytest: ignore occasional bad gossip in reorg tests.
It can happen, as expected when we reorg and update for a now-invalid
channel.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-28 04:04:28 +00:00
Rusty Russell
20a2bf9547 pytest: make test_forward_event_notification more explicit.
We seem to be getting intermittant failures, but it's hard
to disgnose.  Simplify it by moving all the test logic into
the test itself, and making the plugin dumber.  This means we'll
see exactly what the differences are if it fails again.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-28 04:04:28 +00:00
Rusty Russell
dbc1ebe06a Makefile: fix race when checking manpages.
By not depending on plugins, we can call lightningd before plugins are
built, and not get all options:

    diff of command names vs manpage names:
    --- /dev/fd/63  2019-08-22 05:06:55.265659216 +0000
    +++ /dev/fd/62  2019-08-22 05:06:55.265659216 +0000
    @@ -3,6 +3,8 @@
     allow-deprecated-apis=
     always-use-proxy=
     announce-addr=
    +autocleaninvoice-cycle=
    +autocleaninvoice-expired-by=
     autolisten=
     bind-addr=
     bitcoin-cli=

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-28 04:04:28 +00:00
Rusty Russell
394aef2391 Makefile: fix race with 'make check'
installcheck calls $(MAKE) under the covers, which can race with the
current builds, and we can try installing something which is still
being built:

    collect2: fatal error: ld terminated with signal 11 [Segmentation fault], core dumped
    compilation terminated.
    /usr/bin/ld: can not read symbols: File truncated
    /usr/bin/ld: .eh_frame/.stab edit: File truncated
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.interp' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.note.ABI-tag' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.note.gnu.build-id' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.gnu.hash' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.dynsym' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.dynstr' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.gnu.version' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.gnu.version_r' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.rela.dyn' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.rela.plt' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.init' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.plt' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.plt.got' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.text' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.fini' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.rodata' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.eh_frame' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.init_array' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.fini_array' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.data.rel.ro' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.dynamic' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.got' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `.data' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `xautodata_json_command' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `xautodata_hooks' not in segment
    /usr/bin/ld: lightningd/lightningd: warning: allocated section `xautodata_type_to_string' not in segment
    Makefile:390: recipe for target 'lightningd/lightningd' failed
    make[1]: *** [lightningd/lightningd] Error 1
    make[1]: Leaving directory '/home/rusty/lightning'
    Makefile:553: recipe for target 'installcheck' failed
    make: *** [installcheck] Error 2

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-28 04:04:28 +00:00
Rusty Russell
3f1ca18f2f pytest: optimize test_closing_torture
VALGRIND=1, SLOW_MACHINE=0:
Before: 197.74 seconds
After: 135.43 seconds

Note that we now spend about 13 seconds in teardown, could probably
be optimized.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-28 04:04:28 +00:00
Rusty Russell
686df235a2 pytest: fix race in test_block_backfill.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-28 04:04:28 +00:00
Rusty Russell
11833192a1 tools: fix Makefile warning
Makefile:424: warning: overriding recipe for target 'clean'
    tools/Makefile:12: warning: ignoring old recipe for target 'clean'

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-28 04:04:28 +00:00
Rusty Russell
28185c397c gossipd: fix gossip send in case query_flags cause no output.
Fortunately, again, only happens with EXPERIMENTAL_FEATURES.

If the query causes us not to actually send anything, we won't
get called again.  This can validly happen if they only asked for
the node_announcements, for example.

(Found by protocol tests).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2019-08-27 12:35:25 +02:00