Without this, a stuck test (such as before the previous commit, where a plugin crashed
when running a command) simply gets timed out by the full CI timeout.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
There are several benefits of doing this:
- prevent fuzz target bit rot
- more test coverage in CI
- greater visibility of fuzz tests, encouraging contributions to the
seed corpus and tests themselves
We can require a status to be successful in PRs, but they are
referenced by name, and so we'd have to add each matrix job as
required. That's rather cumbersome, so have this artificial gather
step at the end which signals success on the entire run.
We often have a number of changes in flight, and we amend PRs before the previous run completes. This allows us to cancel that prior run, preserving our precious runners.
They are run in parallel with the integration tests, not on the
critical path, but can be a bit lengthy since they build a log of C
code, and run in valgrind.
The tester CI job uses absolute paths to ensure it is testing the
correct binaries. That clashes with the transfer between builder and
tester job using the `install` target because that switches things
around. This commit introduces a new target that just collects
artifacts in place, and tars them. Then we can use `tar` to unpack
them on the tester jobs again.
We want to compile with one set of dependencies, and run the tests
with another. This also helps us cut down on the times we compile CLN
itself, by sharing them among stages, and simplifies the logic of each
stage to have one specific goal.
The lnprototests are often blocking PRs, due to them failing, but we
can't restart them when valgrind runs are still ongoing, since they'd
also restart. Splitting allows us to rerun them selectively and waste
less time.
Ideally we'd just fix them, but I am by no means knowledgeable enough
to fix them now.
We used to install it only when Rust was configured, but for some
reason all builds now seem to be Rust builds. That's ok, so just
provide the necessary dependency.
The runner version was recently bumped causing our tests to fail
because it couldn't find the `poetry` version it just installed. We
instead install python3.10, and force its use, since otherwise we end
up compiling grpcio and fail to do so.
Changelog-None
I was wondering why the workflow never published, turns out we were triggering on
branch pushes only, not tags. So the branch would get pushed, but the tag is pushed
afterwards, and thus not triggering.
Requires us to update to latest lnproto which is now using the most up
to date python-bitcoinlib, as well as updating our python lock files
(which pin the grpcio deps, because of locking problems h/t @cdecker)
If compilation failed, we didn't stop (though except for fuzzing, we
would fail when we try to run the tests).
Also use make -s instead of redirecting make ooutput.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Over time, it has cost us more developer cycles than it has gained.
It has hidden intermittant bugs, and allowed cruft to accumulate:
when we eventually tried to figure out what was going wrong, the
actual change which caused it was now stale and forgotten.
This was a particular bane during the connectd rewrite, and I
worked through some issues which had occurred before, but were not
more likely.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
For some reason Github complains that @wythe doesn't have access to
the public repository, removing in order to silence that warning. Feel
free to contact me to investigate what's happening and re-enable :-)
Changelog-None
I have no idea why someone else owns the directory suddenly, but all git
commands fail. Workaround as suggested by the error message.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Mostly comments and docs: some places are actually paths, which
I have avoided changing. We may migrate them slowly, particularly
when they're user-visible.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
check-dbstmts was just running the normal pytest, AFAICT:
```
export TEST_CHECK_DBSTMTS=0
+ TEST_CHECK_DBSTMTS=0
```
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
```
[gw1] [ 98%] PASSED tests/test_wallet.py::test_hsmtool_dump_descriptors
tests/test_wallet.py::test_fundchannel_listtransaction
[gw0] [ 98%] PASSED tests/test_plugin.py::test_channel_opened_notification
tests/test_wallet.py::test_hsmtool_generatehsm
[gw0] [ 98%] PASSED tests/test_wallet.py::test_hsmtool_generatehsm
tests/test_wallet.py::test_withdraw_nlocktime_fuzz
[gw1] [ 98%] ERROR tests/test_wallet.py::test_fundchannel_listtransaction
tests/test_wallet.py::test_fundchannel_listtransaction
tests/test_wallet.py::test_withdraw_nlocktime_fuzz
tests/test_wallet.py::test_fundchannel_listtransaction
[gw0] [ 99%] ERROR tests/test_wallet.py::test_withdraw_nlocktime_fuzz
tests/test_wallet.py::test_multiwithdraw_simple
[gw1] [ 99%] ERROR tests/test_wallet.py::test_fundchannel_listtransaction
tests/test_wallet.py::test_withdraw_nlocktime
tests/test_wallet.py::test_multiwithdraw_simple
tests/test_wallet.py::test_withdraw_nlocktime
tests/test_wallet.py::test_multiwithdraw_simple
tests/test_wallet.py::test_withdraw_nlocktime
[gw0] [ 99%] ERROR tests/test_wallet.py::test_multiwithdraw_simple
tests/test_wallet.py::test_repro_4258
[gw1] [ 99%] ERROR tests/test_wallet.py::test_withdraw_nlocktime
...
2021-10-12 06:36:09.203 UTC [224552] STATEMENT: SELECT version FROM version LIMIT 1
2021-10-12 06:36:09.566 UTC [224523] PANIC: could not write to file "pg_wal/xlogtemp.224523": No space left on device
2021-10-12 06:36:09.566 UTC [224523] STATEMENT: VACUUM FULL;
Error vacuuming db: BEGIN command failed: PANIC: could not write to file "pg_wal/xlogtemp.224523": No space left on device
server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
```
Setting VALGRIND=1 actually does nothing here; reduce it to two cases,
covering gcc and clang, sqlite3 and postgres.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Add an extra run configuration for the proto-tests.
Proto-tests require DEVELOPER=1; enabling EXPERIMENTAL_FEATURES=1
by default ensures that experimental additions are automatically put
under test also.
It's been causing me quite some headache, and I don't see the point in
jumping through the hoops for something that can be trivially fixed by
having the required build tools.
A new target `check-gen-updated` to verify that all derived/generated
files that were modified were also checked in. This is used on CI to
check, and is not added to `check` since it'll complain on dirty
trees, i.e., before the devs check in their changes.
Suggested-by: Rusty Russell <@rustyrussell>
We split into quick smoke-tests, normal-tests, and valgrind-tests. The
first provides a quick-abort preventing overusing resources on what
will be a failed run anyway.
We'd like to have some more structure to the tests, with smoke-tests
going first, followed by valgrind and normal tests running in
parallel. So to cut down on the copy-pasta we extract the scripts from
the yaml, into their own files from where they can be easily invoked.
We don't have a good way of referring to the configuration that
failed, so let's give them a numberic ID. Particularly useful for the
artifacts that'd be overwritten otherwise.
Changelog-Added: We now have `multifundchannel` as a builtin plugin command to fund multiple channels to different peers all in a single onchain transaction.
Header from folded patch 'fixup-use-json_add_psbt.patch':
fixup!
Header from folded patch 'use-goto-no-ok-chain.patch':
fixup!
Header from folded patch 'destinations-at-parse-time.patch':
fixup!
Header from folded patch 'multifundchannel__use_jsmntoks_to_pass_through_json_string,_not_strings.patch':
multifundchannel: use jsmntoks to pass through json string, not strings
Passing in "" for utxos would crash lightningd on the command-line
otherwise. Now returns an error.
Header from folded patch 'update_plugins-multifundchannel.c.patch':
Update plugins/multifundchannel.c
Co-authored-by: Darosior <darosior@protonmail.com>
