From 05d4e8519cecd00f76270e46952d78bef29d1552 Mon Sep 17 00:00:00 2001 From: Mark Beckwith Date: Fri, 20 Jul 2018 11:15:02 -0500 Subject: [PATCH] improved testing section I went to the Nakamoto dinner last week and told some guys they could get involved by improving our test coverage. So I updated the docs for newbs like me. (I only recently discovered `PYTEST_PAR`). Signed-off-by: Mark Beckwith --- doc/HACKING.md | 55 ++++++++++++++++++++++++++++++++++++-------------- 1 file changed, 40 insertions(+), 15 deletions(-) diff --git a/doc/HACKING.md b/doc/HACKING.md index e2b55e402..0bf46891b 100644 --- a/doc/HACKING.md +++ b/doc/HACKING.md @@ -90,7 +90,7 @@ Here's a list of parts, with notes: * closingd/ - daemon to handle mutual closing negotiation with a single peer. -* onchaind/ - daemon to hand a single channel which has had its funding +* onchaind/ - daemon to handle a single channel which has had its funding transaction spent. Debugging @@ -160,27 +160,52 @@ lightning funds. Testing ------- +Install `valgrind` and `pytest-xdist` for best results: -There are three kinds of tests. For best results, you should have -valgrind installed, and build with DEVELOPER=1. +``` +sudo apt install valgrind +pip3 install pytest-xdist +``` -* source tests - run by `make check-source`, looks for whitespace, +Tests are run with: `make check [flags]` where the pertinent flags are: + +``` +DEVELOPER=[0|1] - developer mode increases test coverage +VALGRIND=[0|1] - detects memory leaks during test execution but adds a significant delay +PYTEST_PAR=n - runs pytests in parallel +``` + +A modern desktop can build and run through all the tests in a couple of minutes with: + + make -j12 check PYTEST_PAR=24 DEVELOPER=1 VALGRIND=0 + +Adust `-j` and `PYTEST_PAR` accordingly for your hardware. + +There are three kinds of tests: + +* **source tests** - run by `make check-source`, looks for whitespace, header order, and checks formatted quotes from BOLTs if BOLTDIR exists (currently disabled, since BOLTs are being re-edited). -* unit tests - run by `make check`, these are `run-*.c` files in test/ - subdirectories which can test routines inside C source files. - You should insert `/* AUTOGENERATED MOCKS START */` and - `/* AUTOGENERATED MOCKS END */` lines, and `make update-mocks` - will automatically generate stub functions which will allow you to - link (which will conveniently crash if they're called). +* **unit tests** - standalone programs that can be run individually. + They are `run-*.c` files in test/ subdirectories used to test routines + inside C source files. + + You should insert the lines when implementing a unit test: + + `/* AUTOGENERATED MOCKS START */` + + `/* AUTOGENERATED MOCKS END */` + + and `make update-mocks` will automatically generate stub functions which will + allow you to link (and conveniently crash if they're called). + +* **blackbox tests** - These test setup a mini-regtest environment and test + lightningd as a whole. They can be run individually: -* blackbox tests - run by `make check` or directly as `PYTHONPATH=contrib/pylightning python3 tests/test_lightningd.py -f`. - You can run these much faster by putting `VALGRIND=0` after `make check`, - which has the added bonus of doing - memory leak detection. You can also append `LightningDTests.TESTNAME` - to run a single test. + + You can also append `LightningDTests.TESTNAME` to run a single test. Our Travis CI instance (see `.travis.yml`) runs all these for each pull request.