This change makes configurable the amount of time (ms) each Bisq
instance is given to initialize before starting (a) another Bisq
instance, or (b) tests.
Some of these 'method' tests make more than one gRPC call, but
they are for checking correctness of a single gRPC method,
and don't quite fall into the 'scenario' test category.
Also added a new base ScenarioTest class, which extends
MethodTest; Method and Scenario tests cases have access to gRPC
service stubs, but end to end test cases should never use them.
This change replaces the non-intuitive numSetupTasks=<Integer>
configuration option with a supportingApps=<String> option. It
accepts a comma delimited list of app names, and determines
which background apps will be started in @BeforeClass.
None of the current method tests need all supporting apps, and
this change will reduce scaffolding setup and teardown time.
The current method test cases were changed to use this option.
WalletProtectionTest and GetVersionTest only need to start "alicedaemon".
GetBalanceTest needs "bitcoind,seednode,arbdaemon,alicedaemon".
API test cases are not in a maven/gradle project test folder.
IDEs may not automatically configure JUnit test launchers,
and a gradle build command will not automatically run tests.
This class is provided as a convenience until gradle tasks
for running test cases are implemented.
These @Order-ed test cases should be run with OrderedRunner.
They are called 'method' tests because they are the closest
thing to an API 'unit' test. Method tests are not end to end
tests; they have access to gRPC service stubs and do asserts
on Java values. We can write 'method' tests for the new
gRPC methods we need to implement (test driven development).
Some notes about the three API test categories...
The plan is to categorize some of the narrower functional tests
as 'scenario' tests. Like 'method' tests, 'scenario' tests will
have access to gRPC service stubs and do asserts on Java values,
but they will cover use cases requiring several gRPC and bitcoin-cli
method calls. One example is funding a wallet: using bitcoin-cli
and bisq-cli to generate new addresses, send BTC from bitcoin-core
to a Bisq wallet, generate a bitcoin block, and check the Bisq
wallet's balance. Another example is wallet unlocking/locking
scenarios that are a bit more complex than 'unit' or 'method' tests.
The third category of tests will be end to end, and live in an
'e2e' package. They will cover more complex use cases and
do asserts on the Bisq CLI's console output (stdout/stderr),
not return values from gRPC service stubs.
There may sometimes be a fine line between what is a 'scenario'
test and an 'e2e' test. But what starts out as a 'scenario' test
during test driven development can migrate to an 'e2e' test after
thorough testing in the quicker 'scenario' dev/test cycle.
Although JUnit is geared towards running randomly ordered unit
tests, we can use it to write API tests to be run in a pre-defined
order. This is necessary because API tests will be functional
(narrow to broad) and end-to-end; we want to be able to write
@Test methods that run in @Order.
The next commit will add a JUnit OrderedRunner class, and an @Order
annotation.
Many test cases will need to run on a clean, initialized dao-setup;
we do not want call dao-setup.gradle tasks from Java.
For now, every JUnit test case will refresh the dao-setup data in
@BeforeClass, but @AfterClass will not clean out the dao-setup files.
We want to be able to look at bitcoin-core and bisq app data & log
files after a test.
Given the difficulty of refreshing dao-setup files using Java,
I took a short cut and used the apitest.linux pkg's BashCommand
to replace dao-setup files.
Some bitcoin.conf and blocknotify configuratation logic was
also moved from ApiTestConfig to Scaffold in this commit.
Added :proto to the :apitest classpath for access to grpc
service stubs (to be) used in method (unit) tests. Added new
GrpcStubs class to expose the grpc service stubs to method and
scenario tests.
The larger goal of :apitest is end to end testing, where :cli's
console output is checked for correctness.
This change partially addresses two other important use cases:
* "method" testing -- an analog to unit testing
* "scenario" testing -- an analog to narrow functional testing
For example, tests in the apitest.method package will directly
call grpc services, and asserts will be made on the return values
instead of console output.
Tests in the apitest.scenario package will check correctness
for broader use cases, such as funding a wallet, encrypting then
unlocking a wallet for a specific time frame, or checking error
messages from the server when a "getbalance" call is made after
an "unlockwallet" timeout has expired.
The broader end to end tests will not use grpc stubs.
ApiTestMain will run all defined tests, but we also want to run
individual test suites and test cases, and they will need to
run the setup tasks as well.
SetupTask submissions for Bisq background apps seednode, arbnode,
etc., would not always complete due to a blocking stderr stream
handler thread.join() call. This change makes waiting on a bash
process err stream optional.
The `berkeleyDbLibPath` option now defaults to an empty string.
If not set to a berkeley lib path on the command line or the
apitest.properties file, this option is ignored, and 'bitcoind'
will be started without first exporting the berkeley db library
path.
In other words: If the bitcoind binary is dynamically linked to
berkeley db libs, export the configured berkeley-db lib path before
starting 'bitcoind'. If statically linked, the berkeley db lib
path will not be exported.
Also fixed exception msgs to show missing config file's absolute path.
Unnecessary use of fully qualified name 'System.exit' due to existing
static import 'java.lang.System.exit'. (line 100)
Avoid throwing raw exception types. (lines 295, 302)
This gradle file is 'applied' by the main build file.
Usage:
Run a full clean, build, download dao-setup.zip,
and install the zip files contents in directory
apitest/build/resources/main:
./gradlew clean build :apitest:installDaoSetup
Download (if necessary) the dao-setup.zip file
and install its contents in directory
apitest/build/resources/main (no build).
./gradlew :apitest:installDaoSetup
The driver class uses an ExecutorService to submit Callable
tasks for starting bitcoind and Bisq nodes as Linux background
processes.
By default, ApiTestMain starts background processes to support
regtest/dao testing, runs a few bitcoin-cli commands, then
shuts down all background processes before exiting.
(Actual API test suites have not been implemented yet.)
ApiTestConfig options can be used to skip tests and/or leave
background processes running indefinitely.
The apitest.linux package is for running random bash commands,
running 'bitcoind -regtest', running random 'bitcoin-cli -regtest'
commands, and spinning up Bisq apps such as seednode, arbnode,
and bob & alice nodes.
All but random bash and bitcoin-cli commands are run in the background.
The bitcoin-cli response processing is crude; a more sophiticated
bitcoin-core rpc interface is not in the scope of this PR.
ApiTestConfig works like :common Config, but specific to this subproject.
BisqAppConfig is an enumeration specifying Bisq desktop and daemon
options for running seednode, arbnode, bob & alices nodes in regtest /
full-dao mode.
