Update testing doc for TestPad

This commit is contained in:
Devin Bileck 2019-02-23 00:05:07 -08:00
parent 25498a9e1a
commit b1c8e82b16
No known key found for this signature in database
GPG Key ID: 38750B26EA8B8C93

View File

@ -8,9 +8,7 @@ In order to take part in the testing process, you will need to do the following:
- Build Bisq from source (see [build.md](build.md))
- Setup a development/testing environment (see [dev-setup.md](dev-setup.md))
- Register an account with [TestQuality](https://bisq.testquality.com)
_**Note:** Once you have registered an account, or if you have trouble registering, contact us in Slack (see below) to be granted permissions to execute test runs._
- Request access to [TestPad](https://bisq.ontestpad.com) (our test management tool)
## Communication Channels
@ -18,51 +16,58 @@ If you would like to discuss and/or contribute to Bisq's testing effort, join us
## Compensation
Testing activities are eligible for [compensation](https://docs.bisq.network/dao/phase-zero.html#how-to-request-compensation). When submitting a compensation request, please include links to artifacts on TestQuality indicating the activities that were performed (e.g. test plan runs that were executed), as well as any bugs that were discovered and entered as a result of testing.
Testing activities are eligible for [compensation](https://docs.bisq.network/dao/phase-zero.html#how-to-request-compensation).
When submitting a compensation request, please include links to artifacts on TestPad (results/reports) indicating the activities that were performed (e.g. tests that were executed), as well as any bugs that were discovered and entered as a result of testing.
## Testing Process
[TestQuality](https://bisq.testquality.com) is used to manage and track the manual testing process. For specific usage or functionality of TestQuality, please see the guided tour or refer to the help content within TestQuality.
[TestPad](https://bisq.ontestpad.com) is used to manage and track the manual testing process.
For specific usage or functionality of TestPad, please see the flash card introduction within TestPad.
### Definitions
Some definitions within the context of TestQuality:
Some definitions within the context of TestPad and how they apply to our specific testing process:
- **Test Case:** a set of conditions under which a tester will determine whether the system under test satisfies requirements or works correctly.
- **Test Suite:** a collection of test cases that are intended to be used to test the system to show that it has some specified set of behaviours.
- **Test Plan:** defines a particular testing scope with testing activities.
- **Test Plan Run:** an occurrence in which a particular test plan is executed.
- **Project:** Defines a particular testing scope with relevant tests.
- **Script:** Each script is a collection of related tests that are intended to be used to test a particular component.
- **Folder:** Defines a group of scripts for each release.
### Test Structure
Tests are written using Behaviour-Driven Development (BDD) style syntax (given/when/then).
- **Given:** This states the preconditions that are assumed for the test. It is not a test step (one that requires a result to be recorded), but instead you must ensure the preconditions are satisfied in order to perform the test.
- **When:** This states the actions to be performed for the test. This also does not require a result to be recorded.
- **Then:** This states the expected results of the test. This requires a result to be recorded.
### Testing Workflow
Once logged in to TestQuality, select the Bisq project from the dashboard.
Once logged in to TestPad, select the `Desktop Client` project from the left navigation menu.
#### Executing a Test Plan Run
Each upcoming release will have a new folder created with applicable scripts that need to be executed.
A new test plan run is created every time a test plan needs to be executed. This allows for tracking the history of test plans executions.
#### Executing a Script
To execute a test plan run:
Test runs allow for tracking the results of test execution. Each script may have several test runs created in order to perform the tests on different environments (e.g. operating systems) and assigned to different people. An overview of all test runs for the release can be observed from the main project view, which allows you to quickly find test runs assigned to yourself.
1. Open the test plan run to be executed.
To execute a test run:
1. Select the "Running" state in order to update the start time (used for time tracking purposes).
1. Open the script to be executed.
1. Navigate the test suites and perform each test case.
1. Hover over the applicable test run column and select the play button to start executing the test run.
- Specify a status for each step or for the overall test case and optionally enter the time spent on each step. Select from one of the following statuses:
1. Follow the script and perform each test.
- **Pass:** the test case or step has passed successfully.
- **Pending:** the test case or step has yet to be performed.
- **Fail:** there is an issue (defect) related to the test case or step.
- **Block:** the test case or step is unable to be performed.
- **Retest:** the test case or step needs to be retested.
- **Skip:** the test case or step does not need to be performed.
- Select a status for each test. Select from one of the following statuses:
- If required, use the `Reason for Status` field to add any comments, notes and actual test results. This is especially beneficial to provide details if a test fails.
- **Pass:** the test has passed successfully.
- **Fail:** there is an issue (defect) related to the test.
- **Blocked:** the test cannot be performed for a particular reason.
- **Query:** you are unsure about the test and require further information.
- **Exclude:** the test does not need to be performed for a particular reason.
- If applicable, link an existing or create a new issue (defect) if it was found during the test plan run execution.
- If necessary, use the `Comments` field to add any comments, notes and actual test results. This is especially beneficial to provide details if a test did not pass.
1. Once all test cases within the test plan run have been executed, select the "Complete" state in order to update the end time.
- If applicable, link an existing or create a new issue (defect) if it was found during the test run execution.
### Creating Issues