mirrors/bitcoin
1
0
Fork 0
mirror of https://github.com/bitcoin/bitcoin.git synced 2024-11-19 01:42:58 +01:00
Code Issues Projects Releases Wiki Activity

Merge bitcoin/bitcoin#30244: ci: parse TEST_RUNNER_EXTRA into an array

Browse Source 
8131bf7483 ci: parse TEST_RUNNER_EXTRA into an array (Max Edwards)
c4762b0aa0 test: allow excluding func test by name and arg (Max Edwards)

Pull request description:

  While working on CI I wanted to disable some functional tests so I used the `TEST_RUNNER_EXTRA` var. The problem I had was tests that have flags such as `rpc_bind.py --ipv6` must be passed in quotes otherwise the `--ipv6` portion will be considered an argument to `test_runner.py` rather than a test name.

  This change allows proper parsing of quotes and complex values such as:

  ```shell
  TEST_RUNNER_EXTRA='--exclude "rpc_bind.py --ipv6,feature_proxy.py"'
  ```

  Update:

  While testing this it was noticed that `test_runner.py` when given `--exclude "rpc_bind.py --ipv6"` will exclude all `rpc_bind.py` tests so this PR has been updated to include a change to the test runner to only exclude the specific test if you pass an arg or exclude all tests of that name if you do not pass an arg. `--exclude rpc_bind.py` will exclude all three variants and `--exclude rpc_bind --ipv6` will only exclude the IPV6 variant.

ACKs for top commit:
  maflcko:
    ACK 8131bf7483
  achow101:
    ACK 8131bf7483
  hebasto:
    ACK 8131bf7483, tested on Ubuntu 23.10 and Windows 11.

Tree-SHA512: 82b73f12d627f533d8e5be4a518d455ef4427a755bbe03ccd11d0bb70c7ff3cee76220b0264fcfb236661c4cf5deba034cbfc2372b96d5861f3436c21eae8264
This commit is contained in:
Ava Chow 2024-09-04 15:55:42 -04:00
commit 93e48240bf
No known key found for this signature in database
GPG Key ID: 17565732E08E5E41
2 changed files with 18 additions and 9 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
ci/test/03_test_script.sh
View File

@ -146,8 +146,9 @@ if [ "$RUN_UNIT_TESTS_SEQUENTIAL" = "true" ]; then
fi
if [ "$RUN_FUNCTIONAL_TESTS" = "true" ]; then
# shellcheck disable=SC2086
LD_LIBRARY_PATH="${DEPENDS_DIR}/${HOST}/lib" test/functional/test_runner.py --ci "${MAKEJOBS}" --tmpdirprefix "${BASE_SCRATCH_DIR}"/test_runner/ --ansi --combinedlogslen=99999999 --timeout-factor="${TEST_RUNNER_TIMEOUT_FACTOR}" ${TEST_RUNNER_EXTRA} --quiet --failfast
# parses TEST_RUNNER_EXTRA as an array which allows for multiple arguments such as TEST_RUNNER_EXTRA='--exclude "rpc_bind.py --ipv6"'
eval "TEST_RUNNER_EXTRA=($TEST_RUNNER_EXTRA)"
LD_LIBRARY_PATH="${DEPENDS_DIR}/${HOST}/lib" test/functional/test_runner.py --ci "${MAKEJOBS}" --tmpdirprefix "${BASE_SCRATCH_DIR}"/test_runner/ --ansi --combinedlogslen=99999999 --timeout-factor="${TEST_RUNNER_TIMEOUT_FACTOR}" "${TEST_RUNNER_EXTRA[@]}" --quiet --failfast
fi
if [ "${RUN_TIDY}" = "true" ]; then

22
test/functional/test_runner.py
View File

@ -523,14 +523,22 @@ def main():
# Remove the test cases that the user has explicitly asked to exclude.
if args.exclude:
exclude_tests = [test.split('.py')[0] for test in args.exclude.split(',')]
def print_warning_missing_test(test_name):
print("{}WARNING!{} Test '{}' not found in current test list.".format(BOLD[1], BOLD[0], test_name))
exclude_tests = [test.strip() for test in args.exclude.split(",")]
for exclude_test in exclude_tests:
# Remove <test_name>.py and <test_name>.py --arg from the test list
exclude_list = [test for test in test_list if test.split('.py')[0] == exclude_test]
for exclude_item in exclude_list:
test_list.remove(exclude_item)
if not exclude_list:
print("{}WARNING!{} Test '{}' not found in current test list.".format(BOLD[1], BOLD[0], exclude_test))
if exclude_test.endswith('.py'):
# Remove <test_name>.py and <test_name>.py --arg from the test list
exclude_list = [test for test in test_list if test.split('.py')[0] == exclude_test.split('.py')[0]]
if not exclude_list:
print_warning_missing_test(exclude_test)
for exclude_item in exclude_list:
test_list.remove(exclude_item)
else:
try:
test_list.remove(exclude_test)
except ValueError:
print_warning_missing_test(exclude_test)
if args.filter:
test_list = list(filter(re.compile(args.filter).search, test_list))