Commit Graph

229 Commits

Author SHA1 Message Date
MarcoFalke
faba4672b6
ci: Fix QEMU_USER_CMD parse issues
* Fix bash syntax by adding missing -

* Disable QEMU_USER_CMD fallback when it is set

  Apparently bash can't distinguish an unset variable from a variable
  that is set to the empty string

* Export the environment variable to the docker env, otherwise it
  couldn't be used there
2020-01-03 07:04:45 -05:00
MarcoFalke
fa5d709fb2
ci: Move wrap-qemu into separate script 2020-01-03 07:03:46 -05:00
Hennadii Stepanov
87744b16b0
ci: Fix brew python link 2020-01-02 18:08:10 +02:00
MarcoFalke
aaaaad6ac9
scripted-diff: Bump copyright of files changed in 2019
-BEGIN VERIFY SCRIPT-
./contrib/devtools/copyright_header.py update ./
-END VERIFY SCRIPT-
2019-12-30 10:42:20 +13:00
MarcoFalke
890eac8f82
Merge #17725: ci: Add valgrind run
facb416ad5 ci: Add valgrind run (MarcoFalke)

Pull request description:

  Fixes #17460

ACKs for top commit:
  practicalswift:
    ACK facb416ad5

Tree-SHA512: 55396e548a76f976d7b7170b68bc5f93cfd44656162267172f66db7eb549699a2a22d3b1bb0d5f180fe0697931939e652c8cdb86b435e81e7ce572485798009d
2019-12-17 11:46:22 -05:00
Elichai Turkel
e190000869
ci-s390x: Add qemu and depends support in the ci script 2019-12-14 18:27:13 +02:00
MarcoFalke
facb416ad5
ci: Add valgrind run 2019-12-10 19:37:37 -05:00
MarcoFalke
fc0da027e8
Merge #17635: ci: Add CentOS 7 build
711e0449cf ci: Remove trusty build (Hennadii Stepanov)
7f3ae22468 ci: Add CentOS 7 build (Hennadii Stepanov)

Pull request description:

  Arguably, CentOS is the most conservative distro of all the popular ones. Thus, it could be a good way to check the Bitcoin Core compatibility with aged dependencies.

  Currently, CentOS 7 has:
  - Berkeley DB == 4.8.30
  - Boost == 1.53.0
  - GCC == 4.8.5
  - libevent == 2.0.21 < minimum required [2.0.22](https://github.com/bitcoin/bitcoin/blob/master/doc/dependencies.md), but tests passed
  - MiniUPnPc == 2.0
  - Python == 3.6.8
  - qrencode == 3.4.1
  - Qt == 5.9.7
  - ZeroMQ == 4.1.4

  ~Please note that this PR is based on the bugfix #17634.~

  Also trusty build has been removed for the following reasons:
  - https://github.com/bitcoin/bitcoin/issues/17628#issuecomment-559448201:
  > Maybe it'd make sense to replace Ubuntu Trusty with Centos 7 as the "check ancient backward compatibililty" Travis run. It's supported until 2024, apparently.

  - https://github.com/bitcoin/bitcoin/pull/17635#discussion_r354811792:
  > Our travis is currently running at its limit and this doesn't seem like it is adding a lot new coverage compared to the other builds.

  Close #17628

ACKs for top commit:
  MarcoFalke:
    ACK 711e0449cf 🚠

Tree-SHA512: 614ec8394943f482a5867067f7119bffd052924a51e32ffda9a08e10c392c4a955a3539e2f8907cb65bfd9347dadf0ba62f6d1530bbc49927c347360a5a7f73c
2019-12-06 23:32:21 -05:00
Hennadii Stepanov
711e0449cf
ci: Remove trusty build
Co-authored-by: Emil Engler <me@emilengler.com>
2019-12-06 21:54:37 +02:00
Hennadii Stepanov
7f3ae22468
ci: Add CentOS 7 build 2019-12-06 21:44:35 +02:00
practicalswift
1f9d5af4f1 tests: Add initialization order fiasco detection in Travis 2019-12-05 10:54:18 +00:00
MarcoFalke
fa40e48c50
ci: Remove unparseable lines from supp file for old xenial clang tsan 2019-12-04 12:16:52 -05:00
MarcoFalke
fa1bfc476c
ci: ubsan report_error_type=1 and add suppressions 2019-12-04 12:16:38 -05:00
MarcoFalke
faa8023ce9
ci: Bump to clang-8 for asan build to avoid segfaults on ppc64le 2019-12-04 09:32:15 -05:00
MarcoFalke
1705f1949a
Merge #17549: ci: misc cleanups
fad88e6f86 ci: Remove use of cd (MarcoFalke)
fa2941bbf4 ci: Remove unmaintained extended_lint (MarcoFalke)
fa04187528 scripted-diff: Use ci DEPENDS_DIR, remove BASE_BUILD_DIR (MarcoFalke)
fa0656d1eb ci: Add DEPENDS_DIR variable, Add documentation for folders (MarcoFalke)
faeeca87b6 scripted-diff: Move various folders to ci scratch dir (MarcoFalke)

Pull request description:

  Some minor cleanups:

  * Remove unused and unmaintained extended_lint to avoid ci bottlenecks and waste of CPU
  * Move all folders that hold temporary ci files to the ci scratch dir (except for the build dirs)
  * Add some documentation to folders and remove the `BASE_BUILD_DIR` alias for the root directory
  * Fixes #17178 by removing `cd`

Top commit has no ACKs.

Tree-SHA512: f6eb9d47ab98c08af59a79c2a4bf62fc331f0f6f3174909a28f2c2f48b0234423da5aee876225d2e188619b71b008d882a20d29a7aca68248039ea5080be5af5
2019-12-03 11:02:06 -05:00
MarcoFalke
69ca6cdd66
Merge #17599: ci: Run functional tests on s390x
fabd71076c ci: Print free disk space (MarcoFalke)
fad9fdbea5 test: Properly deserialize integers in little-endian (MarcoFalke)
fa94fc10c8 ci: Run functional tests on s390x (MarcoFalke)

Pull request description:

Top commit has no ACKs.

Tree-SHA512: 98ba77eb56f283131fdaeb393fda86cc308f1bf9781e1e0e5736b8d616528dc8ff2e494d55ba107c138083025c66a59e382fcfa9962d4349a5fd6cbbc52484c3
2019-12-03 10:34:11 -05:00
Wladimir J. van der Laan
1f59885d27
Merge #17361: script: Lint Gitian descriptors with ShellCheck
17f81e9648 script: Enable SC2001 rule for Gitian scripts (Hennadii Stepanov)
61bb21b418 script: Enable SC2155 rule for Gitian scripts (Hennadii Stepanov)
577682d9e8 script: Enable SC2006 rule for Gitian scripts (Hennadii Stepanov)
14aded46df script: Lint Gitian descriptors with ShellCheck (Hennadii Stepanov)

Pull request description:

  This PR extracts shell scripts from Gitian descriptors (`contrib/gitian-descriptors/`) and checks for ShellCheck warnings as any other one.

  Some non-controversial warnings are fixed.

ACKs for top commit:
  practicalswift:
    ACK 17f81e9648 -- diff looks correct

Tree-SHA512: bdfa3d35bbb65ff634c90835d75c3df63e958b558599771d21366724f5cf64da83a68957d926e926a99c3704b9529e96a17697dc8d9ff3adf7154d9cb1999a8d
2019-11-28 10:18:22 +01:00
MarcoFalke
fabd71076c
ci: Print free disk space 2019-11-25 16:23:36 -05:00
MarcoFalke
fa94fc10c8
ci: Run functional tests on s390x 2019-11-25 15:07:28 -05:00
Elichai Turkel
2fa65e0de9
Add ci script to install on s390x 2019-11-25 14:34:47 +02:00
MarcoFalke
fad88e6f86
ci: Remove use of cd 2019-11-22 13:57:55 -05:00
fanquake
7d7bf2ff4a
build: set minimum supported macOS to 10.12 2019-11-21 11:08:47 -05:00
MarcoFalke
fa2941bbf4
ci: Remove unmaintained extended_lint 2019-11-21 11:03:05 -05:00
MarcoFalke
fa04187528
scripted-diff: Use ci DEPENDS_DIR, remove BASE_BUILD_DIR
-BEGIN VERIFY SCRIPT-
 # Use ci DEPENDS_DIR
 sed -i -e 's|BASE_BUILD_DIR/depends|DEPENDS_DIR|g' $(git grep -l depends ci)
 sed -i -e 's| depends/| ${DEPENDS_DIR}/|g'         $(git grep -l depends ci)

 # Remove redundant alias
 sed -i -e 's|BASE_BUILD_DIR|BASE_ROOT_DIR|g'       $(git grep -l BASE_BUILD_DIR ci)
-END VERIFY SCRIPT-
2019-11-21 11:01:34 -05:00
MarcoFalke
fa0656d1eb
ci: Add DEPENDS_DIR variable, Add documentation for folders 2019-11-21 11:01:17 -05:00
MarcoFalke
faeeca87b6
scripted-diff: Move various folders to ci scratch dir
-BEGIN VERIFY SCRIPT-
 # move ci sanitizer-output
 sed -i -e 's|BASE_BUILD_DIR}/sanitizer-output|BASE_SCRATCH_DIR}/sanitizer-output|g' $(git grep -l BASE_BUILD_DIR ci)
 # move qa-assets
 sed -i -e 's|BASE_BUILD_DIR}/qa-assets|BASE_SCRATCH_DIR}/qa-assets|g'               $(git grep -l BASE_BUILD_DIR ci)
 # move out dir
 sed -i -e 's|BASE_BUILD_DIR/out|BASE_SCRATCH_DIR/out|g'                             $(git grep -l BASE_BUILD_DIR ci)
-END VERIFY SCRIPT-
2019-11-20 19:56:26 -05:00
MarcoFalke
41162c0ba0
Merge #17423: ci: Make ci system read-only on the git work tree
fa7523d3aa ci: Extend docs (MarcoFalke)
fa493ef088 ci: Make ci system read-only on the git work tree (MarcoFalke)
fab1333292 ci: Remove git from required packages on host (MarcoFalke)
fa00393bce ci: Make all filesystem operations inside docker (MarcoFalke)

Pull request description:

  Running the ci completely in a docker, without leaving any traces on the host system is not possible right now because the ccache and depends dir needs to be propagated back and picked up by the host for caching.

  Fixes #17372

ACKs for top commit:
  JeremyRubin:
    tested ACK fa7523d3aa

Tree-SHA512: 4bce1a0f883bcbdb34abf409bdbc80d420c5da2045d2f9c5536ac433f9e5b490f23df084546c8c049f688b487572bbfc4f9c4029e9e672f4d9279739d066ed2e
2019-11-20 16:41:55 -05:00
fanquake
76e777df83
Merge #16161: util: Fix compilation errors in support/lockedpool.cpp
30fb598737 Fix segfault in allocator_tests/arena_tests (Jeffrey Czyz)
15c84f53f4 Define ARENA_DEBUG in Travis test runs (Jeffrey Czyz)
ad71548822 Fix compilation errors in support/lockedpool.cpp (Jeffrey Czyz)

Pull request description:

  Changes in #12048 cause a compilation error in Arena::walk() when
  ARENA_DEBUG is defined. Specifically, Arena's chunks_free map was
  changed to have a different value type.

  Additionally, missing includes cause other compilation errors when
  ARENA_DEBUG is defined.

  Reproduced with:

  make CPPFLAGS=-DARENA_DEBUG

ACKs for top commit:
  laanwj:
    ACK 30fb598737
  fanquake:
    ACK 30fb598737 - thanks for following up jkczyz.

Tree-SHA512: 4eec368a4e9c67e4e2a27bc05608a807c2892d50c60d06ed21490cd274c0369f9671bc05b3006acc2a193316caf4896454c9c299603bfed29bd488f1987ec446
2019-11-20 09:54:31 -05:00
Wladimir J. van der Laan
ea3c7e585c test: Remove libssl-dev packages from CI scripts 2019-11-19 15:04:43 +01:00
fanquake
397dbae070
ci: remove OpenSSL installation 2019-11-18 08:56:48 -05:00
Jeffrey Czyz
15c84f53f4 Define ARENA_DEBUG in Travis test runs
The definition and uses of Arena::walk() are compiled only if
ARENA_DEBUG is defined. Configure Travis to define ARENA_DEBUG so
compilation errors do not go unnoticed.
2019-11-16 08:45:21 -08:00
MarcoFalke
fa6e01b2f3
ci: Use clang-8 for fuzzing to run on aarch64 ci systems 2019-11-13 17:53:04 -05:00
MarcoFalke
fa7523d3aa
ci: Extend docs 2019-11-10 22:30:13 -05:00
MarcoFalke
fa493ef088
ci: Make ci system read-only on the git work tree 2019-11-09 09:13:41 -05:00
MarcoFalke
fab1333292
ci: Remove git from required packages on host 2019-11-09 09:12:24 -05:00
MarcoFalke
fa00393bce
ci: Make all filesystem operations inside docker 2019-11-09 09:11:32 -05:00
MarcoFalke
faf757a125
ci: Guess the native host when not cross compiling 2019-11-07 13:42:57 -05:00
Hennadii Stepanov
14aded46df
script: Lint Gitian descriptors with ShellCheck 2019-11-06 15:10:11 +02:00
MarcoFalke
fa8a60bce9
ci: Run non-cross-compile builds natively 2019-11-04 14:10:24 -05:00
MarcoFalke
fa56bcbb01
ci: Run CI_WAIT only on travis 2019-11-04 14:10:07 -05:00
MarcoFalke
def69e5a86
Merge #17233: travis: Run unit and functional tests on native arm
facc0da63a travis: Run unit and functional tests on native arm (MarcoFalke)
fafa064d2a ci: Remove ccache requirement on the host (MarcoFalke)

Pull request description:

  This keeps the cross-compilation to make it easy to run the ci on non-arm hardware. To run this locally in qemu-user as it used to be, just `export QEMU_USER_CMD="qemu-arm -L /usr/arm-linux-gnueabihf/"`.

ACKs for top commit:
  laanwj:
    LGTM ACK facc0da63a
  practicalswift:
    ACK facc0da63a -- diff looks correct and Travis seems happy

Tree-SHA512: 0dc1bc82eb93e2bd8b159e044f20fe3055f8cdfd73aaa238bd2e178397582144dfc0c6a87bd8270115dafea1a623e642bde5d5f30254f94140f1a2cdb12fc2da
2019-11-04 08:23:21 -05:00
randymcmillann
58d0393bec
build: update retry to current version
from: 7d2caa94a7
2019-10-30 18:49:57 -04:00
MarcoFalke
facc0da63a
travis: Run unit and functional tests on native arm 2019-10-30 14:45:32 -04:00
MarcoFalke
fafa064d2a
ci: Remove ccache requirement on the host
ccache is only needed to create the cache dir on the host, if it didn't
already exist. The same can be achieved with mkdir, so just use that
instead.
2019-10-24 18:37:38 -04:00
fanquake
3548e4aac7
Remove BIP70 Support 2019-10-24 16:01:43 -04:00
fanquake
67328bb7ca
build: remove protobuf from depends 2019-10-24 16:01:42 -04:00
MarcoFalke
fa710066b9
ci: Disable functional tests on mac host 2019-10-24 12:43:55 -04:00
MarcoFalke
4aaeb04497
Merge #17205: ci: Enable address sanitizer (ASan) stack-use-after-return checking
8d22ab0e50 ci: Enable address sanitizer (ASan) stack-use-after-return checking (practicalswift)

Pull request description:

  Enable address sanitizer (ASan) stack-use-after-return checking (`detect_stack_use_after_return=1`).

  Example:

  ```
  #include <iostream>
  #include <string>

  const std::string& get_string(int i) {
      return std::to_string(i);
  }

  int main() {
      std::cout << get_string(41) << "\n";
  }
  ```

  Without address sanitizer (ASan) stack-use-after-return checking:

  ```
  $ ./stack-use-after-return

  $
  ```

  With address sanitizer (ASan) stack-use-after-return checking:

  ```
  $ ASAN_OPTIONS="detect_stack_use_after_return=1" ./stack-use-after-return
  =================================================================
  ==10400==ERROR: AddressSanitizer: stack-use-after-return on address 0x7f7fa0400030 at pc 0x00000049d2cc bp 0x7ffcbd617070 sp 0x7ffcbd616820
  READ of size 2 at 0x7f7abbecd030 thread T0
      #0 0x439781 in fwrite
      #1 0x7f7ac0504cb3 in std::basic_ostream<char, std::char_traits<char> >& std::__ostream_insert<char, std::char_traits<char> >(std::basic_ostream<char, std::char_traits<char> >&, char const*, long) (/usr/lib/x86_64-linux-gnu/libstdc++.so.6+0x113cb3)
      #2 0x4f9b5f in main stack-use-after-return.cpp:9:15
      #3 0x7f7abf440b96 in __libc_start_main
      #4 0x41bbc9 in _start
  …
  $
  ```

Top commit has no ACKs.

Tree-SHA512: 6557a9ff184023380fd9aa433cdf413e01a928ea99dbc59ec138e5d69cb9e13592e8bb5951612f231ff17a37a895bec5c0940c8db5f328a5c840a5771bdeeba5
2019-10-22 13:14:16 -04:00
practicalswift
8d22ab0e50 ci: Enable address sanitizer (ASan) stack-use-after-return checking 2019-10-21 10:04:28 +00:00
MarcoFalke
fa677d1801
ci: Remove redundant check for TRAVIS_OS_NAME
Can be reviewed with
git diff --ignore-all-space --function-context
2019-10-17 11:30:36 -04:00
MarcoFalke
4444704ca9
ci: Cleanup macOS runs 2019-10-17 10:47:02 -04:00
Sjors Provoost
1f6c650c99
travis: run tests on macOS native
Review hint:
git show -w

Co-authored-by: MarcoFalke <falke.marco@gmail.com>
Co-authored-by: keneanung <keneanung@googlemail.com>
Co-authored-by: Vadim Peretokin <vperetokin@gmail.com>
2019-10-14 23:00:49 +02:00
MarcoFalke
42d0eca725
Merge #13728: lint: Run the CI lint stage on mac
cd82f75a43 lint: Install grep and git via brew on mac for --perl-regexp (Ben Woosley)
eafa747ca5 lint: Fix shellcheck SC2155 (Ben Woosley)
615ff4e7db lint: Run the linters against Mac OS on Travis (Ben Woosley)

Pull request description:

  This helps ensure ongoing compatibility with macOS-distributed version of GNU bash.

ACKs for top commit:
  MarcoFalke:
    ACK cd82f75a43

Tree-SHA512: 8d56d2303bbebedba8ea2291f4ab35b7fdf3245b7a4c3f04557eee4f19d83573798ad32facc92bfa060aaeb294e6d2c95e6d1c3b795fd7951dcf3aa1cccec107
2019-10-09 14:37:53 -04:00
Ben Woosley
cd82f75a43
lint: Install grep and git via brew on mac for --perl-regexp
Particularly `--with-pcre2` is needed to run `git grep --perl-regexp` in
`test/link/check-doc.py`
2019-10-09 15:50:42 +01:00
Ben Woosley
eafa747ca5
lint: Fix shellcheck SC2155
Declare and assign separately to avoid masking return values.

https://github.com/koalaman/shellcheck/wiki/SC2155
2019-10-09 15:50:42 +01:00
Ben Woosley
615ff4e7db
lint: Run the linters against Mac OS on Travis
This helps ensure ongoing compatibility with macOS-distributed version
of GNU bash.
2019-10-09 15:49:51 +01:00
MarcoFalke
fa79dff624
ci: Run tests on arm 2019-10-09 09:33:07 -04:00
Wladimir J. van der Laan
34919e0033
Merge #17011: ci: Use busybox utils for one build
ddddd8961b ci: Use busybox utils for one build (MarcoFalke)

Pull request description:

  To make sure Bitcoin Core can be built with BusyBox, see https://github.com/bitcoin/bitcoin/pull/16927#issuecomment-536483706

ACKs for top commit:
  laanwj:
    ACK ddddd8961b

Tree-SHA512: da3a4654ee7975206d04643675d309b4973a510ca344acaec97fb1ed19c43cf13489bdf236c92c4a90499ec5b3c18c3338fff096110b26abee5ffe955089f267
2019-10-08 11:31:48 +02:00
MarcoFalke
fa95503d64
ci: Make apt-get more verbose, to debug travis timeouts 2019-10-03 10:50:33 -04:00
MarcoFalke
ddddd8961b
ci: Use busybox utils for one build 2019-10-02 10:01:04 -04:00
practicalswift
aa81e2cc0e Enable UBSan for Travis fuzzer job 2019-10-01 08:23:34 +00:00
MarcoFalke
4c4ff4911a
Merge #16961: test: Remove python dead code linter
f4beb4996d test: Remove python dead code linter (Wladimir J. van der Laan)

Pull request description:

  Primarily I'd like to remove this because it is very imprecise, due to Python's dynamic nature, giving it a large list of false positives that need to be listed as exceptions. See for example #16906.

  It's also a frequent source of complaints. I'm doubtful of the usefulness of checking for dead code in a linter in the first place.
  Having some dead code in the test framework for a while is not a
  disaster.

ACKs for top commit:
  sdaftuar:
    utACK f4beb4996d
  practicalswift:
    ACK f4beb4996d -- diff looks correct
  jamesob:
    ACK f4beb4996d

Tree-SHA512: 329b1555210311d5d15799fd2cb794b3208b0ac4d8a2ffaf4dece1bcc3e0e8b1fe952d5e7a394f94a98919cab579fb579eae7db2a796cc9a1a42ef495dd17507
2019-09-25 13:14:22 -04:00
Wladimir J. van der Laan
f4beb4996d test: Remove python dead code linter
Primarily I'd like to remove this because it is very imprecise, due to
Python's dynamic nature, giving it a large list of false positives that
need to be listed as exceptions. See for example #16906.

It's also a frequent source of complaints. I'm doubtful of the
usefulness of checking for dead code in a linter in the first place.
Having some dead code in the test framework for a while is not a
disaster.
2019-09-25 11:16:09 +02:00
MarcoFalke
fadd76acc2
ci: Remove TRAVIS env vars 2019-09-24 16:43:44 -04:00
MarcoFalke
fa449b89b5
ci: Set $HOST before setting fallback values 2019-09-24 16:21:49 -04:00
MarcoFalke
fac35b21e2
test: lint: Add DisabledOpcodeTemplates to whitelist
Also, bump vulture version to include the whitelist for threading module
2019-09-18 11:38:37 -04:00
fanquake
107e030723
build: make protobuf optional in depends
Those that want to build it can now pass PROTOBUF=1.
2019-09-14 12:58:59 +08:00
fanquake
376f4929f8
build: disable BIP70 support by default 2019-09-12 20:28:17 +10:00
Ben Woosley
0ef0e51fe4
lint: Bump flake8 to 3.7.8 2019-09-03 14:41:43 -04:00
MarcoFalke
fa27372e65
ci: Move CCACHE_DIR and test_runner tmp dir into ./ci/scratch/ 2019-08-27 15:58:50 -04:00
MarcoFalke
fa60583d23
ci: Pass down $MAKEJOBS to test_runner.py 2019-08-27 15:58:49 -04:00
MarcoFalke
8b42db130f
Merge #16623: ci: Add environment files for all settings
fa21737ba7 ci: Add environment files for all settings (MarcoFalke)

Pull request description:

  This moves all environment settings from travis to files in the ci folder. Now, it is possible to easily run each travis configuration with a single command.

Top commit has no ACKs.

Tree-SHA512: 989c6b62eb3839eb1fa5461e986496e9660167e2438a789c7588a6fee4f9c37b332782c010fe5c7de8f606bcf98dffb2481d2777cbce88f87cc9f0c42fb2d7fc
2019-08-19 09:39:53 -04:00
MarcoFalke
faba46da07
ci: Set --ansi in test_runner 2019-08-16 08:50:01 -04:00
MarcoFalke
fa21737ba7
ci: Add environment files for all settings 2019-08-16 08:46:37 -04:00
MarcoFalke
faeacf3269
ci: Add note that this assumes a fresh git clone 2019-08-15 11:13:10 -04:00
MarcoFalke
fa6cbdc3c9
ci: Use ./ci/ on non-travis host 2019-08-15 11:13:09 -04:00
MarcoFalke
fa31bc35eb
ci: Remove dependence on travis, use it as fallback env 2019-08-15 11:12:34 -04:00
MarcoFalke
fa0aac0f43
ci: Add retry
From: a1b1826bdb
2019-08-15 11:12:22 -04:00
MarcoFalke
fafe78f6ae
ci: Rename .travis/ to ./ci/ 2019-08-15 11:11:45 -04:00