Commit Graph

51 Commits

Author SHA1 Message Date
josibake
bc90b8d869
[move only] remove is_wallet_compiled checks 2022-03-24 11:00:22 +01:00
josibake
0bfbf7fb24
test: use MiniWallet in interfaces_zmq
make interfaces_zmg run deterministically.
this test is for the zmg notifications,
so it doesn't need the wallet compiled to run
2022-03-24 10:57:38 +01:00
Sebastian Falbesoner
df5d783aef test: refactor: take use of create_block txlist parameter
Passing a list of transactions `txlist` to `create_block` appends
them to the block, hence we don't need to do that manually anymore.
The merkle root calculation can also be removed, since this is done
in the end of the helper.
2021-11-17 15:45:30 +01:00
MarcoFalke
fac23c2114
scripted-diff: Bump copyright headers
The previous diff touched most files in ./test/, so bump the headers to
avoid having to touch them again for a bump later.

-BEGIN VERIFY SCRIPT-
./contrib/devtools/copyright_header.py update ./test/
-END VERIFY SCRIPT-
2021-11-10 11:10:24 +01:00
MarcoFalke
fa974f1f14
scripted-diff: Remove redundant sync_all and sync_blocks
The sync calls are redundant after a call to generate, because generate
already syncs itself.

-BEGIN VERIFY SCRIPT-
perl -0777 -pi -e 's/(generate[^\n]*\)[^\n]*)(\n|\s)+self.sync_(all|blocks)\([^\)]*\)\n/\1\n/g' $(git grep -l generate ./test)
-END VERIFY SCRIPT-
2021-11-10 11:10:15 +01:00
MarcoFalke
facc352648
test: Implicitly sync after generate*, unless opted out 2021-10-29 13:34:52 +02:00
Samuel Dobson
eb02dbba3c Use self.generate not node.generate throughout tests 2021-10-06 12:18:33 +13:00
W. J. van der Laan
b05d3e76e7
Merge bitcoin/bitcoin#22079: zmq: Add support to listen on IPv6 addresses
e6998838e5 doc: Add IPv6 address to zmq example (nthumann)
8abe5703a9 test: Add IPv6 test to zmq (nthumann)
ded449b726 zmq: Enable IPv6 on listening socket (nthumann)

Pull request description:

  This PR adds support for listening on IPv6 addresses with bitcoinds ZMQ interface, just like the RPC server.
  Currently, it is not possible to specify an IPv6 address, as the `ZMQ_IPV6` [socket option](http://api.zeromq.org/master:zmq-setsockopt#toc27) is not set and therefore the ZMQ initialization fails, if one does so. The absence of this option has also been noted [here](https://github.com/bitcoin/bitcoin/issues/15198#issuecomment-617378512).
  With this PR one can e.g. set `-zmqpubhashblock=tcp://[::1]:28333` to listen on the IPv6 loopback address.

ACKs for top commit:
  laanwj:
    Code review ACK e6998838e5
  theStack:
    Tested ACK e6998838e5 🌱

Tree-SHA512: 43c3043d8d5c79794d475926259c1be975b694db4fcc1f7750a9a28e242f0fa1b531735a63ea5777498003aa5834f6243f39742d0f3941f2f37593d0c7890700
2021-09-09 15:37:13 +02:00
MarcoFalke
fa0b916971
scripted-diff: Use generate* from TestFramework
-BEGIN VERIFY SCRIPT-
 sed --regexp-extended -i \
     's/((self\.)?(nodes\[[^]]+\]|[a-z_]*(wallet|node)[0-9a-z_]*))\.(generate(|toaddress|block|todescriptor)(\(|, ))/self.\5\1, /g' \
     $(git grep -l generate ./test | grep -v 'test_framework/' | grep -v 'feature_rbf')
-END VERIFY SCRIPT-
2021-09-02 10:34:35 +02:00
MarcoFalke
fab2e23b57
Use generate* from TestFramework
The changes in feature_rbf can be reviewed with
--color-moved=dimmed-zebra --color-moved-ws=ignore-all-space
2021-08-19 19:02:11 +02:00
fanquake
c2a5d560df
test: use f-strings in interface_*.py tests 2021-08-18 12:39:20 +08:00
Sebastian Falbesoner
a79396fe5f test: remove ToHex helper, use .serialize().hex() instead 2021-06-21 14:30:03 +02:00
Sebastian Falbesoner
2ce7b47958 test: introduce tx_from_hex helper for tx deserialization
`FromHex` is mostly used for transactions, so we introduce a
shortcut `tx_from_hex` for `FromHex(CTransaction, hex_str)`.
2021-06-21 14:28:05 +02:00
nthumann
8abe5703a9
test: Add IPv6 test to zmq 2021-06-20 16:56:08 +02:00
Sebastian Falbesoner
8a8c6383f6 zmq test: fix sync-up by matching notification to generated block
It turned out that the "sync up" procedure of repeatedly generating a
block and waiting for a notification once with timeout is too naive in
its current form, as the following scenario could happen:
    - generate block A
    - receive notification, timeout happens -> repeat procedure
    - generate block B
    - node publishes block A notification
    - receive notification, we receive the one caused by block A
      -> sync-up procedure is completed
    - node publishes block B
    - the actual test starts
    - on the first notification reception, one caused by block B is received,
      rather than the one actually caused by test code, leading to failure

This change ensures that after each test block generation, we wait for
the notification that is actually caused by that block and ignore others
from possibly earlier blocks.

Co-authored-by: Jon Atack <jon@atack.com>
2021-03-01 01:31:41 +01:00
Sebastian Falbesoner
ef21fb7313 zmq test: speedup test by whitelisting peers (immediate tx relay)
Speeds up the zmq test roughly by a factor of 2x (~20 sec. instead of
~40 sec.) and also avoids timeouts on the synchronization methods
(sync_mempool() / sync_blocks()) that happened with a slight chance.
This is due to the fact that there is no upper bound on the trickle
relay time, so even the default of 60s is sometimes too low. Fixed by
enabling immediate tx relay on node1.
2021-02-09 23:55:32 +01:00
Sebastian Falbesoner
5c6546362d zmq test: fix flakiness by using more robust sync method
After connecting the subscriber sockets to the node, there is no
guarantee that the node's zmq publisher interfaces are ready yet, which
means that potentially the first expected notification messages could
get lost and the test fails. Currently this is handled by just waiting
for a short period of time (200ms), which works most of the time but is
still problematic, as in some rare cases the setup time takes much
longer, even in the range of multiple seconds.

The solution in this commit approaches the problem by using a more
robust method of syncing up, originally proposed by instagibbs:
    1. Generate a block on the node
    2. Try to receive a notification on all subscribers
    3. If all subscribers get a message within the timeout (1 second),
       we are done, otherwise repeat starting from step 1
2021-02-09 23:55:23 +01:00
Sebastian Falbesoner
8666033630 zmq test: accept arbitrary sequence start number in ZMQSubscriber
The ZMQSubscriber reception methods currently assert that the first
received publisher message has a sequence number of zero. In order to
fix the current test flakiness via "syncing up" to nodes in the setup
phase, we have to cope with the situation that messages get lost and the
first actual received message has a sequence number larger than zero.
2021-02-09 22:54:01 +01:00
Sebastian Falbesoner
6014d6e1b5 zmq test: dedup message reception handling in ZMQSubscriber 2021-02-09 22:54:01 +01:00
Andrew Chow
09514e1bef Setup wallets for interface_zmq.py 2021-01-27 12:52:46 -05:00
Sebastian Falbesoner
4efb6c2d3b zmq test: deduplicate test setup code (node restart, topics subscription) 2021-01-17 12:58:46 +01:00
MarcoFalke
fa0074e2d8
scripted-diff: Bump copyright headers
-BEGIN VERIFY SCRIPT-
./contrib/devtools/copyright_header.py update ./
-END VERIFY SCRIPT-
2020-12-31 09:45:41 +01:00
Prayank
4b16c61461 scripted-diff: test: Replace uses of (dis)?connect_nodes global
-BEGIN VERIFY SCRIPT-

 # max-depth=0 excludes test/functional/test_framework/...
 FILES=$(git grep -l --max-depth 0 "connect_nodes" test/functional)

 # Replace (dis)?connect_nodes(self.nodes[a], b) with self.(dis)?connect_nodes(a, b)
 sed -i 's/\b\(dis\)\?connect_nodes(self\.nodes\[\(.*\)\]/self.\1connect_nodes(\2/g' $FILES

 # Remove imports in the middle of a line
 sed -i 's/\(dis\)\?connect_nodes, //g' $FILES
 sed -i 's/, \(dis\)\?connect_nodes//g' $FILES

 # Remove imports on a line by themselves
 sed -i '/^\s*\(dis\)\?connect_nodes,\?$/d' $FILES
 sed -i '/^from test_framework\.util import connect_nodes$/d' $FILES

-END VERIFY SCRIPT-

Co-authored-by: Elliott Jin <elliott.jin@gmail.com>
2020-10-20 00:42:00 -07:00
nthumann
241803da21
test: Add zmq test to support multiple interfaces 2020-10-01 00:33:38 +02:00
Gregory Sanders
68c3c7e1bd Add functional tests for zmq sequence topic and mempool sequence logic 2020-09-22 11:34:30 -04:00
Gregory Sanders
1b615e61bf zmq test: Actually make reorg occur 2020-09-22 11:17:50 -04:00
Gregory Sanders
7356292e1d Have zmq reorg test cover mempool txns 2020-07-14 10:14:11 -04:00
Gregory Sanders
a0f4f9c983 Add zmq test for transaction pub during reorg 2020-07-13 14:18:01 -04:00
Gregory Sanders
2399a0600c Add test case for mempool->block zmq notification 2020-07-13 14:18:01 -04:00
Gregory Sanders
e70512a83c Make ordering of zmq consumption irrelevant to functional test 2020-07-13 14:18:01 -04:00
João Barbosa
3e730bf90a zmq: Fix due to invalid argument and multiple notifiers 2019-11-11 22:22:32 +00:00
João Barbosa
403e372407 qa: Relax so that the subscriber is ready before publishing zmq messages 2019-08-28 00:42:29 +01:00
João Barbosa
abdfc5e89b qa: Test ZMQ notification after chain reorg 2019-08-19 01:25:33 +01:00
João Barbosa
aa2622a726 qa: Refactor ZMQ test 2019-08-19 01:25:31 +01:00
Elichai Turkel
afc0966d72
Moved and renamed hash256 from util.py to zmq_interface.py 2019-08-13 15:39:46 -04:00
MarcoFalke
faa7cdf764
scripted-diff: Update copyright in ./test
-BEGIN VERIFY SCRIPT-
./contrib/devtools/copyright_header.py update ./test/
-END VERIFY SCRIPT-
2019-03-02 10:58:35 -05:00
MarcoFalke
fa6bf21f5e
scripted-diff: test: Use py3.5 bytes::hex() method
-BEGIN VERIFY SCRIPT-
sed -i -e "s/def bytes_to_hex_str/def b_2_x/g" $(git grep -l bytes_to_hex_str)

export RE_B_0="[^()]*"                          # match no bracket
export RE_B_1="${RE_B_0}\(${RE_B_0}\)${RE_B_0}" # match exactly one ()
export RE_B_2="${RE_B_0}\(${RE_B_1}\)${RE_B_0}" # match wrapped (())

export RE_M="(b2x|bytes_to_hex_str)\(((${RE_B_0}|${RE_B_1}|${RE_B_2})*)\)"

sed -i --regexp-extended -e "s/${RE_M}/\2.hex()/g"      $(git grep -l -E '(b2x|bytes_to_hex_str)')

sed -i --regexp-extended -e "/  +bytes_to_hex_str( as b2x)?,/d"    $(git grep -l bytes_to_hex_str)
sed -i --regexp-extended -e "s/ +bytes_to_hex_str( as b2x)?,//g"   $(git grep -l bytes_to_hex_str)
sed -i --regexp-extended -e "s/, bytes_to_hex_str( as b2x)?//g"    $(git grep -l bytes_to_hex_str)

export RE_M="(binascii\.)?hexlify\(((${RE_B_0}|${RE_B_1}|${RE_B_2})*)\).decode\(${RE_B_0}\)"

sed -i --regexp-extended -e "s/${RE_M}/\2.hex()/g" $(git grep -l hexlify -- ':(exclude)share')

sed -i --regexp-extended -e  "/from binascii import hexlify$/d" $(git grep -l hexlify -- ':(exclude)share')
sed -i --regexp-extended -e "s/(from binascii import) .*hexlify/\1 unhexlify/g" $(git grep -l hexlify -- ':(exclude)share')

sed -i -e 's/ignore-names "/ignore-names "b_2_x,/g' ./test/lint/lint-python-dead-code.sh
-END VERIFY SCRIPT-
2019-03-02 10:40:12 -05:00
Wladimir J. van der Laan
dac2caa371
Merge #14060: ZMQ: add options to configure outbound message high water mark, aka SNDHWM
a4edb168b6 ZMQ: add options to configure outbound message high water mark, aka SNDHWM (mruddy)

Pull request description:

  ZMQ: add options to configure outbound message high water mark, aka SNDHWM

  This is my attempt at https://github.com/bitcoin/bitcoin/pull/13315

Tree-SHA512: a4cc3bcf179776899261a97c8c4f31f35d1d8950fd71a09a79c5c064879b38e600b26824c89c4091d941502ed5b0255390882f7d44baf9e6dc49d685a86e8edb
2018-11-05 13:45:41 +01:00
John Newbery
3fd7e76f6d [tests] Move deterministic address import to setup_nodes
This requires a small changes to a few tests, but means that
deterministic addresses will always be imported (unless setup_nodes
behaviour is explicitly overridden).
2018-11-01 12:53:49 -04:00
mruddy
a4edb168b6 ZMQ: add options to configure outbound message high water mark, aka SNDHWM 2018-10-19 07:36:13 -04:00
MarcoFalke
d12e3d9acc
Merge #14324: qa: Run more tests with wallet disabled
faa4043c66 qa: Run more tests with wallet disabled (MarcoFalke)

Pull request description:

  Instead of skipping the whole test, only skip the wallet specific section of a test if the wallet is not compiled in. This is mostly an indentation change, so can be reviewed with `--ignore-all-space`.

Tree-SHA512: 5941a8b6b00dca5cf9438c5f6f010ba812115188a69e427d7ade4c1ab8cfe7a57c73daf52c66235dbb24b1cd9ab7c7a17c49bc23d931e041b605d79116a71f66
2018-10-08 01:42:02 -03:00
John Newbery
42a995ae48 [tests] Remove rpc_zmq.py
rpc_zmq.py is racy and fails intermittently. Remove that test file and
move the getzmqnotifications RPC test into interface_zmq.py
2018-10-07 09:32:50 +09:00
MarcoFalke
faa4043c66
qa: Run more tests with wallet disabled 2018-09-25 18:08:08 -04:00
MarcoFalke
fa8433e379
qa: Remove unneded import_deterministic_coinbase_privkeys overwrite, add comments 2018-09-13 09:21:44 -04:00
MarcoFalke
fac9539836
qa: Run all tests even if wallet is not compiled 2018-09-10 17:53:21 -04:00
MarcoFalke
faa669cbcd
qa: Premine to deterministic address with -disablewallet 2018-09-10 17:51:46 -04:00
practicalswift
68400d8b96 tests: Use explicit imports 2018-08-13 14:13:39 +02:00
DrahtBot
eb7daf4d60 Update copyright headers to 2018 2018-07-27 07:15:02 -04:00
James O'Beirne
a0b604c166 [tests] skip rpc_zmq functional test when python3 zmq lib is not present
Also refactors zmq-related test skipping logic into distinct functions.
2018-07-12 13:08:35 -04:00
MarcoFalke
fa811b0b1d
qa: Normalize executable location 2018-04-21 16:55:42 -04:00