eth-tester icon indicating copy to clipboard operation
eth-tester copied to clipboard

Transaction not mined when running multiple unit tests (PyEthereum21Backend)

Open artemistomaras opened this issue 6 years ago • 4 comments

  • Version: 0.1.0b26
  • Python: 3.6
  • OS: linux

Suppose the following django project structure:

  • api
    • tests.py
  • client
    • tests.py

Also suppose the following TestCase setup for both test files:

    def setUp(self):
        # Retrieve a fresh client copy
        self.client = MyClient(
            provider=self.provider, ...
        )

        super().setUp()

    def tearDown(self):
        # Revert to the clean state for every test case
        self.eth_tester.revert_to_snapshot(self.clean_state_snapshot)
        super().tearDown()

    @classmethod
    def setUpTestData(cls):
        """Only once setup"""
        cls.eth_tester = EthereumTester(backend=PyEthereum21Backend())
        cls.provider = EthereumTesterProvider(cls.eth_tester)
        cls.web3 = Web3(cls.provider)

        # deploying smart contracts etc ...

        cls.clean_state_snapshot = cls.eth_tester.take_snapshot()

Below is the output of running each TestCase on its own:

# client.tests output
Failed to import bitcoin. This is not a fatal error but does
mean that you will not be able to determine the address from
your wallet file.
Using existing test database for alias 'default'...
System check identified no issues (0 silenced).
Initializing chain from provided state
Initializing chain from provided state
INFO:eth.block	Block pre-sealed, 21000 gas used 
INFO:eth.chain	Adding to head head=c6745cf3
Saved 3 address change logs
INFO:eth.chain	Added block 1 (5f5f608a) with 1 txs and 21000 gas 
INFO:eth.block	Block pre-sealed, 470791 gas used 
INFO:eth.chain	Adding to head head=5f5f608a
Saved 2 address change logs
INFO:eth.chain	Added block 2 (3c04531f) with 1 txs and 470791 gas 
INFO:eth.block	Block pre-sealed, 1641161 gas used 
INFO:eth.chain	Adding to head head=3c04531f
Saved 2 address change logs
INFO:eth.chain	Added block 3 (94a8e4d5) with 1 txs and 1641161 gas 
INFO:eth.block	Block pre-sealed, 505467 gas used 
INFO:eth.chain	Adding to head head=94a8e4d5
Saved 2 address change logs
INFO:eth.chain	Added block 4 (92e1a399) with 1 txs and 505467 gas 
INFO:eth.block	Block pre-sealed, 283526 gas used 
INFO:eth.chain	Adding to head head=92e1a399
Saved 4 address change logs
INFO:eth.chain	Added block 5 (08aa138d) with 1 txs and 283526 gas 
.INFO:eth.block	Block pre-sealed, 283526 gas used 
INFO:eth.chain	Adding to head head=92e1a399
Saved 4 address change logs
INFO:eth.chain	Added block 5 (08aa138d) with 1 txs and 283526 gas 
.INFO:eth.block	Block pre-sealed, 283526 gas used 
INFO:eth.chain	Adding to head head=92e1a399
Saved 4 address change logs
INFO:eth.chain	Added block 5 (08aa138d) with 1 txs and 283526 gas 
INFO:eth.block	Block pre-sealed, 98148 gas used 
INFO:eth.chain	Adding to head head=08aa138d
Saved 7 address change logs
INFO:eth.chain	Added block 6 (c1174bae) with 1 txs and 98148 gas 
.INFO:eth.block	Block pre-sealed, 283526 gas used 
INFO:eth.chain	Adding to head head=92e1a399
Saved 4 address change logs
INFO:eth.chain	Added block 5 (08aa138d) with 1 txs and 283526 gas 
.INFO:eth.block	Block pre-sealed, 283526 gas used 
INFO:eth.chain	Adding to head head=92e1a399
Saved 4 address change logs
INFO:eth.chain	Added block 5 (08aa138d) with 1 txs and 283526 gas 
INFO:eth.block	Block pre-sealed, 98148 gas used 
INFO:eth.chain	Adding to head head=08aa138d
Saved 7 address change logs
INFO:eth.chain	Added block 6 (c1174bae) with 1 txs and 98148 gas 
.INFO:eth.block	Block pre-sealed, 283526 gas used 
INFO:eth.chain	Adding to head head=92e1a399
Saved 4 address change logs
INFO:eth.chain	Added block 5 (08aa138d) with 1 txs and 283526 gas 
INFO:eth.block	Block pre-sealed, 98020 gas used 
INFO:eth.chain	Adding to head head=08aa138d
Saved 7 address change logs
INFO:eth.chain	Added block 6 (d4d6425a) with 1 txs and 98020 gas 
.
----------------------------------------------------------------------
Ran 6 tests in 1.762s

OK
Preserving test database for alias 'default'..

and for the api tests:

Failed to import bitcoin. This is not a fatal error but does
mean that you will not be able to determine the address from
your wallet file.
Using existing test database for alias 'default'...
System check identified no issues (0 silenced).
Initializing chain from provided state
Initializing chain from provided state
INFO:eth.block	Block pre-sealed, 21000 gas used 
INFO:eth.chain	Adding to head head=c6745cf3
Saved 3 address change logs
INFO:eth.chain	Added block 1 (5f5f608a) with 1 txs and 21000 gas 
INFO:eth.block	Block pre-sealed, 470791 gas used 
INFO:eth.chain	Adding to head head=5f5f608a
Saved 2 address change logs
INFO:eth.chain	Added block 2 (3c04531f) with 1 txs and 470791 gas 
INFO:eth.block	Block pre-sealed, 1641161 gas used 
INFO:eth.chain	Adding to head head=3c04531f
Saved 2 address change logs
INFO:eth.chain	Added block 3 (94a8e4d5) with 1 txs and 1641161 gas 
INFO:eth.block	Block pre-sealed, 505467 gas used 
INFO:eth.chain	Adding to head head=94a8e4d5
Saved 2 address change logs
INFO:eth.chain	Added block 4 (92e1a399) with 1 txs and 505467 gas 
INFO:eth.block	Block pre-sealed, 283526 gas used 
INFO:eth.chain	Adding to head head=92e1a399
Saved 4 address change logs
INFO:eth.chain	Added block 5 (08aa138d) with 1 txs and 283526 gas 
.INFO:eth.block	Block pre-sealed, 283526 gas used 
INFO:eth.chain	Adding to head head=92e1a399
Saved 4 address change logs
INFO:eth.chain	Added block 5 (08aa138d) with 1 txs and 283526 gas 
.INFO:eth.block	Block pre-sealed, 283526 gas used 
INFO:eth.chain	Adding to head head=92e1a399
Saved 4 address change logs
INFO:eth.chain	Added block 5 (08aa138d) with 1 txs and 283526 gas 
.INFO:eth.block	Block pre-sealed, 283526 gas used 
INFO:eth.chain	Adding to head head=92e1a399
Saved 4 address change logs
INFO:eth.chain	Added block 5 (08aa138d) with 1 txs and 283526 gas 
INFO:eth.block	Block pre-sealed, 98148 gas used 
INFO:eth.chain	Adding to head head=08aa138d
Saved 7 address change logs
INFO:eth.chain	Added block 6 (2d599c9a) with 1 txs and 98148 gas 
.INFO:eth.block	Block pre-sealed, 283526 gas used 
INFO:eth.chain	Adding to head head=92e1a399
Saved 4 address change logs
INFO:eth.chain	Added block 5 (08aa138d) with 1 txs and 283526 gas 
.INFO:eth.block	Block pre-sealed, 283526 gas used 
INFO:eth.chain	Adding to head head=92e1a399
Saved 4 address change logs
INFO:eth.chain	Added block 5 (08aa138d) with 1 txs and 283526 gas 
INFO:eth.block	Block pre-sealed, 98148 gas used 
INFO:eth.chain	Adding to head head=08aa138d
Saved 7 address change logs
INFO:eth.chain	Added block 6 (2d599c9a) with 1 txs and 98148 gas 
.
----------------------------------------------------------------------
Ran 6 tests in 1.652s

OK
Preserving test database for alias 'default'...

Everything looks ok. The problem appears when I try to run both test cases at once, using a single python3 manage.py test command.

What will happen is that inside my code, a call to web3.eth.waitForTransactionReceipt will always timeout or, a call to EthereumTester.get_transaction_receipt will always raise a TransactionNotFound

Below is the output of executing both test cases together, and me pressing ctrl+c to terminate when the call to web3.eth.waitForTransactionReceipt blocks ( setting the timeout to 1 second for example will just make all my unit tests fail )

Failed to import bitcoin. This is not a fatal error but does
mean that you will not be able to determine the address from
your wallet file.
Using existing test database for alias 'default'...
System check identified no issues (0 silenced).
Initializing chain from provided state
Initializing chain from provided state
INFO:eth.block	Block pre-sealed, 21000 gas used 
INFO:eth.chain	Adding to head head=c6745cf3
Saved 3 address change logs
INFO:eth.chain	Added block 1 (5f5f608a) with 1 txs and 21000 gas 
INFO:eth.block	Block pre-sealed, 470791 gas used 
INFO:eth.chain	Adding to head head=5f5f608a
Saved 2 address change logs
INFO:eth.chain	Added block 2 (3c04531f) with 1 txs and 470791 gas 
INFO:eth.block	Block pre-sealed, 1641161 gas used 
INFO:eth.chain	Adding to head head=3c04531f
Saved 2 address change logs
INFO:eth.chain	Added block 3 (94a8e4d5) with 1 txs and 1641161 gas 
INFO:eth.block	Block pre-sealed, 505467 gas used 
INFO:eth.chain	Adding to head head=94a8e4d5
Saved 2 address change logs
INFO:eth.chain	Added block 4 (92e1a399) with 1 txs and 505467 gas 
INFO:eth.block	Block pre-sealed, 283526 gas used 
INFO:eth.chain	Adding to head head=92e1a399
Saved 4 address change logs
INFO:eth.chain	Added block 5 (08aa138d) with 1 txs and 283526 gas 
.INFO:eth.block	Block pre-sealed, 283526 gas used 
INFO:eth.chain	Adding to head head=92e1a399
Saved 4 address change logs
INFO:eth.chain	Added block 5 (08aa138d) with 1 txs and 283526 gas 
.INFO:eth.block	Block pre-sealed, 283526 gas used 
INFO:eth.chain	Adding to head head=92e1a399
Saved 4 address change logs
INFO:eth.chain	Added block 5 (08aa138d) with 1 txs and 283526 gas 
.INFO:eth.block	Block pre-sealed, 283526 gas used 
INFO:eth.chain	Adding to head head=92e1a399
Saved 4 address change logs
INFO:eth.chain	Added block 5 (08aa138d) with 1 txs and 283526 gas 
INFO:eth.block	Block pre-sealed, 98148 gas used 
INFO:eth.chain	Adding to head head=08aa138d
Saved 7 address change logs
INFO:eth.chain	Added block 6 (0fa9f473) with 1 txs and 98148 gas 
.INFO:eth.block	Block pre-sealed, 283526 gas used 
INFO:eth.chain	Adding to head head=92e1a399
Saved 4 address change logs
INFO:eth.chain	Added block 5 (08aa138d) with 1 txs and 283526 gas 
.INFO:eth.block	Block pre-sealed, 283526 gas used 
INFO:eth.chain	Adding to head head=92e1a399
Saved 4 address change logs
INFO:eth.chain	Added block 5 (08aa138d) with 1 txs and 283526 gas 
INFO:eth.block	Block pre-sealed, 98148 gas used 
INFO:eth.chain	Adding to head head=08aa138d
Saved 7 address change logs
INFO:eth.chain	Added block 6 (0fa9f473) with 1 txs and 98148 gas 
.Initializing chain from provided state
Initializing chain from provided state
INFO:eth.block	Block pre-sealed, 21000 gas used 
INFO:eth.chain	Adding to head head=c6745cf3
Saved 3 address change logs
INFO:eth.chain	Added block 1 (5f5f608a) with 1 txs and 21000 gas 
INFO:eth.block	Block pre-sealed, 470791 gas used 
INFO:eth.chain	Adding to head head=5f5f608a
Saved 2 address change logs
INFO:eth.chain	Added block 2 (3c04531f) with 1 txs and 470791 gas 
INFO:eth.block	Block pre-sealed, 1641161 gas used 
INFO:eth.chain	Adding to head head=3c04531f
Saved 2 address change logs
INFO:eth.chain	Added block 3 (94a8e4d5) with 1 txs and 1641161 gas 
INFO:eth.block	Block pre-sealed, 505467 gas used 
INFO:eth.chain	Adding to head head=94a8e4d5
Saved 2 address change logs
INFO:eth.chain	Added block 4 (92e1a399) with 1 txs and 505467 gas 
INFO:eth.block	Block pre-sealed, 283526 gas used 
INFO:eth.chain	Adding to head head=92e1a399
Saved 4 address change logs
INFO:eth.chain	Added block 5 (08aa138d) with 1 txs and 283526 gas 
^C^CTraceback (most recent call last):
  File "manage.py", line 15, in <module>
    execute_from_command_line(sys.argv)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/django/core/management/__init__.py", line 371, in execute_from_command_line
    utility.execute()
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/django/core/management/__init__.py", line 365, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/django/core/management/commands/test.py", line 26, in run_from_argv
    super().run_from_argv(argv)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/django/core/management/base.py", line 288, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/django/core/management/base.py", line 335, in execute
    output = self.handle(*args, **options)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/django/core/management/commands/test.py", line 59, in handle
    failures = test_runner.run_tests(test_labels)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/django/test/runner.py", line 603, in run_tests
    result = self.run_suite(suite)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/django/test/runner.py", line 569, in run_suite
    return runner.run(suite)
  File "/usr/lib/python3.6/unittest/runner.py", line 176, in run
    test(result)
  File "/usr/lib/python3.6/unittest/suite.py", line 84, in __call__
    return self.run(*args, **kwds)
  File "/usr/lib/python3.6/unittest/suite.py", line 122, in run
    test(result)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/django/test/testcases.py", line 206, in __call__
    super().__call__(result)
  File "/usr/lib/python3.6/unittest/case.py", line 653, in __call__
    return self.run(*args, **kwds)
  File "/usr/lib/python3.6/unittest/case.py", line 605, in run
    testMethod()
  File "/home/artemis/PycharmProjects/uport_relay_server/client/tests/test_client.py", line 110, in test_create_identity
    log = self._create_identity(owner, owner)
  File "/home/artemis/PycharmProjects/uport_relay_server/client/tests/test_client.py", line 91, in _create_identity
    tx_receipt = self.web3.eth.waitForTransactionReceipt(tx_hash)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/web3/eth.py", line 199, in waitForTransactionReceipt
    return wait_for_transaction_receipt(self.web3, transaction_hash, timeout)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/web3/utils/transactions.py", line 54, in wait_for_transaction_receipt
    txn_receipt = web3.eth.getTransactionReceipt(txn_hash)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/web3/eth.py", line 204, in getTransactionReceipt
    [transaction_hash],
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/web3/manager.py", line 103, in request_blocking
    response = self._make_request(method, params)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/web3/manager.py", line 86, in _make_request
    return request_func(method, params)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/web3/middleware/gas_price_strategy.py", line 18, in middleware
    return make_request(method, params)
  File "cytoolz/functoolz.pyx", line 232, in cytoolz.functoolz.curry.__call__
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/web3/middleware/formatting.py", line 48, in apply_formatters
    response = make_request(method, formatted_params)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/web3/middleware/attrdict.py", line 18, in middleware
    response = make_request(method, params)
  File "cytoolz/functoolz.pyx", line 232, in cytoolz.functoolz.curry.__call__
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/web3/middleware/formatting.py", line 50, in apply_formatters
    response = make_request(method, params)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/web3/middleware/normalize_errors.py", line 9, in middleware
    result = make_request(method, params)
  File "cytoolz/functoolz.pyx", line 232, in cytoolz.functoolz.curry.__call__
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/web3/middleware/formatting.py", line 50, in apply_formatters
    response = make_request(method, params)
  File "cytoolz/functoolz.pyx", line 232, in cytoolz.functoolz.curry.__call__
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/web3/middleware/formatting.py", line 48, in apply_formatters
    response = make_request(method, formatted_params)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/web3/providers/eth_tester/middleware.py", line 322, in middleware
    return make_request(method, params)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/web3/middleware/fixture.py", line 12, in middleware
    return make_request(method, params)
  File "cytoolz/functoolz.pyx", line 232, in cytoolz.functoolz.curry.__call__
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/web3/middleware/formatting.py", line 50, in apply_formatters
    response = make_request(method, params)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/web3/providers/eth_tester/main.py", line 46, in make_request
    response = delegator(self.ethereum_tester, params)
  File "cytoolz/functoolz.pyx", line 758, in cytoolz.functoolz.excepts.__call__
  File "cytoolz/functoolz.pyx", line 491, in cytoolz.functoolz.Compose.__call__
  File "cytoolz/functoolz.pyx", line 232, in cytoolz.functoolz.curry.__call__
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/web3/providers/eth_tester/defaults.py", line 36, in call_eth_tester
    return getattr(eth_tester, fn_name)(*fn_args, **fn_kwargs)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/eth_tester/main.py", line 336, in get_transaction_receipt
    raw_receipt = self.backend.get_transaction_receipt(raw_transaction_hash)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/eth_tester/backends/pyethereum/v20/main.py", line 376, in get_transaction_receipt
    transaction_hash,
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/eth_tester/backends/pyethereum/v20/main.py", line 155, in _get_transaction_by_hash
    if transaction.hash == transaction_hash:
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/ethereum/transactions.py", line 141, in hash
    return utils.sha3(rlp.encode(self))
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/ethereum/utils.py", line 192, in sha3
    return sha3_256(to_string(seed))
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/ethereum/utils.py", line 4, in sha3_256
    def sha3_256(x): return keccak.new(digest_bits=256, data=x).digest()
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/Crypto/Hash/keccak.py", line 173, in new
    return Keccak_Hash(data, digest_bytes, update_after_digest)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/Crypto/Hash/keccak.py", line 70, in __init__
    state = VoidPointer()
  File "/usr/lib/python3.6/unittest/signals.py", line 36, in __call__
    self.default_handler(signum, frame)
KeyboardInterrupt

artemistomaras avatar Jun 07 '18 08:06 artemistomaras