pytest-rerunfailures icon indicating copy to clipboard operation
pytest-rerunfailures copied to clipboard
verified publisher icon pytest-dev

Tests are terminating with internal error

Open vivekvasu95 opened this issue 2 years ago • 7 comments

  • Tests are terminating with the error 'INTERNALERROR> def worker_internal_error(self, node, formatted_error)'
14:02:03  INTERNALERROR> def worker_internal_error(self, node, formatted_error):
14:02:03  INTERNALERROR>         """
14:02:03  INTERNALERROR>         pytest_internalerror() was called on the worker.
14:02:03  INTERNALERROR>     
14:02:03  INTERNALERROR>         pytest_internalerror() arguments are an excinfo and an excrepr, which can't
14:02:03  INTERNALERROR>         be serialized, so we go with a poor man's solution of raising an exception
14:02:03  INTERNALERROR>         here ourselves using the formatted message.
14:02:03  INTERNALERROR>         """
14:02:03  INTERNALERROR>         self._active_nodes.remove(node)
14:02:03  INTERNALERROR>         try:
14:02:03  INTERNALERROR> >           assert False, formatted_error
14:02:03  INTERNALERROR> E           AssertionError: Traceback (most recent call last):
14:02:03  INTERNALERROR> E               File "/opt/jenkins/virtual_env/moe_3_9_15/lib/python3.9/site-packages/_pytest/main.py", line 270, in wrap_session
14:02:03  INTERNALERROR> E                 session.exitstatus = doit(config, session) or 0
14:02:03  INTERNALERROR> E               File "/opt/jenkins/virtual_env/moe_3_9_15/lib/python3.9/site-packages/_pytest/main.py", line 324, in _main
14:02:03  INTERNALERROR> E                 config.hook.pytest_runtestloop(session=session)
14:02:03  INTERNALERROR> E               File "/opt/jenkins/virtual_env/moe_3_9_15/lib/python3.9/site-packages/pluggy/_hooks.py", line 433, in __call__
14:02:03  INTERNALERROR> E                 return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
14:02:03  INTERNALERROR> E               File "/opt/jenkins/virtual_env/moe_3_9_15/lib/python3.9/site-packages/pluggy/_manager.py", line 112, in _hookexec
14:02:03  INTERNALERROR> E                 return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
14:02:03  INTERNALERROR> E               File "/opt/jenkins/virtual_env/moe_3_9_15/lib/python3.9/site-packages/pluggy/_callers.py", line 155, in _multicall
14:02:03  INTERNALERROR> E                 return outcome.get_result()
14:02:03  INTERNALERROR> E               File "/opt/jenkins/virtual_env/moe_3_9_15/lib/python3.9/site-packages/pluggy/_result.py", line 108, in get_result
14:02:03  INTERNALERROR> E                 raise exc.with_traceback(exc.__traceback__)
14:02:03  INTERNALERROR> E               File "/opt/jenkins/virtual_env/moe_3_9_15/lib/python3.9/site-packages/pluggy/_callers.py", line 80, in _multicall
14:02:03  INTERNALERROR> E                 res = hook_impl.function(*args)
14:02:03  INTERNALERROR> E               File "/opt/jenkins/virtual_env/moe_3_9_15/lib/python3.9/site-packages/xdist/remote.py", line 146, in pytest_runtestloop
14:02:03  INTERNALERROR> E                 self.run_one_test()
14:02:03  INTERNALERROR> E               File "/opt/jenkins/virtual_env/moe_3_9_15/lib/python3.9/site-packages/xdist/remote.py", line 161, in run_one_test
14:02:03  INTERNALERROR> E                 self.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem)
14:02:03  INTERNALERROR> E               File "/opt/jenkins/virtual_env/moe_3_9_15/lib/python3.9/site-packages/pluggy/_hooks.py", line 433, in __call__
14:02:03  INTERNALERROR> E                 return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
14:02:03  INTERNALERROR> E               File "/opt/jenkins/virtual_env/moe_3_9_15/lib/python3.9/site-packages/pluggy/_manager.py", line 112, in _hookexec
14:02:03  INTERNALERROR> E                 return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
14:02:03  INTERNALERROR> E               File "/opt/jenkins/virtual_env/moe_3_9_15/lib/python3.9/site-packages/pluggy/_callers.py", line 155, in _multicall
14:02:03  INTERNALERROR> E                 return outcome.get_result()
14:02:03  INTERNALERROR> E               File "/opt/jenkins/virtual_env/moe_3_9_15/lib/python3.9/site-packages/pluggy/_result.py", line 108, in get_result
14:02:03  INTERNALERROR> E                 raise exc.with_traceback(exc.__traceback__)
14:02:03  INTERNALERROR> E               File "/opt/jenkins/virtual_env/moe_3_9_15/lib/python3.9/site-packages/pluggy/_callers.py", line 80, in _multicall
14:02:03  INTERNALERROR> E                 res = hook_impl.function(*args)
14:02:03  INTERNALERROR> E               File "/opt/jenkins/virtual_env/moe_3_9_15/lib/python3.9/site-packages/pytest_rerunfailures.py", line 582, in pytest_runtest_protocol
14:02:03  INTERNALERROR> E                 if _should_not_rerun(item, report, reruns):
14:02:03  INTERNALERROR> E               File "/opt/jenkins/virtual_env/moe_3_9_15/lib/python3.9/site-packages/pytest_rerunfailures.py", line 336, in _should_not_rerun
14:02:03  INTERNALERROR> E                 is_terminal_error = _should_hard_fail_on_error(item.session.config, report)
14:02:03  INTERNALERROR> E               File "/opt/jenkins/virtual_env/moe_3_9_15/lib/python3.9/site-packages/pytest_rerunfailures.py", line 327, in _should_hard_fail_on_error
14:02:03  INTERNALERROR> E                 if not re.search(rerun_regex, report.longrepr.reprcrash.message):
14:02:03  INTERNALERROR> E             AttributeError: 'NoneType' object has no attribute 'message'
14:02:03  INTERNALERROR> E           assert False

vivekvasu95 avatar Aug 23 '23 08:08 vivekvasu95

i took the liberty to edit for readability

based on the traceback its a issue with pytest_rerunfailures

RonnyPfannschmidt avatar Aug 23 '23 09:08 RonnyPfannschmidt

im unfamiliar with rerunfailures, so its not clear to me if this is a bug with rerunfailures itself or something else interacting with it

RonnyPfannschmidt avatar Aug 23 '23 09:08 RonnyPfannschmidt

@vivekvasu95 Could you please provide some version information? (version of pytest, pytest-rerunfailures, other plugins, Python, …) What did you already do to circumvent the problem? Just pasting a traceback is not enough as you seem to be the only one with this problem.

icemac avatar Aug 25 '23 05:08 icemac

@icemac Here is the complete packages used

Package Version

adbutils 1.0.8 allure-pytest 2.9.44 allure-python-commons 2.9.44 apkutils2 1.0.0 appdirs 1.4.4 Appium-Python-Client 2.7.1 astroid 2.15.2 async-generator 1.10 attrs 22.1.0 audioread 3.0.0 beautifulsoup4 4.11.2 boto3 1.28.11 botocore 1.31.11 Brotli 1.0.9 cached-property 1.5.2 certifi 2022.6.15 cffi 1.15.1 chainmap 1.0.3 chardet 3.0.4 charset-normalizer 2.1.1 cigam 0.0.3 click 8.1.3 combomethod 1.0.12 ConfigArgParse 1.5.3 configparser 5.3.0 contourpy 1.0.5 cryptography 37.0.4 curlify 2.2.1 cycler 0.11.0 decorator 5.1.1 Deprecated 1.2.13 deprecation 2.1.0 dill 0.3.6 emoji 2.2.0 et-xmlfile 1.1.0 exceptiongroup 1.1.1 execnet 1.9.0 Faker 18.3.0 ffmpeg-python 0.2.0 filelock 3.9.1 Flask 2.2.3 Flask-BasicAuth 0.2.0 Flask-Cors 3.0.10 fonttools 4.37.4 future 0.18.2 geographiclib 2.0 geopy 2.3.0 gevent 21.12.0 geventhttpclient 2.0 greenlet 2.0.1 h11 0.13.0 html2text 2020.1.16 hvac 1.1.1 idna 2.10 influxdb 5.3.1 influxdb-client 1.37.0 iniconfig 1.1.1 isort 5.10.1 itsdangerous 2.1.2 Jinja2 3.0.3 jmespath 1.0.1 joblib 1.2.0 kiwisolver 1.4.4 lazy-object-proxy 1.7.1 lc 0.1 librosa 0.9.2 llvmlite 0.39.1 locust 2.8.6 logzero 1.7.0 lxml 4.9.1 mailosaur 7.14.0 MarkupSafe 2.1.1 matplotlib 3.6.1 mccabe 0.6.1 msgpack 1.0.4 names 0.3.0 noisereduce 2.0.1 nulltype 2.3.1 numba 0.56.3 numpy 1.24.2 openpyxl 3.1.1 options 1.4.10 outcome 1.2.0 packaging 20.9 pandas 1.5.3 Pillow 9.2.0 pip 21.3.1 pixelmatch 0.3.0 platformdirs 2.5.2 playwright 1.28.0 pluggy 1.0.0 pooch 1.6.0 progress 1.6 psutil 5.9.1 py 1.11.0 pycparser 2.21 pydub 0.25.1 pyee 9.0.4 pyelftools 0.29 pyhcl 0.4.4 PyJWT 2.6.0 pylint 2.16.2 pyOpenSSL 22.0.0 pyparsing 3.0.9 PySocks 1.7.1 pytest 7.2.2 pytest-base-url 2.0.0 pytest-check 2.1.4 pytest-forked 1.4.0 pytest-json-report 1.5.0 pytest-metadata 2.0.4 pytest-parallel 0.1.1 pytest-playwright 0.3.0 pytest-playwright-visual 2.1.2 pytest-rerunfailures 11.1.1 pytest-timeout 2.1.0 pytest-xdist 3.2.0 python-dateutil 2.8.2 python-slugify 6.1.2 pytz 2022.7.1 pytz-deprecation-shim 0.1.0.post0 pyzmq 23.2.0 ratelimiter 1.2.0.post0 reactivex 4.0.4 requests 2.28.1 resampy 0.4.2 retry 0.9.2 roundrobin 0.0.2 s3transfer 0.6.1 scikit-learn 1.1.2 scipy 1.9.2 selenium 4.8.3 setuptools 65.6.3 simplejson 3.17.6 six 1.12.0 sniffio 1.2.0 sortedcontainers 2.4.0 soundfile 0.11.0 soupsieve 2.3.2.post1 SpeechRecognition 3.8.1 tblib 1.7.0 text-unidecode 1.3 threadpoolctl 3.1.0 toml 0.10.2 tomli 2.0.1 tomlkit 0.11.6 tqdm 4.64.1 trio 0.21.0 trio-websocket 0.9.2 typing_extensions 4.3.0 tzdata 2022.1 tzlocal 4.2 uiautomator2 2.16.19 urllib3 1.26.12 websockets 10.1 Werkzeug 2.2.2 wheel 0.37.1 whichcraft 0.6.1 wrapt 1.13.3 wsproto 1.1.0 xmltodict 0.13.0 xvfbwrapper 0.2.9 zope.event 4.5.0 zope.interface 5.4.0

vivekvasu95 avatar Aug 25 '23 06:08 vivekvasu95

@vivekvasu95 There are newer versions of both pytest and pytest-rerunfailures. Could you please try that versions as a first mitigation step?

icemac avatar Aug 25 '23 06:08 icemac

Sure.

vivekvasu95 avatar Aug 25 '23 12:08 vivekvasu95

I am closing this old issue. Feel free to re-open it if there are any news about the topic.

icemac avatar Oct 10 '23 06:10 icemac