pytest-sentry icon indicating copy to clipboard operation
pytest-sentry copied to clipboard

Published 20 hours ago • verified publisher icon getsentry

IndexError on retried test case

Open benregn opened this issue 6 years ago • 4 comments

Hey,

I was trying out your plugin and I wanted to see if was setup properly, so I added the test case from your test to my test suite. When I ran it I got an IndexError.

Test case:

i = 0
@pytest.mark.flaky(reruns=2)
@pytest.mark.sentry_client()
def test_basic(request):
    global i
    i += 1
    if i < 2:
        1 / 0

Stacktrace:

INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/usr/local/lib/python3.6/site-packages/_pytest/main.py", line 209, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/usr/local/lib/python3.6/site-packages/_pytest/main.py", line 249, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "/usr/local/lib/python3.6/site-packages/pluggy/hooks.py", line 289, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "/usr/local/lib/python3.6/site-packages/pluggy/manager.py", line 68, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/usr/local/lib/python3.6/site-packages/pluggy/manager.py", line 62, in <lambda>
INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR>   File "/usr/local/lib/python3.6/site-packages/pluggy/callers.py", line 208, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/usr/local/lib/python3.6/site-packages/pluggy/callers.py", line 80, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/usr/local/lib/python3.6/site-packages/pluggy/callers.py", line 187, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/usr/local/lib/python3.6/site-packages/_pytest/main.py", line 270, in pytest_runtestloop
INTERNALERROR>     item.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem)
INTERNALERROR>   File "/usr/local/lib/python3.6/site-packages/pluggy/hooks.py", line 289, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "/usr/local/lib/python3.6/site-packages/pluggy/manager.py", line 68, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/usr/local/lib/python3.6/site-packages/pluggy/manager.py", line 62, in <lambda>
INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR>   File "/usr/local/lib/python3.6/site-packages/pluggy/callers.py", line 208, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/usr/local/lib/python3.6/site-packages/pluggy/callers.py", line 80, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/usr/local/lib/python3.6/site-packages/pluggy/callers.py", line 187, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/usr/local/lib/python3.6/site-packages/pytest_rerunfailures.py", line 176, in pytest_runtest_protocol
INTERNALERROR>     reports = runtestprotocol(item, nextitem=nextitem, log=False)
INTERNALERROR>   File "/usr/local/lib/python3.6/site-packages/_pytest/runner.py", line 93, in runtestprotocol
INTERNALERROR>     reports.append(call_and_report(item, "call", log))
INTERNALERROR>   File "/usr/local/lib/python3.6/site-packages/_pytest/runner.py", line 175, in call_and_report
INTERNALERROR>     report = hook.pytest_runtest_makereport(item=item, call=call)
INTERNALERROR>   File "/usr/local/lib/python3.6/site-packages/pluggy/hooks.py", line 289, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "/usr/local/lib/python3.6/site-packages/pluggy/manager.py", line 68, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/usr/local/lib/python3.6/site-packages/pluggy/manager.py", line 62, in <lambda>
INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR>   File "/usr/local/lib/python3.6/site-packages/pluggy/callers.py", line 208, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/usr/local/lib/python3.6/site-packages/pluggy/callers.py", line 80, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/usr/local/lib/python3.6/site-packages/pluggy/callers.py", line 182, in _multicall
INTERNALERROR>     next(gen)  # first yield
INTERNALERROR>   File "/usr/local/lib/python3.6/site-packages/pytest_sentry.py", line 53, in pytest_runtest_makereport
INTERNALERROR>     _report_flaky_test(item, call, old_exc_chain)
INTERNALERROR>   File "/usr/local/lib/python3.6/site-packages/pytest_sentry.py", line 100, in _report_flaky_test
INTERNALERROR>     hub = _resolve_hub_marker_value(item.get_closest_marker("sentry_client"))
INTERNALERROR>   File "/usr/local/lib/python3.6/site-packages/pytest_sentry.py", line 74, in _resolve_hub_marker_value
INTERNALERROR>     marker_value = marker_value.args[0]
INTERNALERROR> IndexError: tuple index out of range

Please, let me know if you need more information :)

benregn avatar Apr 08 '19 07:04 benregn

Hey, the sentry_client marker is only necessary if you want to override the client object on a per-test basis, instead of using PYTEST_SENTRY_DSN for all tests. It's not required for the example to work. If you do use it, it requires an argument.

On April 8, 2019 9:39:04 AM GMT+02:00, Tomas Thor Jonsson [email protected] wrote:

Hey,

I was trying out your plugin and I wanted to see if was setup properly, so I added the test case from your test to my test suite. When I ran it I got an IndexError.

Test case:

i = 0
@pytest.mark.flaky(reruns=2)
@pytest.mark.sentry_client()
def test_basic(request):
   global i
   i += 1
   if i < 2:
       1 / 0

Stacktrace:

INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File
"/usr/local/lib/python3.6/site-packages/_pytest/main.py", line 209, in
wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File
"/usr/local/lib/python3.6/site-packages/_pytest/main.py", line 249, in
_main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File
"/usr/local/lib/python3.6/site-packages/pluggy/hooks.py", line 289, in
__call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(),
kwargs)
INTERNALERROR>   File
"/usr/local/lib/python3.6/site-packages/pluggy/manager.py", line 68, in
_hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File
"/usr/local/lib/python3.6/site-packages/pluggy/manager.py", line 62, in
<lambda>
INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if
hook.spec else False,
INTERNALERROR>   File
"/usr/local/lib/python3.6/site-packages/pluggy/callers.py", line 208,
in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File
"/usr/local/lib/python3.6/site-packages/pluggy/callers.py", line 80, in
get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File
"/usr/local/lib/python3.6/site-packages/pluggy/callers.py", line 187,
in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File
"/usr/local/lib/python3.6/site-packages/_pytest/main.py", line 270, in
pytest_runtestloop
INTERNALERROR>     item.config.hook.pytest_runtest_protocol(item=item,
nextitem=nextitem)
INTERNALERROR>   File
"/usr/local/lib/python3.6/site-packages/pluggy/hooks.py", line 289, in
__call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(),
kwargs)
INTERNALERROR>   File
"/usr/local/lib/python3.6/site-packages/pluggy/manager.py", line 68, in
_hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File
"/usr/local/lib/python3.6/site-packages/pluggy/manager.py", line 62, in
<lambda>
INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if
hook.spec else False,
INTERNALERROR>   File
"/usr/local/lib/python3.6/site-packages/pluggy/callers.py", line 208,
in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File
"/usr/local/lib/python3.6/site-packages/pluggy/callers.py", line 80, in
get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File
"/usr/local/lib/python3.6/site-packages/pluggy/callers.py", line 187,
in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File
"/usr/local/lib/python3.6/site-packages/pytest_rerunfailures.py", line
176, in pytest_runtest_protocol
INTERNALERROR>     reports = runtestprotocol(item, nextitem=nextitem,
log=False)
INTERNALERROR>   File
"/usr/local/lib/python3.6/site-packages/_pytest/runner.py", line 93, in
runtestprotocol
INTERNALERROR>     reports.append(call_and_report(item, "call", log))
INTERNALERROR>   File
"/usr/local/lib/python3.6/site-packages/_pytest/runner.py", line 175,
in call_and_report
INTERNALERROR>     report = hook.pytest_runtest_makereport(item=item,
call=call)
INTERNALERROR>   File
"/usr/local/lib/python3.6/site-packages/pluggy/hooks.py", line 289, in
__call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(),
kwargs)
INTERNALERROR>   File
"/usr/local/lib/python3.6/site-packages/pluggy/manager.py", line 68, in
_hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File
"/usr/local/lib/python3.6/site-packages/pluggy/manager.py", line 62, in
<lambda>
INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if
hook.spec else False,
INTERNALERROR>   File
"/usr/local/lib/python3.6/site-packages/pluggy/callers.py", line 208,
in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File
"/usr/local/lib/python3.6/site-packages/pluggy/callers.py", line 80, in
get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File
"/usr/local/lib/python3.6/site-packages/pluggy/callers.py", line 182,
in _multicall
INTERNALERROR>     next(gen)  # first yield
INTERNALERROR>   File
"/usr/local/lib/python3.6/site-packages/pytest_sentry.py", line 53, in
pytest_runtest_makereport
INTERNALERROR>     _report_flaky_test(item, call, old_exc_chain)
INTERNALERROR>   File
"/usr/local/lib/python3.6/site-packages/pytest_sentry.py", line 100, in
_report_flaky_test
INTERNALERROR>     hub =
_resolve_hub_marker_value(item.get_closest_marker("sentry_client"))
INTERNALERROR>   File
"/usr/local/lib/python3.6/site-packages/pytest_sentry.py", line 74, in
_resolve_hub_marker_value
INTERNALERROR>     marker_value = marker_value.args[0]
INTERNALERROR> IndexError: tuple index out of range

Please, let me know if you need more information :)

-- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/untitaker/pytest-sentry/issues/1

untitaker avatar Apr 08 '19 16:04 untitaker

Of course! Just copy-pasted from the test file.

After removing the client marker there is still no error in Sentry :/

benregn avatar Apr 09 '19 07:04 benregn

What I specifically mean is that @pytest.mark.sentry_client() does not pass any argument, but I don't have any file in this repo that doesn't pass either a string or a Client object or None when using this marker.

How do you set the DSN to use?

untitaker avatar Apr 09 '19 07:04 untitaker

I'm using the PYTEST_SENTRY_DSN env var.

On Tue, Apr 9, 2019, 09:28 Markus Unterwaditzer [email protected] wrote:

What I specifically mean is that @pytest.mark.sentry_client() does not pass any argument, but I don't have any file in this repo that doesn't pass either a string or a Client object or None when using this marker.

How do you set the DSN to use?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/untitaker/pytest-sentry/issues/1#issuecomment-481135755, or mute the thread https://github.com/notifications/unsubscribe-auth/AAMk8m2SQzM1F8DU8UPX40pGFrJbAI5tks5vfEEmgaJpZM4chgQm .

benregn avatar Apr 09 '19 13:04 benregn