pytest-dev
Logs before rerun appear in html report
import logging
logger = logging.getLogger(__name__)
logger.setLevel(logging.DEBUG)
seq_num = 0
def seq():
global seq_num
seq_num += 1
return seq_num
def test_fail():
logger.info(f'Test log {seq()}')
assert False
pytest --reruns 1 --html=report.html
The report has logs of both runs:
def test_fail():
logger.info(f'Test log {seq()}')
> assert False
E assert False
test_my.py:17: AssertionError
------------------------------ Captured log call -------------------------------
INFO test_my:test_my.py:16 Test log 1
------------------------------ Captured log call -------------------------------
INFO test_my:test_my.py:16 Test log 2
It looks like there is logic in _process_logs to skip rerun logs, but this function is not called (maybe it was changed in pytest).
# Don't add captured output to reruns
if report.outcome != "rerun":
What's the expectation here? 🤔
Shouldn't the final result have all logs? Including those of reruns?
Maybe it looks like it doesn't on a trivial example, but in the real world the logs might contain crucial information to solve flaky logic.
Ok, maybe that makes sense if the logs for previous runs are available in those runs in the report.
Can you clarify what you mean when you say that "_process_logs" isn't called?
@orgads
I didn't say it isn't called, maybe it's called earlier, or maybe not in the right context, or maybe I misunderstood its purpose :)
From my point of view, only the logs of the last (re)run matter.
I would think the exact opposite, myself. Why don't you care seeing logs from previous failures that force a rerun? Those test cases are the most flaky, and therefore of the most interest.
Right, but if it failed 3 times, and each run produces many logs, having the logs duplicated is cumbersome. I assume that if all runs failed, then the last run shouldn't be much different than the previous ones.
We're running a plugin called pytest-rerunfailures. We might expect to see output from reruns :-)