pytest-html
pytest-html copied to clipboard
Published
20 hours ago •
pytest-dev
pytest-dev
pytest 3.0.0 captures all or nothing
trafficstars
With pytest 2.1.1 I could finegrain my logs and only capture stdout etc. But when I upgraded to v3.0.0 there was "No log output captured." allover, and I'm only able switch between all or nothing the -s and --capture settings:
-s --capture=sys
------------------------------Captured stderr call------------------------------
20201120142916.606|INFO|/builds/id-pro/access/access-scim-test/tests/test_03_scim_insert.py:53|lookup 100 users took 6.34sec
20201120142916.607|INFO|/builds/id-pro/access/access-scim-test/tests/test_03_scim_insert.py:55|users already preregistered on username:[]
20201120142926.660|INFO|/builds/id-pro/access/access-scim-test/tests/test_03_scim_insert.py:79|insert 100 users took 10.02sec
-------------------------------Captured log call--------------------------------
20201120142916|INFO|lookup 100 users took 6.34sec
20201120142916|INFO|users already preregistered on username:[]
20201120142926|INFO|insert 100 users took 10.02sec
How can I finegrain this in the new version, or is the a way I can hack this in a fixture.
Hi @fenchu, thank you for opening this issue!
I had a question before I dig any deeper though. Is your issue with pytest or pytest-html?
Also, could you please paste the output of pip list and the full command you used for your reproducer?
Hi
Thanks for looking into this. Would be nice to minimize test output. Or a workaround via conftest.py and fixtures.
(base) root@65005c93c5bd:/test/access-scim-test# pip freeze
ansi2html==1.6.0
astroid==2.4.2
atomicwrites==1.4.0
attrs==20.3.0
certifi==2020.11.8
cffi==1.14.3
chardet==3.0.4
colorama==0.4.4
cryptography==3.2.1
idna==2.10
iniconfig==1.1.1
isort==5.6.4
lazy-object-proxy==1.4.3
mccabe==0.6.1
packaging==20.4
pep8==1.7.1
pluggy==0.13.1
py==1.9.0
pycparser==2.20
PyJWT==1.7.1
pylint==2.6.0
pyparsing==2.4.7
pytest==6.1.2
pytest-html==3.0.0
pytest-metadata==1.10.0
requests==2.24.0
six==1.15.0
toml==0.10.2
urllib3==1.25.11
wrapt==1.12.1
pytest seem to be fine, this is what I like to log in the report: (below is ubuntu 16.04LTS container, it is exactly the same output in windows):
(base) root@65005c93c5bd:/test/access-scim-test# pytest -s --capture=sys --log-cli-level=INFO --tb=short tests/test_05_scim_filter.py -k 03 --webhost=test1
==================================================================================== test session starts =====================================================================================
platform linux -- Python 3.9.0, pytest-6.1.2, py-1.9.0, pluggy-0.13.1
rootdir: /test/access-scim-test, configfile: pytest.ini
plugins: metadata-1.10.0, html-3.0.0
collected 8 items / 7 deselected / 1 selected
tests/test_05_scim_filter.py::TestClass::test03_get_users_filter_username_eq[test1]
--------------------------------------------------------------------------------------- live log call ----------------------------------------------------------------------------------------
20201124103104|INFO|loaded 100 users from file:'mockusers.json'
20201124103104|INFO|filter: 'username eQ "10019029110"'
20201124103105|INFO|totalResults returned: 1
20201124103105|INFO|itemsPerPage returned: 100
20201124103105|INFO|results counted:1
20201124103105|INFO|['109824433']
PASSED [100%]
============================================================================== 1 passed, 7 deselected in 0.66s ===============================================================================
If I use pytest-html:
(base) root@65005c93c5bd:/test/access-scim-test# pytest -s --capture=sys --log-cli-level=INFO --tb=short tests/test_05_scim_filter.py -k 03 --webhost=test1 --html=test1.html
==================================================================================== test session starts =====================================================================================
platform linux -- Python 3.9.0, pytest-6.1.2, py-1.9.0, pluggy-0.13.1
rootdir: /test/access-scim-test, configfile: pytest.ini
plugins: metadata-1.10.0, html-3.0.0
collected 8 items / 7 deselected / 1 selected
tests/test_05_scim_filter.py::TestClass::test03_get_users_filter_username_eq[test1]
--------------------------------------------------------------------------------------- live log setup ---------------------------------------------------------------------------------------
20201124103220|INFO|creating directory:test1
--------------------------------------------------------------------------------------- live log call ----------------------------------------------------------------------------------------
20201124103220|INFO|loaded 100 users from file:'mockusers.json'
20201124103220|INFO|filter: 'username eQ "10019029110"'
20201124103221|INFO|totalResults returned: 1
20201124103221|INFO|itemsPerPage returned: 100
20201124103221|INFO|results counted:1
20201124103221|INFO|['109824433']
PASSED [100%]
--------------------------------------------------------------- generated html file: file:///test/access-scim-test/test1.html ----------------------------------------------------------------
============================================================================== 1 passed, 7 deselected in 1.51s ===============================================================================
and if I dig into the html file, it logs anything (lynx output of test1.html in container)
Results
Result Time Test Description Duration
No results found. Try to check the filters
Passed 2020-11-24 10:32:21.286 tests/test_05_scim_filter.py::TestClass::test03_get_users_filter_username_eq[test1] filter userName eq 0.50
-----------------------------Captured stdout setup------------------------------
using ext fixture:screenshots:None,urls:None,_dir_:None
-----------------------------Captured stderr setup------------------------------
20201124103220.762|INFO|/test/access-scim-test/conftest.py:128|creating directory:test1
-------------------------------Captured log setup-------------------------------
20201124103220|INFO|creating directory:test1
------------------------------Captured stderr call------------------------------
20201124103220.765|INFO|/test/access-scim-test/tests/test_05_scim_filter.py:36|loaded 100 users from file:'mockusers.json'
20201124103220.766|INFO|/test/access-scim-test/tests/test_05_scim_filter.py:68|filter: 'username eQ "10019029110"'
20201124103221.248|INFO|/test/access-scim-test/tools/scim.py:308|totalResults returned: 1
20201124103221.249|INFO|/test/access-scim-test/tools/scim.py:310|itemsPerPage returned: 100
20201124103221.249|INFO|/test/access-scim-test/tools/scim.py:314|results counted:1
20201124103221.250|INFO|/test/access-scim-test/tests/test_05_scim_filter.py:70|['109824433']
-------------------------------Captured log call--------------------------------
20201124103220|INFO|loaded 100 users from file:'mockusers.json' 20201124103220|INFO|filter: 'username eQ "10019029110"'
20201124103221|INFO|totalResults returned: 1
20201124103221|INFO|itemsPerPage returned: 100 20201124103221|INFO|results counted:1
20201124103221|INFO|['109824433']
it is the same with pytest-html3..1.1:
pytest==6.2.1
pytest-base-url==1.4.2
pytest-clarity==0.3.0a0
pytest-html==3.1.1
pytest-metadata==1.10.0
pytest-mock==3.3.1
pytest-variables==1.9.0
pytest-watch==4.2.0
pytest 6.2.1 and pytest-html 2.1.1 works fine, so I'm pretty sure it is in pytest-html
I forked the v3.1.1 and added a filter so it only captures the stderr or file, also sets screenshots on a new line after log at a fixed size also clicking on screenshots open in new window and some more robust decoding of files I have 30+ tests using this version now. the diff enclosed for the hack:
git diff 86cfadc43cb0e9068be1561ffdb7ec447f5a85d5 805e3b6b84fb50520d3e4a7c90e64639f2525173 .\src\pytest_html\result.py
diff --git a/src/pytest_html/result.py b/src/pytest_html/result.py
index da0d240..3069057 100644
--- a/src/pytest_html/result.py
+++ b/src/pytest_html/result.py
@@ -9,8 +9,7 @@ from html import escape
from os.path import isfile
from _pytest.logging import _remove_ansi_escape_sequences
-from py.xml import html
-from py.xml import raw
+from py.xml import html, raw
from . import extras
from .util import ansi_support
@@ -80,6 +79,8 @@ class TestResult:
return order.index(self.outcome) < order.index(other.outcome)
def create_asset(self, content, extra_index, test_index, file_extension, mode="w"):
+ if not content:
+ return None
asset_file_name = "{}_{}_{}.{}".format(
re.sub(r"[^\w\.]", "_", self.test_id),
str(extra_index),
@@ -175,14 +176,17 @@ class TestResult:
duration_as_gmtime = time.gmtime(report.duration)
return time.strftime(duration_formatter, duration_as_gmtime)
- def _populate_html_log_div(self, log, report):
+ def _populate_html_log_div(self, log, report, filter:list=['stderr']):
if report.longrepr:
# longreprtext is only filled out on failure by pytest
# otherwise will be None.
# Use full_text if longreprtext is None-ish
# we added full_text elsewhere in this file.
+ # filter is either stderr or log or both in a list
+
text = report.longreprtext or report.full_text
for line in text.splitlines():
+
separator = line.startswith("_ " * 10)
if separator:
log.append(line[:80])
@@ -196,9 +200,7 @@ class TestResult:
for section in report.sections:
header, content = map(escape, section)
- log.append(f" {header:-^80} ")
- log.append(html.br())
-
+ #print(f"report.section header:\"{header}\"")
if ansi_support():
converter = ansi_support().Ansi2HTMLConverter(
inline=False, escaped=False
@@ -207,8 +209,13 @@ class TestResult:
else:
content = _remove_ansi_escape_sequences(content)
- log.append(raw(content))
- log.append(html.br())
+ if filter:
+ for f in filter:
+ if re.search(f"Captured {f} ", header):
+ log.append("-" * 10 + header + "-" * 10)
+ log.append(html.br())
+ log.append(raw(content))
+ #log.append(html.br())
def append_log_html(
self,
@@ -250,22 +257,27 @@ class TestResult:
)
html_div = html.a(
- raw(base_extra_string.format(extra.get("content"))), href=content
+ raw(base_extra_string.format(extra.get("content"))), href=content, target="_blank"
)
elif self.self_contained:
src = f"data:{extra.get('mime_type')};base64,{content}"
html_div = raw(base_extra_string.format(src))
else:
- content = b64decode(content.encode("utf-8"))
- href = src = self.create_asset(
- content, extra_index, test_index, extra.get("extension"), "wb"
- )
- html_div = html.a(
- raw(base_extra_string.format(src)),
- class_=base_extra_class,
- target="_blank",
- href=href,
- )
+ try:
+ content = b64decode(content.encode("utf-8"))
+ except Exception as e:
+ print("Error: unable to decode data")
+ return None
+ else:
+ href = src = self.create_asset(
+ content, extra_index, test_index, extra.get("extension"), "wb"
+ )
+ html_div = html.a(
+ raw(base_extra_string.format(src)),
+ class_=base_extra_class,
+ target="_blank",
+ href=href,
+ )
return html_div
def _append_image(self, extra, extra_index, test_index):
Hi ,
is there anything about this issue ? it blocks me for upgrading to pytest-html v3.0.0 .
I want to capture only the log calls, instead its all or nothing indeed.
my packages are these but got same issue:
Package Version
----------------------------- -----------
alabaster 0.7.12
ansi2html 1.7.0
astroid 2.11.6
asttokens 2.0.5
......
pytest 7.1.2
pytest-cov 3.0.0
pytest-forked 1.4.0
pytest-html 3.1.1
pytest-icdiff 0.5
pytest-metadata 2.0.1
pytest-mock 3.7.0
pytest-sugar 0.9.4
pytest-tldr 0.2.4
pytest-xdist 2.5.0
python-dateutil 2.8.2
python-docx 0.8.11
python-gitlab 3.5.0
Sorry that you're having issues and that this has lingered for so long.
I will make sure to revisit this for v4.x which is going to be in beta in a week or two.
Hopefully, with the complete rebuild we're doing for v4, this won't be an issue.
