okken
Add real-time error messages during testing
Sometimes it is useful to have some information about your test asserts during execution, particularly in long period tests. I propose to add real-time error messages using python's logging module.
This is a nice feature. I'd like to look into it. It needs to add real-time logging without duplicating the overall error report.
What do you think about this option?
We could add the flag --log-check-level and maybe the setting log_check_level, defaulting to NOTSET.
And if set, the failures are logged.
This would add real time logging support, but not change the default behavior.
yeah, of course it could be a great idea make this feature optional :)
I would add something else on my side. It would be wonderful to have logged the passes as well (optionally ofc) Here is a sample output:
test_sandbox.py::TestSandbox::test_case1
------------------------------- live log setup --------------------------------
13:17:04.319 [INFO] INIT
-------------------------------- live log call --------------------------------
13:17:04.320 [FAIL] check 2 == 1 FAIL1
13:17:04.853 [PASS] check 1 == 1 PASS
13:17:04.854 [FAIL] check 0 == 1 FAIL2
FAILED [100%]
testsuites\sandbox_test\test_sandbox.py:42 (TestSandbox.test_case1)
FAILURE: check 2 == 1: FAIL1
test_sandbox.py:44 in test_case1() -> self.steps.step_check()
scripts\steps.py:25 in step_check() -> check.equal(2, 1, "FAIL1")
FAILURE: check 0 == 1: FAIL2
test_sandbox.py:44 in test_case1() -> self.steps.step_check()
scripts\steps.py:27 in step_check() -> check.equal(0, 1, "FAIL2")
------------------------------------------------------------
Failed Checks: 2
What I did, I added two logging levels (PASS and FAIL) and just log them with message. I think adding different log levels should not be a problem. I can create them on root logger in my project, and just specify them from command line as you suggest
Why I even need a pass in test case? Well, I am using pytest for functional tests in my project, after execution I need to create a report what was tested. Having this in log will simplify the process
@okken, does it make sense ?
@kapis90 I understand the request. I also understand the want. It's really a separate request, I think, from the original want. PR #115 solves the original issue, hopefully, but doesn't address the pass logging.
I've added #116 for your request.
@kapis90 I understand the request. I also understand the want. It's really a separate request, I think, from the original want. PR #115 solves the original issue, hopefully, but doesn't address the pass logging.
I've added #116 for your request.
I agree with you, it can be treated as separate issue