`logging.error` calls from non-robot code are treated as robot errors

[anandautam1]

Exception thrown from another import is not being suppressed when calling pytest.raises I tested the scenario where a module right away throwing the import - and that works fine

OS: Running on windows

requirements:

python-can==3.3.4 pytest_robotframework==4.2.3

import can
import pytest
from robot.errors import HandlerExecutionFailed

def test_exception_from_other_module():
    with pytest.raises(OSError):
        bus = can.ThreadSafeBus(app_name='dummy', channel='vcan0', bustype='socketcan')

Without pytest_robotframework installed i dont get a failure

With pytest_robotframework installed i failed

I got:

================================== FAILURES ===================================
______________________ test_exception_from_other_module _______________________
robot errors occurred and were caught by pytest-robotframework:

- fcntl not available on this platform
---------------------------- Captured stderr call -----------------------------
[ ERROR ] fcntl not available on this platform
[ WARN ] libc is unavailable
------------------------------ Captured log call ------------------------------
ERROR    can.interfaces.socketcan.socketcan:socketcan.py:22 fcntl not available on this platform
WARNING  can.interfaces.socketcan.socketcan:socketcan.py:55 libc is unavailable

[anandautam1]

@DetachHead could you help me investigate ?

[DetachHead]

so it looks like what's happening here is:

1. the can module calls logging.error with that message
2. robotframework seems to listen for logging.error calls and reports them as robot errors
3. since robot internal errors and errors from listeners are handled by robot in the exact same way, pytest-robotframework has checks to listen for robot errors and raise an exception at the end of the pytest session if any occurred

robot handles errors in the stupidest way imaginable. ideally there'd be a way to differentiate random logging.error calls from internal errors that come from robot itself. not sure what the best way to fix this would be unfortunately

[anandautam1]

@DetachHead can we detect whether the logging.error is actually within a pytest.raises and not taken account the logging.error from that section of the robot log ? i'm not quite sure how deep the pytest listener is actually being implemented.