AssertionError vs Exception

[cafykit]

Currently Allure fails only on AssertionError and all Exception is treated as Broken. This is a good feature but pytest reports them as Fail. This causes a bit of confusion on the report status.

An intermediate approach can be to allow for option on what constitutes a fail. A suggestion is to provide an API where one can register baseclasses that can be as failures. This provides for better control to use with infrastructures that can have some common analysis to look for additional failures.

def get_status(exception):
    if exception:
        if isinstance(exception, AssertionError) or isinstance(exception, pytest.fail.Exception):
            return Status.FAILED
        elif isinstance(exception, pytest.skip.Exception):
            return Status.SKIPPED
        return Status.BROKEN
    else:
        return Status.PASSED

=> Something like...

def get_status(exception):
    if exception:
        for base_class in allure.FailExceptionList:
            if isinstance(exception, base_class):  
                return Status.FAILED
        elif isinstance(exception, pytest.skip.Exception):
            return Status.SKIPPED
        return Status.BROKEN
    else:
        return Status.PASSED