Research ways of displaying exceptions in @After functions in Allure

[occupant23]

At the moment an exception within an @After annotated method masks errors that came up during the real test method (@Test).

• [ ] Check if we have a bug in our setup?
• [ ] Research if there are way to configure this?
• [ ] Find out if this is the expected behavior of Allure? File an issue otherwise.
• [ ] Have a look at other JUnit annotations like @Before, @BeforeClass, @AfterClass.
• [ ] Come up with a solution to fix this. (e.g. A function likeSelenideAddons.wrapAssertionError that catches the exceptions and creates an Allure step with a special state and necessary information for each)

Start with the code below demonstration the issue:

package template.neodymium.tests.smoke;

import org.junit.After;
import org.junit.Assert;
import org.junit.Test;

import template.neodymium.tests.AbstractTest;

public class DemoTest extends AbstractTest
{
    @Test
    public void test() throws Throwable
    {
        Assert.assertFalse("This is another assertion", true);
        throw new RuntimeException("This is another failure");
    }

    @Test
    public void testInstanceOf()
    {
        Assert.assertTrue("This is an assertion", true);
        throw new RuntimeException("This is a failure!");
    }

    @After
    public void after1()
    {
        Assert.fail("after1");
    }

    @After
    public void after2()
    {
        Assert.fail("after2");
    }
}

[occupant23]

Please test this with the current dev version since we updated Allure.

[oomelianchuk]

• Our bug?:
  • no, this issue also takes place in the tests, written in pure Selenide
• Is there a way to configure?:
  • not directly (via configuration properties or so). Allure provides the Allure.getLifecycle() method to manipulate the logging, which allows us to customize some listeners to achieve the desired behavior. See customization suggestion in PR!166
• Expected behavior?
  • not clear (no pro or contras found)
• JUnit annotations & wrapper function:
  • IMO, it's better to use the listeners for this purpose because they guarantee execution at the exactly needed point and also don't require any additional effort from the library user.

[wurzelkuchen]

@kqmpetenz please check if this is still valid. Use the branch with the selenide update and try to upgrade allure. Maybe this issue is already resolved by time.

[wurzelkuchen]

issue is resolved in current allure version, closing issue