Support rerunning single failed scenario from plugin window

[ashafey]

If a DataProvider provides 100 scenarios for 1 test, and 99 scenarios pass but 1 fails, there's no way to rerun the failed scenario from Eclipse without rerunning the other 99 scenarios.

The same problem doesn't exist if testng-failed.xml is used to rerun the failing tests from command line. So this is a shortcoming of the Eclipse plugin, not TestNG.

[BartMan999]

Is this in fact as intended (but maybe poorly designed)

The 'All Tests' and 'Failed Tests' tabs seems to refer to @Test methods whereas it seems the 'Rerun Last Test' and 'Run Failed Test' rerun everything, presumably as 'Test' in this instance is referring to the <test> xml tag??

[missedone]

didn't have any chance to dig into 'rerun' on result view, I think it's always not too late to correct the things, any suggestion and pull request is welcome :)

[BartMan999]

@missedone I guess my suggestion would be a new button that runs only failed @Test methods (the ones displayed on the Failed Tests tab).

Per above the current 'Run Failed Test' button seems to run all @Test methods, within the <test> that contained any failures. As @ashafey described the ability to re-run just the failed @Test methods would be very welcome as re-running 100 @Test when all you really want to do is re-run a handful of failed @Test is frustrating currently entails specifying a new configuration containing each method you'd like to re-run.

[missedone]

thanks for the suggestion, will have a look later.

[missedone]

@ashafey how did you make it with command line, to re-run the failed test only?

I can't reproduce it with TestNG 6.9.12, 6.9.13,

import org.testng.annotations.DataProvider;
import org.testng.annotations.Factory;
import org.testng.annotations.Test;

public class TestA
{
    private int n;

    @DataProvider( name = "dp" )
    public static Object[][] dp(  )
    {
        return new Object[][]
               {
                   { 1 },
                   { 2 },
                   { 3 }
               };
    }

    @Factory( dataProvider = "dp" )
    public TestA( int n )
    {
        this.n = n;
    }

    @Test
    public void test1(  )
               throws InterruptedException
    {
        Thread.sleep( 1000 );
        assert( n != 2 );
    }

//    public String toString()
//    {
//        return String.valueOf(n);
//    }
}

and here is the generated testng-failed.xml

cat target/surefire-reports/testng-failed.xml
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite verbose="0" name="Failed suite [mymixed.TestA]">
  <test name="Command line test(failed)">
    <classes>
      <class name="mymixed.TestA">
        <methods>
          <include name="test1"/>
        </methods>
      </class> <!-- mymixed.TestA -->
    </classes>
  </test> <!-- Command line test(failed) -->
</suite> <!-- Failed suite [mymixed.TestA] -->

it only provided that the failed test method name, nothing more about the data provider. even I re-run it: mvn -e -Dsurefire.suiteXmlFiles=arget/surefire-reports/testng-failed.xml test the testng-result shows all tests of 'test1' got re-run: cat target/surefire-reports/testng-results.xml

[missedone]

@BartMan999 as we can see the generated testng-failed.xml contains the failed test method only, but not any data provider context, hence it's really no way to know which specific test need to be rerun. for now I don't have any solution in mind, but will try to see if any TestNG listener can help gathering more context for re-run the failed data provider driven test cases.

[SebastianSolidwork]

@missedone Can't this be achieved with the invocation-number? I'm looking for this feature.