testng icon indicating copy to clipboard operation
testng copied to clipboard

Specifying `invocationCount`, `threadPoolSize` and `successPercentage` causes tests to always pass

Open Yharooer opened this issue 5 months ago • 3 comments

Thank you for looking into #3170 so promptly, however I think there is another related issue.

TestNG Version

Note: only the latest version is supported

7.11.0-SNAPSHOT at f55ca8e

Expected behavior

Expect the following test to fail:

import org.assertj.core.api.Assertions.assertThat
import org.testng.annotations.Test

class TestTest {
    @Test(invocationCount = 10, threadPoolSize = 10, successPercentage = 99)
    fun testTest() {
        assertThat(true).isFalse
    }
}

Actual behavior

The tests pass instead.

Correct behaviour is achieved if successPercentage is 100, or if threadPoolSize is omitted.

Is the issue reproducible on runner?

  • [ ] Shell
  • [ ] Maven
  • [x] Gradle
  • [ ] Ant
  • [ ] Eclipse
  • [ ] IntelliJ
  • [ ] NetBeans

Test case sample

Please, share the test case (as small as possible) which shows the issue

import org.assertj.core.api.Assertions.assertThat
import org.testng.annotations.Test

class TestTest {
    @Test(invocationCount = 10, threadPoolSize = 10, successPercentage = 99)
    fun testTest() {
        assertThat(true).isFalse
    }
}

Contribution guidelines

Incase you plan to raise a pull request to fix this issue, please make sure you refer our Contributing section for detailed set of steps.

Yharooer avatar Sep 07 '24 19:09 Yharooer