testng-team
Fix sporadic failure for “firstTimeOnly” methods
Ensure that the listeners for “firstTimeOnly” methods are executed ONLY if the config method was run else skip listener execution as well.
The test was failing because it relied on config listeners
Fixes the sporadic failure in https://github.com/krmahadevan/testng/actions/runs/3611418844/jobs/6085892554
Sorry but I don't catch the fix. What is the relation between firstTimeOnly and the execution of all results?
Why not having the same on before listener when lastTimeOnly is involved?
Any way to find a test case where the test result is shared and where we want to run the after listener every times? Maybe something with retryAnalyzer and/or data provider. Tbh, I'm surprised the fix is not breaking tests.
What is the relation between firstTimeOnly and the execution of all results?
The execution of results is related because our test is relying on listeners to gather info on what was executed and what wasn't.
Why not having the same on before listener when lastTimeOnly is involved?
I don't think we have any test that is doing that. That perhaps is why I didn't add.
Any way to find a test case where the test result is shared and where we want to run the after listener every times? Maybe something with retryAnalyzer and/or data provider.
I don't know. I will have to dig around to see if we can find something.
Tbh, I'm surprised the fix is not breaking tests. We have tests that are checking this.
Yes, I think we should take the time to dig a bit deeper. The problem here is more the feature than the failing test.
A good step could be to check feature tests and the more exhaustive as possible. #1711 and #792 can be considered in the test scenarii.
The fix will come later.