extent-framework
Failed scenarios not correct in tags
Hi, I just observed some issues on the library,
-
the failed number of scenarios is wrong in the tags section. For e.g.: I had 1 scenario failed for tag @world-shop-catalog-feature, but it appears to be 2 fails in tags section from Dashboard.
-
if for some reason appium server is failing, or a device is disconnected , the next scenarios will be skipped, but they won't record in the report. Maybe because the skipped case is ignored?
version - 1.0.11 Spark report
@zuzeac
- I am able to reproduce this
- The 2 cases are grouped, so both "skipped" and "pending" will go inside the same block of code.
Partial fix was on the API side https://github.com/extent-framework/extentreports-java/issues/139
Hi @anshooarora , thank you for your prompt replay and fix. for case 2, both skipped and pending will go inside the same block of code, but they won't be displayed in the report. And that's the issue here. How can we add them in the report? When I was using an old version of extent report, not cucumber, then all skipped scenarios were recorded in the report.
They should also be logged. Can you check with <version>1.0.12-SNAPSHOT</version>?
I just checked, by using 1.0.12-SNAPSHOT, but i do not see any fix, the Skipped scenario is not logged.
@zuzeac Please create a small project reproducing the issue so I can see the behavior.
Did you check the below as per your original post:
the failed number of scenarios is wrong in the tags section. For e.g.: I had 1 scenario failed for tag @world-shop-catalog-feature, but it appears to be 2 fails in tags section from Dashboard.
Is this working?
EDIT: Please make sure to use the latest snapshot and force refresh the project. If you are using Eclipse, you need to check the box below:
PS. This is what I see in the latest:
-
the fix first issue seems to not work, 1.0.12-SNAPSHOT, do i need to update any other library?
-
second issue: Oh, i got it, you got it wrong, the skipped steps from a scenario are logged in the report. I was reffering to the following: let's assume I have a suite of 10 scenarios/tests, then at the 5th scenario something broken, test execution is interrupted (the device is disconnected) then the rest of the 5th scenarios are not logged. They should be logged as Skipped scenarios.
I'm using IntelliJ, As far as I can see the 1.0.12-SNAPSHOT was downloaded in my .m2 repo. I will try to create a small project to reproduce it
Everything should be logged, I am not sure why it isn't. For me, if 1 scenario fails, the rest still execute as they normally would, they are not skipped by default.
Hi, it might be my configurations, i have a combinations of testng with cucumber annotations. I create a small project and all work as expected. I will look further into this, to figure it out. You can close the ticket, if i'll find something i'll let you know. Thanks
You can find a sample here https://github.com/zuzeac/appium-cucumber-extent-report.git . you'll need Appium installed and a phone connected to you machine. Just start the tests with mvn test, while the test starts, disconnect the device. And you will see that the next scenarios are not recorded. I think it's because the testng annotations *@beforeSuite, @BeforeMethod. When I used the cucumber annotations from hooks it worked ok.
@zuzeac I don't use Appium or other web test tools. I can test your code as-is without the need of any utility if you can help reproduce it. Also, there was a fix that went into the latest snapshot 1.1.0-SNAPSHOT which records pending tests as well. See here: https://github.com/extent-framework/extentreports-cucumber4-adapter/commit/7b027b95d260fe5ced254b55ce67f7e7df746235. Can you check it out?
hi @anshooarora, i test the 1.1.0-SNAPSHOT, I think it has something to do with the fact that i'm using TestNG annotations.
When using cucumber annotation, the scenarios which did not run, will be recorded as failed
When using TestNG annotations, the scenarios which did not run will not be recorded.
Cucumber:
TestNg:
