cypress-plugin-retries
cypress-plugin-retries copied to clipboard
kuceb
Show first error upon failing test
Hi there! We have our CI pipeline configured with CYPRESS_RETRIES: 2. In case the 3 of tries fail, just the last error is shown in the log; but is there a way to show all errors (the 3 errors, in our case)?
Thanks!
I think it'd be useful to have some terminal output so it's we can tell which jobs are retrying
Hey @Bkucera! Yes, I mean the terminal reporter, sorry!
The thing is, our scenarios sometimes fails (it could be a bug, it could be an issue after a change in the code, or even a timeout in the pipeline), and then they retry. But we have seen that sometimes, because of how our tests are designed, they fail in the retry because of another thing: for example, we are not expecting to see something that is going to be created later in that test, but as the 1st scenario has already created it before failing, it's already there and it fails. In this specific cases, it's not easy and first sight to spot what has really failed.
Not sure if my explanation was clear 😅 .
EDIT: I have just seen you added something new about the retries in the terminal yesterday; I'll take a look, as this looks just what I was looking for 😄 Many thanks!
@baronvonbirra it could be sensible to hold onto the first error and if all retries fail, show that one instead of the last one. That way you'll most likely see the real failure.
@Bkucera yes, that's a good idea and totally fits our needs. Is there currently a way to achieve it? 🤔
By the way, I just upgraded to v.1.5.0 and introduced the retry logging, and it's really nice. If we could get the error log for the 1st fail as you say, we will be reaaaally happy. 😄
upgraded the plugin retry version and attempting to add retry logging but for some reason but doing module.exports=(on) => { require('cypress-plugin-retries/lib/plugin')(on); }
shows nothing about retries on my terminal, All i see is the barrage of get and post requests
Do we know an eta for this? I am extremely interested in seeing the error for first failure too?
Hey, I am using a custom reporter and emits test results to our own dashboard and we need to have it capture reties too (and group them by some unique ID if possible) - any way to enable reporting of all failed retries? that would be very helpful!
@gilgold I didn't want to hack in emitting extra events to try to keep the surface area of this plugin small.
But, if you want to hack on top of this, you can handle the logRetry task this plugin adds as done here https://github.com/Bkucera/cypress-plugin-retries/blob/master/lib/plugin.js