CodeceptJS

CodeceptJS copied to clipboard

I am using the retryFailedStep plugin, but I am having a problem with the plugin not sending all configuration options down to the underlying retry mechanism. As specified, 10 retries and minTimeout of 10 seconds, but from the verbose output, there is only 5 retries and it does not appear to wait 10 seconds before attempting the first retry. I verified that enabled: false works as there is no retry attempted. It only appears to use default values.

Details

• CodeceptJS version: v3.0.5
• Operating System: Windows 10
• puppeteer
• Configuration:

    retryFailedStep: {
      enabled: true,
      retries: 10,
      minTimeout: 10000
    }

Test

Feature('Test Home');

Before(({ I }) => {
    I.amOnPage('http://homepage');
});

Scenario('Homepage', async  ({ I }) => {
    I.see("Test Content");

});

Output

Test Home --
    [1]  Starting recording promises
  Homepage
    I am on page "http://homepage"
    » [Url] http://homepage/
    I see "Test Content"
    [1] Retrying... Attempt #2
    [1] Retrying... Attempt #3
    [1] Retrying... Attempt #4
    [1] Retrying... Attempt #5
    [1] Retrying... Attempt #6
    [1] Error | Error
    [1] Error | Error
    [1] Starting <teardown> session
    [1] <teardown> Stopping recording promises
 » <screenshotOnFail> Test failed, try to save a screenshot

Hello, same issue here, I set the retry on 2, on my side, no change :/ Seems to be a regression on this maybe ?

Same her too ! I try different value of retries , no change.

Maybe a regression (like this https://github.com/codeceptjs/CodeceptJS/pull/1633)

Same here. Noticed this isse a long time ago in our repo, so it's not a very recent regression. Retries set to 10, but only the default 5 are actually happening.

Just a comment here regarding this. A little while back, I believe I narrowed it down to the actions in the puppeteer helper. It appears that the plugin would work as expected but gets overridden by the puppeteer helper setting a different retry in the async _before() function.

Just a comment here regarding this. A little while back, I believe I narrowed it down to the actions in the puppeteer helper. It appears that the plugin would work as expected but gets overridden by the puppeteer helper setting a different retry in the async _before() function.

Yes, I have the same issue with Playwright, see me comment https://github.com/codeceptjs/CodeceptJS/issues/3268#issuecomment-1151489381

The Playwright (since 3.2.2, see https://github.com/codeceptjs/CodeceptJS/commit/f55447296f41d8691889da8f452a57a7aa5f63ad) and Puppeteer helpers have hardcoded value retries: 5 (or retries: 3) in _before() and actually they can't have access to the retryFailedStep plugin's config (I see, because plugin is a part of a test config and not of helper's config) so it's not possible to override by a plugin's value (or use greater one). Note sure how to solve it.

A workaround is a patch of the helper, e.g. I use a pnpm patch for Plawyright

diff --git a/lib/helper/Playwright.js b/lib/helper/Playwright.js
index 33cb7bc48ae03ab5c3787aa6a58f65695355b635..750e3fe7b70e614c2b4630b0263c8dc04ba4ccc3 100644
--- a/lib/helper/Playwright.js
+++ b/lib/helper/Playwright.js
@@ -445,7 +445,7 @@ class Playwright extends Helper {
 
   async _before() {
     recorder.retry({
-      retries: 5,
+      retries: 10,
       when: err => {
         if (!err || typeof (err.message) !== 'string') {
           return false;

BTW I use the plugin retryFailedStep also with Appium tests and everything works fine there. I can set retries in my mobile test config and the value is really applied to tests (it overrides the default value from https://github.com/codeceptjs/CodeceptJS/blob/3.x/lib/plugin/retryFailedStep.js#L5).