synpress

synpress copied to clipboard

Describe the bug I've successfully set up the Cucumber + Cypress bundle and now trying make Cucumber + Synpress work in a similar way.

However, I'm getting this error:

tasksetupMetamask, Object{5} CypressError Cypress detected that you returned a promise from a command while also invoking one or more cy commands in that promise.

The command that returned the promise was:

cy.task()

The cy command you invoked inside the promise was:

cy.then()

Because Cypress commands are already promise-like, you don't need to wrap them or return your own promise.

Cypress will resolve your command with whatever the final Cypress command yields.

The reason this is an error instead of a warning is because Cypress internally queues commands serially whereas Promises execute as soon as they are invoked. Attempting to reconcile this would prevent Cypress from ever resolving. Learn more

Stack trace is not very helpful:

    at cy.<computed> [as then] (http://localhost:53096/__cypress/runner/cypress_runner.js:160660:72)
From Your Spec Code:
    at Context.eval (http://localhost:53096/__cypress/tests?p=cypress\support\e2e.ts:17093:21)

My guess is, something's wrong with my config:

import { addCucumberPreprocessorPlugin } from '@badeball/cypress-cucumber-preprocessor';
import createEsbuildPlugin from '@badeball/cypress-cucumber-preprocessor/esbuild';
import createBundler from '@bahmutov/cypress-esbuild-preprocessor';
import synpressPlugins from '@synthetixio/synpress/plugins';
import { defineConfig } from 'cypress';

export default defineConfig({
  video: false,
  e2e: {
    specPattern: ['cypress/e2e/*.feature'],
    supportFile: 'cypress/support/e2e.ts',
    testIsolation: true,
    async setupNodeEvents(
      on: Cypress.PluginEvents,
      config: Cypress.PluginConfigOptions
    ) {
      synpressPlugins(on, config);

      await addCucumberPreprocessorPlugin(on, config);

      on(
        'file:preprocessor',
        createBundler({
          plugins: [createEsbuildPlugin(config)],
        })
      );

      // Make sure to return the config object as it might have been modified by the plugin.
      return config;
    },
  },
});

More precisely, I suspect that I should combine synpressPlugins(on, config) and await addCucumberPreprocessorPlugin(on, config) in a way so that the second command is only called once the first one finishes. However, await doesn't help with that and generally I'm not sure how to combine them correctly. May be pass the rest as a callback to synpressPlugins ~~as the second argument~~?

To Reproduce

1. Clone https://github.com/YakovL/synpress-cucumber,
2. Run npm init i,
3. Run npm run test (or just npx cypress open);
4. Run the only test (in Chrome).

Related repository https://github.com/YakovL/synpress-cucumber

Expected behavior The test should run.

Screenshots The error message and the stack trace covers it all, none informative screenshots.

Desktop (please complete the following information):

• OS: Windows 10 x64
• Synpress version: 3.7.0

Additional context #414 looks related

Ok, like I've reported in #764, I've added debugging in initialSetup in synpress/commands/metamask.js:

  async initialSetup(
    playwrightInstance,
    {
      secretWordsOrPrivateKey,
      network,
      password,
      enableAdvancedSettings,
      enableExperimentalSettings,
    },
  ) {
    console.log(`YL debug point 1, process.env.NETWORK_NAME is ${process.env.NETWORK_NAME}`)
    const isCustomNetwork =
      (process.env.NETWORK_NAME &&
        process.env.RPC_URL &&
        process.env.CHAIN_ID &&
        process.env.SYMBOL) ||
      typeof network == 'object';
    console.log(`YL debug point 1.1: isCustomNetwork is ${isCustomNetwork}`)

Suprisingly, it reports process.env.NETWORK_NAME to be undefined, although I've set it in .env. Do I have to load .env by myself?

PS Ok, I've installed dotenv and used dotenv.config() to fix this. Now execution doesn't seem to finish await playwright.init(); but also fails silently.

Making Synpress works with Cucumber id definitely a great idea! I will look into this issue when I have a chance. Thanks for sharing!

Right, so now that I've defeated several issues, the problem is boiled down to

1. the cypress run vs open issue;
2. I have to retest this, but it seems that unlike Cucumber cli, Cypress/Synpress doesn't suggest test boilerplates.

The first one is quite a problem, as it slows down the feedback loop (when writing tests) to an extent that I'd call it "breaks it".

I'll update the repo to make it work as I have it working locally now (haven't done yet).

These might be related: https://github.com/Synthetixio/synpress/issues/404

Current DX is a total nightmare. The following 2 issues ruin the workflow:

1. the broken watch mode (i.e. cypress open not working) means that the feedback loop for a change is several minutes, plus some of the info about errors is not available;
2. the reports don't include on which (Cucumber) step the test failed.

Combined with the fact that some failures are randomly reproduced/not reproduced due to timeouts (I'm dealing with a complex DApp with behavior quirks), I'm getting the following DX problems:

• very slow feedback loop, plus Synpress makes it hard to do something in parallel as it grabs focus to its Chrome windows/tabs multiple times per test;
• it gives really little info regarding where the error came from (step is unknown, debug info is not available since the window is closed on failure), and I have to put cy.pause() here and there trying to narrow down the issue (combine that with the random reproductivity to imagine the result).

I'll do my best to provide a minimal reproducible example, but for now I'm just outlining the problems and asking to prioritize #417. I'm also looking for a way not to close Cypress window once it fails: this would help determining the step on which the problem occured and do some in-place debugging (like in the watch mode).

PS @kasparkallas I don't really see any connection. Why you think so?

Ok, looks like putting

afterEach(function() {
  if (this.currentTest?.state === 'failed') {
	cy.pause();
  }
});

into a steps definition file is a working approach to beat the "not sure where the failure comes from" problem, as well as to access the sources by the stack trace). This helps a lot.

Another problem, a somewhat complicated one: https://github.com/cypress-io/cypress/issues/27437 – cy.visit fails when using Synpress and Cucumber on a page that uses Chatbase. I'll probably create a separate issue for this here (in Synpress repo) as well, since removing Synpress removes the issue

The reason is that synpress is/was using async/await inside the before hook. My solution was to copy-paste-modify the original supports file and remove async-await:

before(() => {
  if (!Cypress.env('SKIP_METAMASK_SETUP')) {
    cy.setupMetamask();
  }
});

Thanks @MichalLytek, this cures the complain about a promise indeed, created a PR for this. There are other problems with watch mode though, I'll report those later.