swagger-editor icon indicating copy to clipboard operation
swagger-editor copied to clipboard

SwaggerEditor@next: disabled flaky edtior-persistence e2e test

Open char0n opened this issue 2 years ago • 1 comments

File in question: plugin.editor-persistence.spec.js

Basically the flakyness only manifest during the release. I couldn't not repro locally or in PR.

char0n avatar Jul 25 '22 06:07 char0n

I was able to reproduce this yesterday.

I'm on Mac OS X 11.6.7. The following screenshots are results from npm run cy:ci (after building locally).

Couple of scenarios... but note that the errors occur on the assertion for .should('contains.text', '2.3.0'), and BEFORE the cy.reload().

1. current

e.g. .get('.monaco-editor .view-lines')

temp-cy-ci-ori-EditorPersistencePlugin -- should reload while keeping text change from 2 4 0 to 2 3 0 (failed)

This screenshot matches the error from Github Actions in the PR.

2. change to a specific view line

e.g. .get('.monaco-editor .view-lines > :nth-child(1)')

temp-cy-ci-try-fix-get-specific-line-EditorPersistencePlugin -- should reload while keeping text change from 2 4 0 to 2 3 0 (failed)

3. Electron ok!

Setting the npm script to use electron, --browser electron, this test PASSES reliably for me

4. Firefox fails all!

Separately, choosing to run Firefox from cy:dev fails ALL Cypress tests. Both Chrome and Electron options however all pass reliably. Although this issue was for CI, not dev, still surprising to see the failures on Firefox. I did not try Firefox on CI.

Other notes:

  • with the entire definition with the error "red squiggly" (also in the validation pane), that there appears to be errors returned by the ApiDOM parser (or at least a delay in returning an expected validated document). Interestingly, we don't see this behavior elsewhere, e.g. plugin.validation-pane.spec.js.
  • That said, we are forcing a timeout wait already of 10000, which is counter to the Cypress best practices
  • I tried adding a timeout to cy.get('.monaco-editor .view-lines > :nth-child(1)', { timeout: 5000 }) for the post-text change. No effect.

tim-lai avatar Aug 12 '22 18:08 tim-lai