cypress icon indicating copy to clipboard operation
cypress copied to clipboard

feat: add testIsolation option and support config override levels

Open emilyrohrbough opened this issue 1 year ago • 3 comments

  • Closes #22279

  • add testIsolation option - when experimentalSessionAndOrigin=false, the default value is lax. When experimentalSessionAndOrigin=true, this the default value is strict.

    • lax - this is the current test isolation behavior in cypress - clock, viewport, cookies, local storage, etc continue to be reset before each tests
    • strict - all rests from lax plus clearing the page between each test by visiting about:blank. This ensures app state is reset/clean for each test.
  • add support for config override levels - this allows each option to determine what override levels are supported. The override levels are:

    • code - when either the support file or spec file is loaded and config is altered with Cypress.config()
    • test:before:run - config altered during event handler with Cypress.config()
    • test:before:run:async - config altered during event handler with Cypress.config()
    • suite - configuration provided as suite override describe('...', { ...configOverides }, ()=>{})
    • test- configuration provided as test override it('...', { ...configOverides }, ()=>{})
    • runtime - configuration altered from runnable (hook or test execution) using Cypress.config()
    • never - configuration is read-only and cannot be altered without restarting Cypress

User facing changelog

TO DO

Additional details

  • discovered test:before:run does not handle thrown errors correctly (#23039) so config validation is currently skipped for this hook.

How has the user experience changed?

  • Improved error messaging
  • Ability to set the testIsolation intensity when experiementalSessionAndOrigin=true. this preps for when this goes GA and users don't want to update tests.

PR Tasks

  • [x] Have tests been added/updated?
  • [x] Has the original issue (or this PR, if no issue exists) been tagged with a release in ZenHub? (user-facing changes only)
  • [x] Has a PR for user-facing changes been opened in cypress-documentation? https://github.com/cypress-io/cypress-documentation/pull/4634
  • [x] Have API changes been updated in the type definitions?

emilyrohrbough avatar Aug 01 '22 13:08 emilyrohrbough

Thanks for taking the time to open a PR!

cypress-bot[bot] avatar Aug 01 '22 13:08 cypress-bot[bot]



Test summary

4961 0 59 0Flakiness 1


Run details

Project cypress
Status Passed
Commit 2291a29056
Started Aug 12, 2022 4:46 PM
Ended Aug 12, 2022 4:59 PM
Duration 12:29 💡
OS Linux Debian - 11.3
Browser Electron 102

View run in Cypress Dashboard ➡️


Flakiness

cypress/e2e/cypress/proxy-logging.cy.ts Flakiness
1 Proxy Logging > request logging > xhr log has response body/status code when xhr response is logged second

This comment has been generated by cypress-bot as a result of this project's GitHub integration settings. You can manage this integration in this project's settings in the Cypress Dashboard

cypress[bot] avatar Aug 01 '22 15:08 cypress[bot]

@chrisbreiding Added in links to the TODO comments here: https://github.com/cypress-io/cypress/pull/23040/files#diff-47d1dcf44e2076653a4be7d397029774e2b76e083b93d4deeed010a280f62af6R181

emilyrohrbough avatar Aug 08 '22 17:08 emilyrohrbough

Dismissing my review since a lot has changed since, will re-review once conflicts are resolved.

lmiller1990 avatar Aug 10 '22 23:08 lmiller1990