vue components which import refs from external files break test isolation

[SIGSTACKFAULT]

Current behavior

vue components which import refs from external files break test isolation.

Desired behavior

vue components which import refs from external files do not break test isolation. (what else would you want me to say :P)

Test code to reproduce

HelloWorld.cy.ts

describe("HelloWorld", () => {
  it("accepts a value of msg", () => {
    cy.mount(HelloWorld, { props: { msg: "Hello Cypress" } });
    cy.get('[data-cy=foo]').should('contain', 'Hello Cypress')
  });

  it("shows the default if no msg is passed", () => {
    cy.mount(HelloWorld, { props: { msg: null } });
    cy.get('[data-cy=foo]').should('contain', 'foo') // actual value is 'Hello Cypress'
  });
});

HelloWorld.vue

<script setup lang="ts">
const props = defineProps<{
  msg: string | null
}>()

import { foo } from "@/foo";

if(props.msg != null)
  foo.value = props.msg;
</script>

<template>
  <div class="greetings">
    <h1 class="green">{{ msg }}</h1>
    <h3>
      You’ve successfully created a project with
      <a href="https://vitejs.dev/" target="_blank" rel="noopener">Vite</a> +
      <a href="https://vuejs.org/" target="_blank" rel="noopener">Vue 3</a>.
      <p>foo: "<span data-cy="foo">{{ foo }}</span>"</p>
    </h3>
  </div>
</template>

<style scoped>
/*...skipped... */
</style>

foo.ts

import { ref } from "vue";

export const foo = ref<string>("foo")

https://gitlab.com/SIGSTACKFAULT/cypress-test-isolation-broken

• git clone https://gitlab.com/SIGSTACKFAULT/cypress-test-isolation-broken.git
• cd cypress-test-isolation-broken
• npm install
• cy open --component > HelloWorld.cy.ts

Cypress Version

12.17.0

Node version

v20.2.0

Operating System

Manjaro

Debug Logs

(will attach file)

Other

tried putting cy.refresh() in beforeEach but it reloads the whole runner and causes the file to restart

[SIGSTACKFAULT]

cypress.log

[lmiller1990]

This looks like it works as expected. foo is a global, mutable singleton, which is mutated in the first test. Since the value isn't reset, the value remains whatever the first test set it to. This will be the same under any framework / test runner - the code is behaving as I'd expect.

Do you want the ref to be reset to foo between tests? This won't happen automatically - the test runner can't know (and should not know) about what side effects a test might have. If that's the desired behaviour, you'll need to handle the setup / teardown yourself. You could do:

beforeEach(() => {
  foo.value = "foo"
})

You could do this in your spec file, or your support file.

[SIGSTACKFAULT]

1. in that case, it's undocumented intended behaviour. this page ought to mention it.
2. IMO, nothing should break test isolation; there should be at least an option to reset this sort of globals between component tests. I find the current behavior highly unintuitive.

[SIGSTACKFAULT]

also, the current behaviour breaks the principles outlined here

We do this by cleaning up state before each test to ensure that the operation of one test does not affect another test later on. The goal for each test should be to reliably pass whether run in isolation or consecutively with other tests. Having tests that depend on the state of an earlier test can potentially cause nondeterministic test failures which make debugging challenging.

[lmiller1990]

Right, I agree the documentation could be better. To clarify, we do reset what we can:

unmounting the rendered component under test clearing cookies in all domains clearing localStorage in all domains clearing sessionStorage in all domains

We are able to reset browser state, but not application state. The way we are handling this sort of thing at Cypress is in our support file with a beforeEach. We reset the Pinia store, for example. You could import your ref and reset it.

There isn't any way for the test runner to know about your application and reset the state automatically, unfortunately. Another example that might make it more clear why we (and no runner) can provide this behavior is and application using a database. If one test creates a new record, there isn't any way for a test runner (Cypress, Jest, etc) to know the database was changed - you'd need to rollback or reset the database using a beforeEach hook.

A PR to updating the docs making the separation between state Cypress can reset, like browser state / cookies, and state that cannot be reset, such as global application state or side effects, would be welcome.

Hopefully this clarifies what's going on and gives you some ideas on how you might handle this in tests. Just to be clear, this isn't a Cypress limitation, but a general test runner limitation - no runner will be able to reset this kind of fine-grained application state.

[SIGSTACKFAULT]

is there something i can put in beforeEach which would reset all variables imported from ES modules? So I don't have to add every variable individually and don't have to duplicate their default values.

(cy.refresh() doesn't work because it reloads the whole runner and causes the file to restart)

[lmiller1990]

There isn't anyway way to do this - this is basically a limitation of JavaScript, there is no way to monitor the mutation across all ES modules and revert changes.

If you can share you real use case (as opposed to a minimal reproduction) I might be able to make some suggestions. For us, we use things like Vue Router and Pinia, and we create a new on in beforeEach in our support file.

[SIGSTACKFAULT]

I was hoping for, like, a version of cy.refresh() which refreshed the iframe but not the whole test runner

If you can share you real use case

https://gitlab.com/SIGSTACKFAULT/fleetwright-web-designer/-/blob/main/src/views/VTTView.vue
https://gitlab.com/SIGSTACKFAULT/fleetwright-web-designer/-/blob/main/cypress/unit/VTTView.cy.ts

[lmiller1990]

What variable/composable isn't reset that you'd like some input on? This code base looks really clean, nice job! Love to see Vue getting used like this.

Things like localStorage on window, like https://gitlab.com/SIGSTACKFAULT/fleetwright-web-designer/-/blob/main/cypress/unit/VTTView.cy.ts#L31 will reset.

[SIGSTACKFAULT]

the stuff in vtt_globals.ts are the worst offenders i think. maybe also state_machine.ts.

It's not like I couldn't manually reset those to undefined in a cy.beforeEach, it's just that I know damn well that someday i'll rename one of those variables and i'd like it if my tests didn't break

[lmiller1990]

Right, these guys: https://gitlab.com/SIGSTACKFAULT/fleetwright-web-designer/-/blob/bbd12b4795f4d17264de74b8cc893459172f4d8b/src/components/vtt/vtt_globals.ts#L9-13

I don't think we can do anything to reset this kind of application specific state. There's no way for the test runner to know about your module state, and what should/should not be reset, unfortuantely.

I'd say the most common pattern is what you described; likely your best option is a beforeEach in your support/component.ts file that handles and setup/teardown.

Sorry that we can't do more; this is not something we can implement from a technical perspective. I hope this gives you some ideas on how to move forward with your test suite.