Allow ESM stubbing for functions in Vite

[adamdehaven]

What would you like?

Utilizing Vite + Vue and Cypress Component Test Runner, how would you stub a composable function since you can't stub the default export?

I can't find a decent example that doesn't utilize Babel, and the only solution we have come up with is exporting an object with methods that can be stubbed, which, to be honest, would be a large refactor.

When stubbing the default export, as shown below, an error is thrown: ESModules cannot be stubbed. I know this is a valid error (here's a great write-up for reference); however, we need the ability to stub the default exports of imported modules.

// Composable function
import { ref } from 'vue'

export default function useToggle (initialValue = false) {
  const enabled = ref(initialValue)

  return { 
    enabled
  }
}

// Component usage
import useToggle from '../composables/useToggle'
.... 
setup(props) {
  const { enabled } = useToggle(false)

  onMounted(() => console.log(enabled.value)
}

// test.spec.ts
import { ref } from 'vue'
import useToggle from '../composables/useToggle'

// This doesn't work, and we're not using Babel since switching to Vite
// Throws an error: `ESModules cannot be stubbed`
cy.stub(useToggle, 'default').returns(
  { 
    enabled: ref(true),
  }
)

Why is this needed?

It's a standard in Vue 3 to move stateful logic into composable functions. Composables typically export a default function (not an object) and devs need the ability to stub the outputs of a composable.

There is currently a workaround; however, it would require refactoring large blocks of code within our (and most) application whereby the composable exports a utility function that is used to set the returned values. This workaround (shown below) is a bit cumbersome and requires a very explicit, non-standard way of writing composable functions.

Here is the same composable provided above, rewritten for the workaround (not ideal):

// Composable function
import { ref, Ref } from 'vue'

export const getToggleData = {
  enabled: (initialValue = false): Ref<boolean> => ref(initialValue)
}

export function useToggle(initialValue = false) {
  const enabled = getToggleData.enabled(initialValue)

  return {
    enabled,
  }
}

// Component usage
import useToggle from '../composables/useToggle'
.... 
setup(props) {
  const { enabled } = useToggle(false)

  onMounted(() => console.log(enabled))
}

// test.spec.ts
import { ref } from 'vue'
import { getToggleData } from '../composables/useToggle'

// This works, but isn't ideal due to the way the composable has to be written (not the norm)
cy.stub(getToggleData, 'enabled').returns(true)

Other

No response

[adamdehaven]

@JessicaSachs @lmiller1990 let me know if I can provide any additional info

[JessicaSachs]

Thanks for opening this issue. We should be doing what the Vitest plugin does to enable ESM mocking.

[lmiller1990]

The Vitest code doesn't look too bad. It might be a good solution for now (to help vite-dev-server) users. Long term, I wonder what's the best option for people who (eventually) want to write e2e tests using native ESM modules, or another dev server?

A dev-server specific mock/stub doesn't seem ideal, although happy to look into this in the short term until we better know how to handle the general problem of ESM module mock/stubs.

We will discuss internally and figure out the logistics/prioritization of this!

[FelipeLahti]

Can we include allowing esm stubbing functions in NextJS as well? Is that on the roadmap? @JessicaSachs

That's the only thing preventing us to use the new SWC compiler on NextJS. Currently, we're using the following .babelrc to make stubbing work.

{
  "presets": ["next/babel"],
  "plugins": ["@babel/plugin-transform-modules-commonjs", "babel-plugin-styled-components"]
}

[JessicaSachs]

👋 You want @baus or @ZachJW34 for Next-y or roadmap planning things.

[illegalnumbers]

Is there any update for the latest version of Cypress? When trying something similar with a composable I get the error Cannot stub non-existent own property default rather than the ESModules cannot be stubbed. When trying to stub out apply or call it seems like the function doesn't actually get replaced in the component.

[adamdehaven]

@illegalnumbers as a workaround, I export the composables from another file, then stub.

// composables/index.ts
import useOrg from './useOrg'

export {
  useOrg
}

// Component file (usage example)
import composables from '../composables'

const org = composables.useOrg()

// Test spec file
import composables from '../composables'

cy.stub(composables, 'useOrg').returns({ data: [] })

[lmiller1990]

Sorry fam, we did not work on ESM stubbing yet. I'd really like to see this implemented, I hope we can look at it soon.

Vitest supports this but they run everything in a Node.js context. I think the first step is looking at what they do and finding out if we can do the same thing.

Is there any other runners supporting ESM stubs? Jest now has ESM support natively - I wonder how they implement it? I'm thinking the most likely rely on the Node.js module resolution, too - I don't know of any browser runners that support ESM and stubs.

If anyone wants to explore this, I can definitely help out or make some suggestions. Especially if someone can find a reference implementation, I can assist with the "here is where you need to put the code in Cypress".

[iambumblehead]

I'm interested in adding browser support to esmock. it would be awesome if someone would add a cypress test folder alongside esmock's other test folders, and inside the test folder, add a passing test and a broken/failing test that uses esmock in the right place to try and browser-import a module with mock import tree.

[lmiller1990]

Oh nice! Hadn't seen this project, this looks promising.

[lmiller1990]

I think it's time to tackle this, going to get some resources into this in our next sprint. Not sure on the complexity or how best to solve this, but we can at least start investigating.

[lmiller1990]

Hey team! Please add your planning poker estimate with Zenhub @astone123 @marktnoonan @mike-plummer @warrensplayer @ZachJW34

[muratkeremozcan]

There is a nice repo we can test this on once delivered: https://github.com/muratkeremozcan/tour-of-heroes-react-vite-cypress-ts. Enable the lines with the comment // TODO: wait for https://github.com/cypress-io/cypress/issues/22355

[lmiller1990]

Hey team! Please add your planning poker estimate with Zenhub @astone123 @marktnoonan @mike-plummer @warrensplayer @jordanpowell88

[mike-plummer]

Update on ESM stubbing

Unfortunately this is a non-trivial problem due to how ES Modules are designed. The ESM spec requires modules to have an immutable namespace, and since Cypress component tests run inside the browser and use the browser's module loader to ensure spec compliance it isn't possible to do something like this:

import MyModule from './module.js'

// `MyModule` is a sealed namespace, can't resassign/add/delete direct members
MyModule.something = 'somethingElse'

Other testing tools like Jest & Vitest work around this in a couple different ways:

1. Replacing the Node module loader so that modules are mutable This nicely sidesteps the issue for tools that run tests within Node, but also means your code isn't spec-compliant when it runs. This could disguise actual problems in your tests.

2. Mocking an entire module for the duration of a spec Cypress APIs are designed to allow ad-hoc stubbing/spying throughout a test (see cy.stub and cy.spy), and we would like to maintain the capability. Replacing an entire module for the whole spec is a very different way of structuring tests and can be a bit restrictive depending on your application structure and testing use case.

What are your options today?

Rest assured, we are working on a way to support this in a way that doesn't have a massive breaking change and doesn't change the way your code runs. In the interim, there are a couple workarounds that I'll try to outline here:

1. Export wrappers to sidestep module immutability As stated above, this doesn't work:

   /// module.js
   export function myFunc() {}
   
   // module.spec.js
   import * as MyModule from './module.js'
   cy.stub(MyModule, 'myFunc')
   

   However, this does work:

   /// module.js
   function myFunc() {}
   export const MyModule = {
       myFunc
   }
   
   // module.spec.js
   import { MyModule } from './module.js'
   cy.stub(MyModule, 'myFunc')
   

   This is because the module namespace is immutable, but the module can export members that are mutable. In this case, the wrapper object MyModule is mutable, allowing us to stub anything within it. This requires you to structure and use your modules in a particular way and it is only an option for modules under your control.

2. Use an importmap If you have modules that you always want mocked out and are using a fairly modern browser you can add an importmap to your component-index.html to tell the browser to resolve a custom implementation anytime it is requested. Note that this will impact all uses of the module throughout your test suite, but it is an easy way to replace modules you don't want running in your tests or that you want to behave slightly different in all tests.

   Note that the "mock" implementation will itself be loaded by the browser as an ES Module, so it will be subject to the same restrictions as the original. This means you can't use cy.stub on any namespace member of the "mock" implementation, but you can write any logic you want into the mock to be shared across all of your tests.

3. Avoid stubs & spies Stubbing and spying can be an anti-pattern in testing depending on how they're used. You might consider whether it's possible to refactor a component to split out behavior you want to stub/spy into a separate component/hook/etc.

Is Cypress considering any changes?

Short answer - we aren't sure yet. We're kicking around a few ideas, and if you know of another we'd love to hear about it.

To get the same sort of behavior as Jest and others we could introduce a new API like cy.mock('./module.js', {... }) to mock a module for the entirety of a test. This would be a bit less capable than our existing APIs and could potentially be a breaking change. There are also potential issues around how this could work with things like custom commands since the mocking would have to occur before anything accesses that module. Finally, this sort of API would really just be a wrapper around the importmap idea above so you already have the capability to do it, just with a bit more configuration.

Another option we're considering is a plugin that rewrites ES Modules as they're served by your dev server so that they're mutable. This is a fairly complex thing to get right, and we aren't 100% sold on this being the right thing to do. One of the major principles of Cypress is that we try to be as standards-compliant as possible so we don't accidentally hide problems in your code. We'd love to hear your thoughts on this approach, and if anyone knows of a tool that already does this (or is interested in writing one) let us know!

[lmiller1990]

We will have an experimental Vite plugin available this month (April 2023) that will let cy.stub and cy.spy work with Vite and ESM. We will share that here once it's ready to go. After some initial testing, if not blockers/problems emerge, we can make it part of the core offering. It's happening 💯

[adamdehaven]

@lmiller1990 one place where this is currently a bigger blocker is when trying to stub the composables exported by vue-router, i.e. useRoute and useRouter within components.

Other than wrapping the Vue Router composables as outlined in the example from the original post, is there another way to stub these within a component test?

[mike-plummer]

@adamdehaven To my knowledge you have three options at the moment:

1. As you reference, structure your imports/exports using a wrapper object to allow mutability
2. Use an importmap to have the browser completely replace the vue-router library with an implementation of your own. This replacement still wouldn't be stubbable, but you could build any implementation you like into it.
3. Configure your bundler to deliver CJS instead of ESM

We are working on an approach that should allow traditional cy.stub/cy.spy use with ESM - we're hoping to have an initial solution together in the next couple weeks. It will be very "alpha" and likely take the form of an additional plugin to apply to Vite when running within Cypress; if you'd be interested in helping us test it out that would be awesome

[adamdehaven]

if you'd be interested in helping us test it out that would be awesome

💯 yes

[andrew-productiv]

I would also be super interested in helping test this out - I very nearly ended up writing my own Vite plugin to add this kind of support a few months ago, but got pulled into other things and this project ended up in my backlog. Please let me know if there are ways I can help make this successful! 😄

[lmiller1990]

Will keep you updated

@andrew-productiv if you had any learnings or progress or even a strategy on how you'd approach this, please share it! We are finding this quite challenging (many edge cases, integration with Sinon is tricky, etc...)

[lmiller1990]

Tentative PR for this: https://github.com/cypress-io/cypress/pull/26536

See the description for how to get an early release! cc @adamdehaven. It's a plugin, which means you'll need to install it. This will be released under the @cypress namespace soon, and once we've tested it more and ironed out all the bugs, eventually bundled in the binary and available out of the box.

Other browser-based runners that use ES modules could likely use a similar approach, too, eg in https://github.com/vitest-dev/vitest/issues/3046. The next milestones:

1. Test this
2. (probably) use AST based approach (just using regexp for now, to get early testing/feedback)
3. benchmark
4. make part of core offering in the Cypress binary

[adamdehaven]

@lmiller1990 this is awesome 🚀 I'll def get this worked into some test flows next week

[lmiller1990]

There are still some bugs we are working on fixing, please share a repro of any you run into. Latest would be @lmiller1990/vite-plugin-cypress-esm.

[lmiller1990]

This is working pretty well, a bunch of fairly complex React component tests are passing: https://github.com/muratkeremozcan/tour-of-heroes-react-vite-cypress-ts/pull/103

This (as a npm module) will be live soon under @cypress/vite-plugin-cypress-esm (name TBA). It's still got work, but this will let us get more feedback - the more feedback and testing we can get from real projects, the sooner we can make it part of the core offering.

[marktnoonan]

This is published now: https://www.npmjs.com/package/@cypress/vite-plugin-cypress-esm

[lmiller1990]

I found this is not working for some basic cases: https://github.com/lmiller1990/esm-bug

I will file an issue.

[dwilt]

@marktnoonan how would I go about using mocking ESM modules with a NextJS project?

[lmiller1990]

@dwilt you could try: https://www.npmjs.com/package/@cypress/vite-plugin-cypress-esm

Are you actually using ESM (eg "type": "module")? Or just the import/export syntax (depending on your config, that may be transpiled to CJS).

If you have a specific reproduction of something not working, happy to take a look.

[andyhqtran]

@dwilt you could try: npmjs.com/package/@cypress/vite-plugin-cypress-esm

Are you actually using ESM (eg "type": "module")? Or just the import/export syntax (depending on your config, that may be transpiled to CJS).

If you have a specific reproduction of something not working, happy to take a look.

@lmiller1990 Trying out the plugin returns this error:

We just use the import/export syntax. We don't have "type": "module" set in our package.json since we get this error:

core:test:cy: (node:24174) ExperimentalWarning: Custom ESM Loaders is an experimental feature and might change at any time
core:test:cy: (Use `node --trace-warnings ...` to show where the warning was created)
core:test:cy: (node:24174) ExperimentalWarning: The Node.js specifier resolution flag is experimental. It could change or be removed at any time.
core:test:cy: /Users/use/Library/Caches/Cypress/12.17.3/Cypress.app/Contents/Resources/app/packages/telemetry/dist/span-exporters/ipc-span-exporter.js:11
core:test:cy:         super({});
core:test:cy:         ^
core:test:cy: TypeError: Class constructor OTLPTraceExporter cannot be invoked without 'new'
core:test:cy:     at new OTLPTraceExporter (/Users/use/Library/Caches/Cypress/12.17.3/Cypress.app/Contents/Resources/app/packages/telemetry/dist/span-exporters/ipc-span-exporter.js:11:9)
core:test:cy:     at Object.<anonymous> (/Users/user/Library/Caches/Cypress/12.17.3/Cypress.app/Contents/Resources/app/packages/server/lib/plugins/child/require_async_child.js:9:18)
core:test:cy:     at Module._compile (node:internal/modules/cjs/loader:1254:14)
core:test:cy:     at Module.m._compile (/Users/use/Library/Caches/Cypress/12.17.3/Cypress.app/Contents/Resources/app/node_modules/ts-node/dist/index.js:857:29)
core:test:cy:     at Module._extensions..js (node:internal/modules/cjs/loader:1308:10)
core:test:cy:     at Object.require.extensions.<computed> [as .js] (/Users/use/Library/Caches/Cypress/12.17.3/Cypress.app/Contents/Resources/app/node_modules/ts-node/dist/index.js:859:16)
core:test:cy:     at Module.load (node:internal/modules/cjs/loader:1117:32)
core:test:cy:     at Function.Module._load (node:internal/modules/cjs/loader:958:12)
core:test:cy:     at ModuleWrap.<anonymous> (node:internal/modules/esm/translators:169:29)
core:test:cy:     at ModuleJob.run (node:internal/modules/esm/module_job:194:25)```