`Error: Cannot find module` when migrating to vitest

[gabrielcolson]

I tried to migrate a service to Vitest but I got this error :

Error: Cannot find module '/Users/gabrielcolson/Documents/<...>/node_modules/serverless-spy/lib/src/ServerlessSpy' imported from /Users/gabrielcolson/Documents/<...>/node_modules/serverless-spy/lib/index.mjs

I found a workaround thanks to Cursor but I think this is an issue on ServerlessSpy's side:

// vitest.config.ts
import { defineConfig } from "vitest/config"
import path from "path"

export default defineConfig({
  test: {
    typecheck: {
      tsconfig: "./tsconfig.dev.json",
    },
    setupFiles: [`${__dirname}/test/setup.ts`],
    include: ["test/e2e/**/*.e2e.ts"],
    poolOptions: {
      threads: {
        singleThread: true,
      },
    },
    testTimeout: 30000,
  },

  // added by cursor
  resolve: {
    alias: {
      "serverless-spy": path.resolve(__dirname, "node_modules/serverless-spy/lib/index.js"),
      "serverless-spy/lib/src/ServerlessSpy": path.resolve(
        __dirname,
        "node_modules/serverless-spy/lib/src/ServerlessSpy.js"
      ),
    },
  },
})

The same test works with Jest

[Lewenhaupt]

Hi @gabrielcolson,

Could you share your tsconfig and package.json? I'm using vi test myself with serverless-spy but I'd be happy to investigate if I'm able to reproduce this with your setup.

[Lewenhaupt]

@gabrielcolson I think the reason may be how you're importing it. In you vi test conf you set esbuild to use CommonJs, but it appears you're importing serverless-spy as an es module.

[gabrielcolson]

Hi @Lewenhaupt My tsconfig.json:

{
  "compilerOptions": {
    "alwaysStrict": true,
    "declaration": true,
    "esModuleInterop": true,
    "experimentalDecorators": true,
    "inlineSourceMap": true,
    "inlineSources": true,
    "lib": [
      "es2019"
    ],
    "module": "CommonJS",
    "noEmitOnError": false,
    "noFallthroughCasesInSwitch": true,
    "noImplicitAny": true,
    "noImplicitReturns": true,
    "noImplicitThis": true,
    "noUnusedLocals": true,
    "noUnusedParameters": true,
    "resolveJsonModule": true,
    "strict": true,
    "strictNullChecks": true,
    "strictPropertyInitialization": true,
    "stripInternal": true,
    "target": "ES2019"
  },
  "include": [
    "src/**/*.ts",
    "test/**/*.ts",
    ".projenrc.ts",
    "projenrc/**/*.ts"
  ],
  "exclude": [
    "node_modules"
  ]
}

package.json:

{
  "name": "<...>",
  "scripts": { },
  "devDependencies": {
    "serverless-spy": "^2.3.7",
    "typescript": "^4.9.5",
    "vitest": "^3.0.9"
  },
  "engines": {
    "node": ">= 18"
  },
  "license": "UNLICENSED",
  "publishConfig": {
    "access": "public"
  },
  "version": "0.0.0",
}

[gabrielcolson]

Also I edited my original comment, I did not need optimizeDeps to make it work

[Lewenhaupt]

@gabrielcolson is this a public project or private? Or could you share where you import serverless-spy?

[gabrielcolson]

The repo is private but here is the import:

import { ServerlessSpyListener, createServerlessSpyListener } from "serverless-spy"
import { ServerlessSpyEvents } from "../spy"

let serverlessSpyListener: ServerlessSpyListener<ServerlessSpyEvents>
beforeEach(async () => {
  serverlessSpyListener = await createServerlessSpyListener<ServerlessSpyEvents>({
    serverlessSpyWsUrl: ServerlessSpyWsUrl,
  })
})

afterEach(async () => {
  serverlessSpyListener.stop()
})

[Lewenhaupt]

@gabrielcolson you likely need to either change how you import it, or change your module setting in tsconfig. Does it work if you change module to nodenext?

[gabrielcolson]

it does not work with compilerOptions.module: nodenext

How am I supposed to import serverless-spy?

Also I have this error in VsCode but it does not prevent me from running the tests:

Could not find a declaration file for module 'serverless-spy'. '/Users/gabrielcolson/Documents/<...>/node_modules/serverless-spy/lib/index.mjs' implicitly has an 'any' type.
  There are types at '/Users/gabrielcolson/Documents/<...>/node_modules/serverless-spy/lib/index.d.ts', but this result could not be resolved when respecting package.json "exports". The 'serverless-spy' library may need to update its package.json or typings.

[Lewenhaupt]

@gabrielcolson Could you try setting "moduleResolution": "nodenext"? I suspect this might stem from the missing d.mts files, which I'll see if I can fix, but the only thing I can see is different between your project and mine is the missing moduleResolution, and that I'm using ES2022 and not ES2019, but that shouldn't be it.

[gabrielcolson]

Still the same issue with your config (I commented out the fix in vitest.config.e2e.ts):

{
  "compilerOptions": {
    "alwaysStrict": true,
    "declaration": true,
    "esModuleInterop": true,
    "experimentalDecorators": true,
    "inlineSourceMap": true,
    "inlineSources": true,
    "lib": [
      "es2019"
    ],
    "module": "CommonJS",
    "noEmitOnError": false,
    "noFallthroughCasesInSwitch": true,
    "noImplicitAny": true,
    "noImplicitReturns": true,
    "noImplicitThis": true,
    "noUnusedLocals": true,
    "noUnusedParameters": true,
    "resolveJsonModule": true,
    "strict": true,
    "strictNullChecks": true,
    "strictPropertyInitialization": true,
    "stripInternal": true,
    "target": "ES2022",
    "moduleResolution": "nodenext"
  },
  "include": [
    "src/**/*.ts",
    "test/**/*.ts",
    ".projenrc.ts",
    "projenrc/**/*.ts"
  ],
  "exclude": [
    "node_modules"
  ]
}

import { defineConfig } from "vitest/config"
// import path from "path"

export default defineConfig({
  test: {
    typecheck: {
      tsconfig: "./tsconfig.dev.json",
    },
    setupFiles: [`${__dirname}/test/setup.ts`],
    include: ["test/e2e/**/*.e2e.ts"],
    poolOptions: {
      threads: {
        singleThread: true,
      },
    },
    testTimeout: 30000,
  },
  // resolve: {
  //   alias: {
  //     "serverless-spy": path.resolve(__dirname, "node_modules/serverless-spy/lib/index.js"),
  //     "serverless-spy/lib/src/ServerlessSpy": path.resolve(
  //       __dirname,
  //       "node_modules/serverless-spy/lib/src/ServerlessSpy.js"
  //     ),
  //   },
  // },
})

[gabrielcolson]

has anyone found a workaround other than resolve.alias? serverless-spy is really the only package I am having a problem with...

[Lewenhaupt]

Hi @gabrielcolson i will soon have time to try a change to the exports part of the package.json that we produce.

[gabrielcolson]

Hi @Lewenhaupt ! Any news on this one? Can I help?

[Lewenhaupt]

Hi @gabrielcolson, It's not trivial to properly package this for .cjs. What do you have as type in your package.json? If you do not have it as module, could you try that?

[Lewenhaupt]

To properly fix this (if the issue is what I think it is) it will require some pretty big updates to how the project is built (mainly due to jsii). That will have to wait a bit.

[Lewenhaupt]

@gabrielcolson This probably won't fix everything, but this branch should fix your typing issue, you can pull it and run npm run build and then link your project to the lib folder. That will allow you to try it out.

[Lewenhaupt]

@gabrielcolson This answer might help you as well https://stackoverflow.com/a/72215487. I would strongly recommend not using "module": "commonjs". See https://www.typescriptlang.org/docs/handbook/modules/theory.html#the-module-output-format "commonjs, system, amd, and umd: Each emits everything in the module system named, and assumes everything can be successfully imported into that module system. These are no longer recommended for new projects and will not be covered in detail by this documentation."

[gabrielcolson]

Hi @Lewenhaupt thanks for looking into this. I initially wanted to try vitest because we had memory issues with jest but we recently managed to make jest work. Vitest support is not longer blocking us but I will try to find the time to go back to it soon. On the other hand, I tried to migrate one of our service to ESM as we are seeing more and more issues from CJS support being dropped by many of our dependencies. I tried your branch and I confirm that it indeed fixed our type issue.

[Lewenhaupt]

That's good to hear, then I'll merge it soon 👍 Also, if you could replicate the other issue with vitest in a public repo I'd be happy to have a look at it.

[gabrielcolson]

Following up on this after linking the branch (and migrating my service to use ESM), I have the following issues:

A warning when I synth my CDK app:

▲ [WARNING] The condition "types" here will never be used as it comes after both "import" and "require" [package.json]

    ../serverless-spy/package.json:137:4:
      137 │     "types": "./lib/index.d.ts"
          ╵     ~~~~~~~

  The "import" condition comes earlier and will be used for all "import" statements:

    ../serverless-spy/package.json:135:4:
      135 │     "import": "./lib/index.mjs",
          ╵     ~~~~~~~~

  The "require" condition comes earlier and will be used for all "require" calls:

    ../serverless-spy/package.json:136:4:
      136 │     "require": "./lib/index.js",

EventBridge and DynamoDB resources are deleted from my spy.json file (which is weird):

 /* eslint-disable */
 export class ServerlessSpyEvents {
-  EventBridgeEventBus: 'EventBridge#EventBus' = 'EventBridge#EventBus';
-  DynamoDBTable: 'DynamoDB#Table' = 'DynamoDB#Table';
   FunctionListenerRequest: 'Function#Listener#Request' = 'Function#Listener#Request';
   FunctionListenerError: 'Function#Listener#Error' = 'Function#Listener#Error';
   FunctionListenerConsole: 'Function#Listener#Console' = 'Function#Listener#Console';
   FunctionListenerResponse: 'Function#Listener#Response' = 'Function#Listener#Response';
-  EventBridgeRuleEventBusRule: 'EventBridgeRule#EventBus#Rule' = 'EventBridgeRule#EventBus#Rule';
   FunctionApiHandlerRequest: 'Function#ApiHandler#Request' = 'Function#ApiHandler#Request';
   FunctionApiHandlerError: 'Function#ApiHandler#Error' = 'Function#ApiHandler#Error';
   FunctionApiHandlerConsole: 'Function#ApiHandler#Console' = 'Function#ApiHandler#Console';

[Lewenhaupt]

@gabrielcolson Thanks for the update! Those warnings might require me to bump jsii, I'll see if I can make it work :)

Is that from you project or from the serverless-spy project that you cloned? It shouldn't remove listeners unless you change the parameters to serverlessSpy.spy() or remove the resources. From the above it seems the project no longer has an EventBridgeRule, EventBus, or DynamoDBTable in the stack?

[Lewenhaupt]

@gabrielcolson Try either commit 1a788f734ac83a5c74cdad70c3f3bcebfbb92b5d (changes order) or 05726d73d019a872de5f73d9a80b56d6168417fa (I think this is a better approach honestly) from the same branch. Both of those work in my project but I think the second one is best.

[Lewenhaupt]

Ok I just faced this in a new project at work and my last fix in the branch did solve it.

[Lewenhaupt]

Ok now I seem to have encountered the issue again. It seems to be something with vitest causing it to be much more strict about the imports. But it does work if I npm link to my local installation of the latest tag which I think is really strange.

[ServerlessLife]

@Lewenhaupt Thank you for solving this 🙏 🙏 🙏 🙏

[Lewenhaupt]

@gabrielcolson If you're able to I'd love to see if 2.3.12 works for you. I've switched from simple tsc for building to using tsdown for building cjs and esm and now my own project using vitest works again, would love to know if it fixes it for you too :)