Error on dynamic imports using `AsyncConstructor` in Vitest

[leonsilicon]

Describe the bug

When using dynamic imports with AsyncConstructor:

export async function arrayUniqEval() {
	const fn = async function () {}.constructor(
		"const { default: arrayUniq } = await import('array-uniq'); console.log(arrayUniq([1, 2, 2]))"
	);
	await fn();
}
await arrayUniqEval();

It works fine in Node:

But, when run with Vitest, Vitest throws the following error:

 ❯ eval.test.js (1)
   × test

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯ Failed Tests 1 ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯

 FAIL  eval.test.js > test
TypeError: A dynamic import callback was not specified.
 ❯ eval (eval at arrayUniqEval uniq.js:2:34), <anonymous>:3:32
 ❯ Module.arrayUniqEval uniq.js:5:8
      3|                "const { default: arrayUniq } = await import('array-uniq'); console.log(arrayUniq([1, 2, 2]))"
      4|        );
      5|        await fn();
       |        ^
      6| }
      7|
 ❯ eval.test.js:5:7

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[1/1]⎯

Test Files  1 failed (1)
     Tests  1 failed (1)
      Time  2.41s (in thread 4ms, 68841.00%)

Might be related to this error: https://github.com/nodejs/node/issues/30591

Reproduction

https://github.com/leonzalion/vitest-dynamic-import-eval

System Info

System:
    OS: macOS 11.6.4
    CPU: (16) x64 Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz
    Memory: 41.03 GB / 64.00 GB
    Shell: 5.8 - /bin/zsh
  Binaries:
    Node: 16.14.0 - ~/Library/pnpm/node
    Yarn: 1.22.17 - /usr/local/bin/yarn
    npm: 8.3.1 - ~/Library/pnpm/npm
  Browsers:
    Brave Browser: 99.1.36.111
    Chrome: 99.0.4844.51
    Firefox: 98.0
    Safari: 15.3
  npmPackages:
    vitest: ^0.6.3 => 0.6.3

Used Package Manager

pnpm

Validations

• [X] Follow our Code of Conduct
• [X] Read the Contributing Guidelines.
• [X] Read the docs.
• [X] Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
• [X] Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
• [X] The provided reproduction is a minimal reproducible example of the bug.

[sheremet-va]

Native imports are not possible, until Node releases full ESM support for vm module.

Should we just statically replace it? @antfu

[antfu]

Sadly I think it might be the limitations currently. I would like to avoid adding more magic to transform it until we got better support from Node.

[JounQin]

I wasted a few hours to figure out it's vitest's issue. Move to jest then.

[lvqq]

Any updates or solutions about this issue? Face the same error while using dynamic import in vitest.

[igrep]

I faced a similar problem, which prevents me from developing my programming language whose current compile target is JavaScript. So I'm curious what magic vitest does with import, which could give me a hint for a workaround.

[sheremet-va]

I faced a similar problem, which prevents me from developing my programming language whose current compile target is JavaScript. So I'm curious what magic vitest does with import, which could give me a hint for a workaround.

Vite transforms all import statements to __vite_ssr_import__, so we intercept it and run specified file with vm.runInThisContext. When it's not intercepted, vm fails, because we don't specify import callback (requires --experimental-vm-modules flag, which we intentionally avoid)

[sheremet-va]

This feature should work correctly with --pool=vmThreads or --pool=vmForks.

[talatkuyuk]

Hi @sheremet-va,

I tried both --pool=vmThreads and --pool=vmForks but it still throws error: TypeError: A dynamic import callback was not specified.

TypeError: A dynamic import callback was not specified.
    at new NodeError (node:internal/errors:405:5)
    at hydrateFn (node:internal/modules/esm/utils:116:9)
    at eval (eval at runAsync (/Users/talatkuyuk/MyCodeRepo/my_packages/next-mdx-remote-client/src/lib/run.ts:68:29), <anonymous>:7:29)
    at Module.runAsync (/Users/talatkuyuk/MyCodeRepo/my_packages/next-mdx-remote-client/src/lib/run.ts:69:46)
    at Module.evaluate (/Users/talatkuyuk/MyCodeRepo/my_packages/next-mdx-remote-client/src/rsc/evaluate.tsx:54:15)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at /Users/talatkuyuk/MyCodeRepo/my_packages/next-mdx-remote-client/tests/test.evaluate.esm.spec.tsx:77:32
    at runTest (file:///Users/talatkuyuk/MyCodeRepo/my_packages/next-mdx-remote-client/node_modules/@vitest/runner/dist/index.js:719:11)
    at runSuite (file:///Users/talatkuyuk/MyCodeRepo/my_packages/next-mdx-remote-client/node_modules/@vitest/runner/dist/index.js:847:15)
    at runSuite (file:///Users/talatkuyuk/MyCodeRepo/my_packages/next-mdx-remote-client/node_modules/@vitest/runner/dist/index.js:847:15) {
  code: 'ERR_VM_DYNAMIC_IMPORT_CALLBACK_MISSING'
}

the mentioned runAsync (/Users/talatkuyuk/MyCodeRepo/my_packages/next-mdx-remote-client/src/lib/run.ts

export async function runAsync(
  compiledSource: string,
  options: RunOptions,
): Promise<RunResult> {
  const { keys, values } = prepareConstruction(options);

  const AsyncFunction = async function () {}.constructor;

  // await new Promise((resolve) => setTimeout(resolve, 500));

  // constructs the compiled source utilizing Reflect API with "async function constructor"
  const hydrateFn = Reflect.construct(AsyncFunction, keys.concat(compiledSource));
  const { default: Content, ...mod } = await hydrateFn(...values);

  return { Content, mod };
}

My nextjs application works perfect, but couldn't pass the test because of that error. I was using jest and want to migrate to the vitest.

I converted the test using jest instead of vitest again, and the jest works with NODE_OPTIONS=--experimental-vm-modules jest, and finds the dynamically imported module / modules during sync / async function construction.

my vitest version is latest 1.3.0

SORRY, SORRY, SORRY After re-loading the project, --pool=vmThreads worked for me.

But, --pool=vmForks is not stable, because it throws the error below, or refreshing the test it is workin, again if I started the test again it throws again:

Vitest caught 1 unhandled error during the test run.
This might cause false positive tests. Resolve unhandled errors to make sure your tests are not affected.

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯ Unhandled Error ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
Error: Worker exited unexpectedly
 ❯ ChildProcess.<anonymous> node_modules/tinypool/dist/esm/index.js:184:34
 ❯ ChildProcess.emit node:events:529:35
 ❯ ChildProcess._handle.onexit node:internal/child_process:292:12

My test script is:

test.only("works with imported modules", async () => {
    const source = dedent`
      import {Pill} from "./context/components.js"
      
      Hi {name}

      <Pill>!</Pill>
    `;
    const { content } = await evaluate({
      source,
      options: {
        mdxOptions: {
          baseUrl: import.meta.url, // should be provided, finds the 'components.js' in relative path
        },
        scope: {
          name: "foo",
        },
      },
    });

    expect(ReactDOMServer.renderToStaticMarkup(content)).toMatchInlineSnapshot(`
      "<p>Hi foo</p>
      <span style="color:blue">!</span>"
    `);
  });