New `--import` API breaks worker threads

[zner0L]

Problem

I was tracking https://github.com/esbuild-kit/tsx/issues/330 closely and was very happy for you to fix it with https://github.com/esbuild-kit/tsx/pull/337 and release so quickly, but I just now tested it with loading worker threads and that fails with ERR_UNKNOWN_FILE_EXTENSION.

Minimal example:

import { isMainThread, Worker } from 'worker_threads';

if (isMainThread) {
    const worker = new Worker(new URL(import.meta.url));
} else console.log('Hello from worker thread!');

Result:

yarn tsx test.ts
yarn run v1.22.19
$ /<redacted>/node_modules/.bin/tsx test.ts

node:internal/event_target:1083
  process.nextTick(() => { throw err; });
                           ^
TypeError [ERR_UNKNOWN_FILE_EXTENSION]: Unknown file extension ".ts" for /<redacted>/test.ts
    at __node_internal_captureLargerStackTrace (node:internal/errors:497:5)
    at new NodeError (node:internal/errors:406:5)
    at Object.getFileProtocolModuleFormat [as file:] (node:internal/modules/esm/get_format:99:9)
    at defaultGetFormat (node:internal/modules/esm/get_format:142:36)
    at defaultLoad (node:internal/modules/esm/load:120:20)
    at ModuleLoader.load (node:internal/modules/esm/loader:396:13)
    at ModuleLoader.moduleProvider (node:internal/modules/esm/loader:278:56)
    at new ModuleJob (node:internal/modules/esm/module_job:65:26)
    at #createModuleJob (node:internal/modules/esm/loader:290:17)
    at ModuleLoader.getJobFromResolveResult (node:internal/modules/esm/loader:248:34)
    at ModuleLoader.getModuleJob (node:internal/modules/esm/loader:229:17)
    at async ModuleLoader.import (node:internal/modules/esm/loader:315:23)
    at async loadESM (node:internal/process/esm_loader:34:7)
    at async handleMainPromise (node:internal/modules/run_main:66:12)
Emitted 'error' event on Worker instance at:
    at [kOnErrorMessage] (node:internal/worker:326:10)
    at [kOnMessage] (node:internal/worker:337:37)
    at MessagePort.<anonymous> (node:internal/worker:232:57)
    at [nodejs.internal.kHybridDispatch] (node:internal/event_target:807:20)
    at exports.emitMessage (node:internal/per_context/messageport:23:28) {
  code: 'ERR_UNKNOWN_FILE_EXTENSION'
}

Node.js v20.8.0

Expected behavior

If I run the minimal example with tsx v3.13.0 the output is as expected (though with warnings):

❯ yarn tsx test.ts
yarn run v1.22.19
$ /<redacted>/node_modules/.bin/tsx test.ts
(node:35991) ExperimentalWarning: `--experimental-loader` may be removed in the future; instead use `register()`:
--import 'data:text/javascript,import { register } from "node:module"; import { pathToFileURL } from "node:url"; register("file%3A///<redacted>/node_modules/tsx/dist/loader.mjs", pathToFileURL("./"));'
(Use `node --trace-warnings ...` to show where the warning was created)
(node:35991) ExperimentalWarning: `--experimental-loader` may be removed in the future; instead use `register()`:
--import 'data:text/javascript,import { register } from "node:module"; import { pathToFileURL } from "node:url"; register("file%3A///<redacted>/node_modules/tsx/dist/loader.mjs", pathToFileURL("./"));'
(Use `node --trace-warnings ...` to show where the warning was created)
Hello from worker thread!
Done in 0.25s.

Minimal reproduction URL

Included in issue

Version

v3.14.0

Node.js version

v20.8.0

Package manager

yarn

Operating system

Linux

Contributions

• [ ] I plan to open a pull request for this issue
• [ ] I plan to make a financial contribution to this project

[ghost]

I tried passing options to the Worker constructor such as env: { NODE_OPTIONS: "--import tsx" } and execArgv: ["--import", "tsx"] but neither of those worked and then I found nodejs/node#47747 (ESM loaders cannot be defined via Worker option...)

I think this is really a Node bug or design issue. If a loader was specified at the top level, Node should be using that automatically for Worker threads.

[ghost]

Other notes

• Node has a plan to support identifying external modules, e.g. typescript see nodejs/node#49704
• There is a discussion about missing use cases for the module.register() API at nodejs/node#50392
• I found the above 2 notes from a comment in a proposal to use the ESM loader everywhere, see https://github.com/nodejs/node/issues/50356#issuecomment-1779592133

[vafanassieff]

I'm facing the same issue as you do, and I had to stick with tsx v3.13.0

[alanhoff]

I've found a temporary workaround: while developing, you'll want your worker to load tsx/cli instead, then pass the actual script you wanna run as an argv, like so:

	const tsx = new URL(import.meta.resolve('tsx/cli'));
	const worker = new Worker(tsx, {env: SHARE_ENV, argv: ['./your/script.ts']});

[privatenumber]

Looked into this. Sounds like this is a limitation on Node's end? Marking this as blocked

[SynthLuvr]

This is an issue for node v18.19.0 too

[aaronlin0122]

I got an error with this Error: listen EADDRINUSE: address already in use.

I've found a temporary workaround: while developing, you'll want your worker to load tsx/cli instead, then pass the actual script you wanna run as an argv, like so:

	const tsx = new URL(import.meta.resolve('tsx/cli'));
	const worker = new Worker(tsx, {env: SHARE_ENV, argv: ['./your/script.ts']});

[privatenumber]

🛑 Want this fixed?

This is currently a limitation in Node.js. The issue was closed without resolution: https://github.com/nodejs/node/issues/47747

They currently don't think the performance cost is worth it, but are open to learning about more use-cases.

I have requested for it to be re-opened. If you'd like loaders to be supported in Workers,

your best course of action is to share your use-case there.

Again, don't just say "plz fix". Share your use-case in https://github.com/nodejs/node/issues/47747.

[vafanassieff]

Our use case is running tsx --watch as dev server for a typescript project who use a Worker for specific tasks and run tests with vitest

For now the only fix we found was to stick with node 20.9.0 and tsx 3.13.0 We have to start our tests script with NODE_OPTIONS='--loader tsx --no-warnings' vitest --run

[SynthLuvr]

Our use case is using tsx with ava. Similarly, we have to downgrade for now.

[JounQin]

I'm not using any worker threads, but just:

import fs from 'node:fs'

import LinguistLanguages from 'linguist-languages'
import { SupportLanguage } from 'prettier'

https://github.com/un-ts/prettier/actions/runs/7152253084/job/19477387552?pr=327#step:5:96

See source codes at https://github.com/un-ts/prettier/blob/master/scripts/languages.ts

[privatenumber]

This is blocked by Node.js. Please contribute constructively in https://github.com/nodejs/node/issues/47747

Looks like it's been re-opened with a potential solution that could use help.

Locking thread until resolved in Node.