Can't get threads.js working with TypeScript/React/Webpack 5

[achimcc]

Hello,

I've tried quite a few things but I just can't get even a minimal example of threads.js working with React/Typescript/Webpack5. I followed the documentation, added the threads-plugin to webpack. I uploaded the repo on my GitHub: https://github.com/achimcc/threads_example

Running yarn start, I receive a warning:

WARNING in No instantiations of threads.js workers found. Please check that:

1. You have configured Babel / TypeScript to not transpile ES modules
2. You import Worker from threads where you use it

For more details see: https://github.com/andywer/threads-plugin

While the Chrome console is reporting some errors:

5e513c3f-5e23-4eff-ae53-e43b68f8daf7:1 Refused to execute script from 'http://localhost:8080/4cbcab03c903775de194.ts' because its MIME type ('video/mp2t') is not executable. (anonymous) @ 5e513c3f-5e23-4eff-ae53-e43b68f8daf7:1 5e513c3f-5e23-4eff-ae53-e43b68f8daf7:1 Uncaught DOMException: Failed to execute 'importScripts' on 'WorkerGlobalScope': The script at 'http://localhost:8080/4cbcab03c903775de194.ts' failed to load. at blob:http://localhost:8080/5e513c3f-5e23-4eff-ae53-e43b68f8daf7:1:1

I would highly appreciate any advice! Or is there perhaps a sample repo for a minimal TypeScript/Webpack5 setup?

Many thanks in advance, Achim

[achimcc]

I added the following import in App.tsx:

// @ts-ignore import workerURL from "threads-plugin/dist/loader?name=gitWorker!./workers/add.ts";

And I spawn the worker now with:

const add = await spawn(new Worker(workerURL));

This seems to work! However, the warning about non-existing instantiations of Workers remains when running yarn...

[andywer]

Hey Achim!

I'm sorry, but webpack 5 support is still work in progress. We really need to add a message to the readme…

Webpack 5 might actually make it easier to work with threads.js, but there were some issues and it turned out that I couldn't fix them as easily as I originally thought. And since I was very busy the last couple of months and no one from the community seems to have figured it out by now, this is still not solved 😕

So if you can, you might want to downgrade to webpack 4 until this is resolved.

[xuchaoqian]

It works fine after removing threads-plugin and using the new expression for Webpack 5:

import { spawn, Thread, ModuleThread } from "threads";
const worker = await spawn<Hello>(new Worker(new URL("./hello.worker.ts", import.meta.url)));

If we want to run tests in Jest, we need do some extra work:

1. Write our own transfromer, like this:

const babelJest = require("babel-jest").default;
const { createHash } = require("crypto");

module.exports = {
  getCacheKey(fileData, filename, ...rest) {
    const babelCacheKey = babelJest.getCacheKey(fileData, filename, ...rest);
    return createHash("md5").update(babelCacheKey).update("1").digest("hex");
  },

  process(src, filename, ...rest) {
    if (filename.endsWith("src/Main.ts")) {
      src = src.replace(
        /import\s+\{([^}]+)\}\s+from\s+"threads";/,
        'import { Worker, $1} from "threads"'
      );
      src = src.replace(
        /new\s+Worker\(new\s+URL\(".\/hello.worker.ts",\s+import.meta.url\)\)/,
        'new Worker("./hello.worker.ts")'
      );
    }
    return babelJest.process(src, filename, ...rest);
  },
};

2. Put the following settings to tsconfig.json:

  "ts-node": {
    "compilerOptions": {
      "module": "CommonJS"
    }
  }

Update: This is a demo repo: threads-webpack5-demo. Try it as following:

1. npm install --verbose
2. npm run test // run it in node using jest
3. npm run start // run it in browser, check the browser devtool to see the result

[yasinkocak]

any updates of this issue?

[rogue-ninja-creative]

I notice the threads-plugin v. 1.4 says it brings support for Webpack 5, but I'm still having a heck of a time spawning workers. Is this still an unresolved issue with this library, or I should be focusing on possible errors in my config?

[andywer]

I'm sorry, I have been flooded with so many other things… webpack 5 is still not finally resolved, I'm afraid.

@mclassdesigns Did you try what's been previously outlined here in this issue? Might be worth trying to do it without the plugin: https://github.com/andywer/threads.js/issues/387#issuecomment-942866447

[rogue-ninja-creative]

I'm sorry, I have been flooded with so many other things… webpack 5 is still not finally resolved, I'm afraid.

@mclassdesigns Did you try what's been previously outlined here in this issue? Might be worth trying to do it without the plugin: https://github.com/andywer/threads.js/issues/387#issuecomment-942866447

No worries!

I think I have something that's working. I'll post my solution, hopefully this weekend!

Thanks for the reply!

[rogue-ninja-creative]

By the way, I did try most of the solutions I found here and on other sites but was still getting errors. I'll backtrack a bit to replicate those issues since they may be helpful to you when you get some time to work on the project.

I'll post those along with my current solution.

Thanks for your time!

[xuchaoqian]

By the way, I did try most of the solutions I found here and on other sites but was still getting errors. I'll backtrack a bit to replicate those issues since they may be helpful to you when you get some time to work on the project.

I'll post those along with my current solution.

Thanks for your time!

I have updated #387 (comments) , hope it was useful.

[rogue-ninja-creative]

By the way, I did try most of the solutions I found here and on other sites but was still getting errors. I'll backtrack a bit to replicate those issues since they may be helpful to you when you get some time to work on the project. I'll post those along with my current solution. Thanks for your time!

I have updated #387 (comments) , hope it was useful.

I did try this approach but using a pool:

const workerPool = Pool(() => spawn(new Worker(new URL('@workers/dashboardEventsWorker.js', import.meta.url))), 8)

But somehow it's causing an error with Axios:

HookWebpackError: No code generation entry for runtime manifest for /path_to_modules/node_modules/axios/lib/adapters/xhr.js

Working on this today.

[lancety]

I experienced same issue typescript+webpack5.
Instead of letting webpack to handle the bundle build, as a workaround, you can set a webpack.config.worker.js for your worker entry file which is the file path you 'new worker', and build it separately, and when 'new the worker' use dist folder's worker bundle path.

this._worker = await spawn<WorkerModule<any>>(new Worker("./dist/scriptWorker/physicsBundle.worker.js")); With this approach, main webpack config wont need threads-plugin, which make webpack avoid checking the above worker dependency, which is what we want.

Then by using a separate worker webpack build task

• typescript works
• no code change other than the worker file path
• you dont need the threads-plugin in main webpack.config.js
• it avoid having the memory leaking issue casuing the npm build task crash after several round of continue watch build... (which might just happen to some user of threadjs)
• build time for both main project and the worker are few seconds in my project, much shorter compare to 30-50+ seconds when letting webpack to bundle everything from the main project entry in webpack 4

[Leooooyx]

any update on this or any temp solution for js?

[digable1]

Echoing @Leooooyx's comment about information about an update.

I'm a rank newbie in WebWorker that wanted to remain in TypeScript, hence my finding this package. And I stumbled a lot, including when I first found this issue discussion and didn't realize the importance of @xuchaoqian's suggestion - which turned out to be the correct one for my usage.

But the above suggestion is buried in this discussion with no other updated documentation referencing the problem with threads-plugin and Webpack 5. Very hard for people looking to resolve the problem of using web workers without giving up TypeScript.

Until this is with threads-plugin is resolved, can this please be called out prominently in the documentation - so the people after me don't get their anxiety-level raised like mine was for a while.

Of course: Another request to please resolve this - with examples if configuration is needed (presumably including how to align Mime types if necessary so this just works with the webpack plugin - don't have enough context to talk intelligently about this).

And while we're at the topic of documentation: An example on how to pre-transpile the webworker would be greatly appreciated. Basically I'm lazy and want to be a script kiddie. I should be able to implement this concept but haven't yet as I'm including ts-lint the main part of package.json dependencies section instead of the devDependencies section. But I think I intend to move it back to devDependencies:

1. The ts-lint footprint is a bit larger than what I want to include in a public-facing bundle (it's great for initial interaction and development such as with 'webpack serve').
2. I don't want to do the transpiling at every worker invocation (shouldn't need to - it's the same transpiled code each time and therefore wastes cycles).

I'm suggesting this optimization pattern is very important and should be common - and therefore addressed in the documentation.

This package solves an intrinsic issue for people wanting strong types in their languages for the front end development. For those newly delving into this, a little more documentation and support help (plugin bug fix) would be appreciated.