Cannot import built-in node modules in worker in electron/webpack build

[jc-lab]

https://github.com/jc-lab/threads-plugin-problem See the repository above.

Error when using module as esnext in tsconfig. If you use commonjs instead of esnext, threads-plugin will not detect Worker.

> npm run build
...
Child WorkerPluginLoader:
     2 assets
    Entrypoint 0 = 0.bundle.worker.js 0.bundle.worker.js.map
     [6] (webpack)/buildin/harmony-module.js 573 bytes {0} [built]
    [15] ./node_modules/ts-loader??ref--4!./src/test.thread.ts + 2 modules 16.7 KiB {0} [built]
         | ./node_modules/ts-loader??ref--4!./src/test.thread.ts 730 bytes [built]
         |     + 2 hidden modules
        + 26 hidden modules

    ERROR in ./src/test.thread.ts (./node_modules/ts-loader??ref--4!./src/test.thread.ts)
    Module not found: Error: Can't resolve 'fs' in 'D:\temp\webpack-worker-test\src'
     @ ./src/test.thread.ts (./node_modules/ts-loader??ref--4!./src/test.thread.ts) 3:0-25 8:22-42

    ERROR in ./src/test.thread.ts (./node_modules/ts-loader??ref--4!./src/test.thread.ts)
    Module not found: Error: Can't resolve 'zlib' in 'D:\temp\webpack-worker-test\src'
     @ ./src/test.thread.ts (./node_modules/ts-loader??ref--4!./src/test.thread.ts) 4:0-29 9:23-41

[andywer]

It works with TypeScript, you just need to set module to esnext as you already wrote. It's also documented like that: See here.

The error above is unrelated to threads.js and the threads-plugin as far as I can tell.

Try npm install --save-dev @types/node.

[andywer]

Feel free to re-open if I missed something. The error above looks very much like something else is fishy, though.

[jc-lab]

@andywer If you comment out ThreadsPlugin as shown below, no error will occur. That's why I think it's an issue of threads-plugin.

// plugins: [new ThreadPlugin()],

output:

> npm run build
> webpack --config webpack.config.js -p

Hash: 1fb9dddabef6f1be9dc0
Version: webpack 4.41.5
Time: 1404ms
Built at: 2020-01-06 23:04:43
                          Asset       Size  Chunks                   Chunk Names
          ..\dist-ts\index.d.ts   47 bytes          [emitted]
      ..\dist-ts\index.d.ts.map  104 bytes          [emitted]
    ..\dist-ts\test.thread.d.ts   53 bytes          [emitted]
..\dist-ts\test.thread.d.ts.map  116 bytes          [emitted]
                      bundle.js   7.37 KiB       0  [emitted]        main
                  bundle.js.map   4.43 KiB       0  [emitted] [dev]  main
Entrypoint main = bundle.js bundle.js.map
[0] external "threads" 42 bytes {0} [built]
[1] ./src/index.ts 3.26 KiB {0} [built]

[andywer]

Well, if you out-comment the threads-plugin, then webpack won't even know that it should process the worker script and its dependencies, so it won't crash as the worker's code isn't touched by webpack at all 😉

Have you installed the @types/node package?

[jc-lab]

https://github.com/jc-lab/threads-plugin-problem/blob/14b7cdaf46335667902538ecd20009ce2e6812b3/package.json#L9

As shown above, @types/node is installed. The same code in test.worker.ts works fine with index.ts.

[andywer]

Wow, that seems like a weird issue and requires more time to investigate then.

[jc-lab]

@andywer Please review. Thank you.

[andywer]

Can you please check if it's fixed in the latest version? #16 might have fixed it.

[raz-sinay]

I have encountered this issue. solved it by doing:

// webpack.main.js
const NodeTargetPlugin = require('webpack/lib/node/NodeTargetPlugin');
const ThreadsPlugin = require('threads-plugin');
module.exports = function(config) {
  // NodeTargetPlugin must be used in order to use nodejs built-in API in workers.
  config.plugins.unshift(
    new ThreadsPlugin({
      plugins: [new NodeTargetPlugin()]
    })
  );
  return config;
};

this works for both main thread and renderer threads

[andywer]

@raz-sinay Cool, thanks!

@jc-lab Can you try and confirm that it works? If so, let's add a target option to the threads-plugin that defaults to the webpack config's main target option and adds the NodeTargetPlugin if the new setting is node.

[pverscha]

@andywer @jc-lab I've just discovered that you need to enable nodeIntegrationInWorker when creating a new BrowserWindow for fs to work. See https://www.electronjs.org/docs/tutorial/multithreading for more information.

[andywer]

@pverscha Sure, but you shouldn't… 😅 One little XSS vulnerability in your webapp code and the attacker can take over your whole system at once ;)

I wouldn't use fs in a BrowserWindow context at all, but multithreading should still be possible out of the box - in the web context via web workers and outside of it using worker threads.

(That little security insight is brought to you by someone building cryptocurrency wallets with Electron for a living, among other things ;) )

[pverscha]

@andywer Thanks! Didn't think of that.

[arition]

@raz-sinay Cool, thanks!

@jc-lab Can you try and confirm that it works? If so, let's add a target option to the threads-plugin that defaults to the webpack config's main target option and adds the NodeTargetPlugin if the new setting is node.

I met the same problem and the NodeTargetPlugin trick works for me. Maybe you can add this to the documentation?

[andywer]

@arition Hmm, are you using the latest version of the threads-plugin? As you can see in the loader code, the behavior that I outlined before has already been implemented.

So if you set the webpack target option to something containing node, it will automatically add the NodeTargetPlugin to the worker compiler.

[arition]

@andywer I think the problem still exists because the target is electron-main in my case. The target name does not contain node.