threads.js icon indicating copy to clipboard operation
threads.js copied to clipboard

Importing function from package in same monorepo into worker file causes Webpack to run out of memory

Open Nantris opened this issue 4 years ago • 2 comments

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory

This doesn't happen with every single function from every single package, but it seems to be just about every.

We're importing from these packages in a non-circular manner, only importing the function we're using, and the package we're importing from has sideEffects: false set.

If we duplicate the function and put it in a file in the same directory as our worker.js file, no problems.

I'm guessing it's related to tree-shaking? Anyone have and ideas about a solution or a route of investigation for us to pursue?

Big thanks to @andywer for sharing this amazing project!

Nantris avatar Nov 29 '20 23:11 Nantris

Thanks, @Slapbox!

Yeah, sounds like webpack ends up in some kind of infinite loop while compiling and then runs out of memory.

There is to little information, though, to tell what the issue is.

Maybe (semi-educated guess) the issue is rooted in webpack's trying to deduplicate the modules of your local package that need to be bundled twice, once for the worker, once for the main thread.

Or it's really the tree-shaking… Very tricky topic. Btw, what webpack version do you use?

Tbh, I'm not sure if I can help here.

andywer avatar Nov 30 '20 04:11 andywer

Hey @andywer thanks for your reply!

We're using [email protected]. Yeah I know there's nothing really to go on here. I just figured I'd report it and get the ball rolling, because surely someone else will hit this sooner or later.

I've found that the following are not of help:

  • Setting mode: 'none'
  • Setting optimization: {}
  • Setting devtool: false
  • Disabling optional plugins (eg HardSourceWebpackPlugin, HotModuleReplacementPlugin)

So I'm guessing that it really is a tree-shaking problem somehow, but this is not really my wheelhouse ¯\_(ツ)_/¯

Nantris avatar Nov 30 '20 20:11 Nantris