JavaScript heap out of memory

[swernerx]

I am using this in a medium large React project as part of the Webpack flow. For compression we are using the Babili-Webpack-Plugin. As this is just a small wrapper around Babili I figure the following issue is more related to Babili itself:

<--- Last few GCs --->

  344958 ms: Mark-sweep 1324.0 (1434.9) -> 1324.0 (1434.9) MB, 1946.1 / 0.0 ms [allocation failure] [GC in old space requested].
  346872 ms: Mark-sweep 1324.0 (1434.9) -> 1324.0 (1434.9) MB, 1913.9 / 0.0 ms [allocation failure] [GC in old space requested].
  348780 ms: Mark-sweep 1324.0 (1434.9) -> 1327.8 (1405.9) MB, 1907.5 / 0.0 ms [last resort gc].
  350727 ms: Mark-sweep 1327.8 (1405.9) -> 1331.7 (1405.9) MB, 1946.7 / 0.0 ms [last resort gc].


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x381e5ffcfb51 <JS Object>
    1: visitQueue [/workspace/1_ui/ui-foundation/node_modules/babel-traverse/lib/context.js:~114] [pc=0x35f210c516c7] (this=0x2bf0d5c576b9 <a TraversalContext with map 0x22808a4e239>,queue=0x2bf0d5c57701 <JS Array[1]>)
    2: node [/workspace/1_ui/ui-foundation/node_modules/babel-traverse/lib/index.js:~94] [pc=0x35f210cf29bc] (...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node::Abort() [/usr/local/bin/node]
 2: node::FatalException(v8::Isolate*, v8::Local<v8::Value>, v8::Local<v8::Message>) [/usr/local/bin/node]
 3: v8::Utils::ReportApiFailure(char const*, char const*) [/usr/local/bin/node]
 4: v8::Utils::ApiCheck(bool, char const*, char const*) [/usr/local/bin/node]
 5: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [/usr/local/bin/node]
 6: v8::internal::Factory::NewByteArray(int, v8::internal::PretenureFlag) [/usr/local/bin/node]
 7: v8::internal::Factory::NewCode(v8::internal::CodeDesc const&, unsigned int, v8::internal::Handle<v8::internal::Object>, bool, bool, int, bool) [/usr/local/bin/node]
 8: v8::internal::PropertyAccessCompiler::GetCodeWithFlags(unsigned int, char const*) [/usr/local/bin/node]
 9: v8::internal::PropertyHandlerCompiler::GetCode(v8::internal::Code::Kind, v8::internal::Code::StubType, v8::internal::Handle<v8::internal::Name>) [/usr/local/bin/node]
10: v8::internal::NamedLoadHandlerCompiler::CompileLoadNonexistent(v8::internal::Handle<v8::internal::Name>) [/usr/local/bin/node]
11: v8::internal::NamedLoadHandlerCompiler::ComputeLoadNonexistent(v8::internal::Handle<v8::internal::Name>, v8::internal::Handle<v8::internal::Map>) [/usr/local/bin/node]
12: v8::internal::LoadIC::UpdateCaches(v8::internal::LookupIterator*) [/usr/local/bin/node]
13: v8::internal::LoadIC::Load(v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Name>) [/usr/local/bin/node]
14: v8::internal::Runtime_LoadIC_Miss(int, v8::internal::Object**, v8::internal::Isolate*) [/usr/local/bin/node]
15: 0x35f20b8092a7
Abort trap: 6

These are the relevant software versions:

• Mac OS Sierra
• Node 6.9.2
• Babili Preset 0.0.9
• Babel Core 6.18.2
• Babel Traverse 6.19.0

Is there anything I can do to debug this anymore?

[swernerx]

I just played a little with the custom configuration option for using a alternative preset instead of of the default one.

When disabling all processing I get a few of these messages:

• [BABEL] Note: The code generator has deoptimised the styling of "unknown" as it exceeds the max of "100KB".

I figure generally we might have a problem at working at a chunk level for babili as the chunks are generally quite massive.

[swernerx]

It seems to work with this setup:

new BabiliPlugin({
  comments: false,
  babili: {
    plugins: [
      "minify-dead-code-elimination",
      "minify-mangle-names",
      "minify-simplify"
    ]
  }
})

[boopathi]

What's the file size which when crossed gives this error - heap out of memory ?

Or is it happening only for your project for big file sizes ?

[swernerx]

I am not actually sure which chunk is producing this issue.

[swernerx]

With the limited set of plugins it's still using a ton of memory: 1.5GB highest - but it works (it does not do the minification thing for all files it seems - maybe that's related to the 100KB warning).

[swernerx]

BTW: Using a GB+ range of memory destroys the relevance for using this setup on any kind of CI server. These have typically somewhat limited memory/cpu offerings.

[swernerx]

Okay, when placing a "minified" inside the config this message is gone:

[BABEL] Note: The code generator has deoptimised the styling of "unknown" as it exceeds the max of "100KB".

Current config:

new BabiliPlugin({
  comments: false,
  babili: {
    minified: true,
    comments: false,
    plugins: [
      "minify-dead-code-elimination",
      "minify-mangle-names",
      "minify-simplify"
    ]
  }
})

[osidenate]

I get the same Out of Memory issue when running babili using BabiliPlugin/Webpack on my medium size React project. It works fine on my small React project.

[amarant]

I had the same problem, I used : node --max-old-space-size=4076 ... and it finished.

[kangax]

Similar report from @qfox "tried babili on the build and it just keeps on generating stuff. i killed it after 35mb or so." — https://gist.github.com/qfox/7307e414c5414b26b4f72b2b77b1ee30

[derekdon]

Same issues here on a large project.

new BabiliPlugin({
                test: /\.js($|\?)/i,
                comments: false,
                sourceMap: true
            }),

Tried @swernerx settings but the same error occurs.

new BabiliPlugin({
                comments: false,
                babili: {
                    minified: true,
                    comments: false,
                    plugins: [
                        "minify-dead-code-elimination",
                        "minify-mangle-names",
                        "minify-simplify"
                    ]
                }
            })

[postspectacular]

FWIW I noticed that babili goes into an infinite loop unless explicitly locally installed into a project. I.e. Installing globally and then using version via npm link babili will trigger the loop and eat up more & more memory, even with the smallest source files...

[pvdz]

@postspectacular fwiw I don't have it installed globally. There's very little that should need to be installed globally.

[derekdon]

@postspectacular same as @qfox, not installed globally.

[joseSantacruz]

same issue here

[Windstalker]

The issue was solved for me by turning off the option mangle. Please, fix this issue.

"env": {
      "production": {
        "only": [
          "app",
          "webpack"
        ],
        "presets": [
          [
            "babili",
            {
              "mangle": false
            }
          ]
        ],
        ...
      }
   }  

[vigneshshanmugam]

@Windstalker can you confirm the babili version?

[Windstalker]

@vigneshshanmugam [email protected]

[davisford]

Can confirm -- turning off mangle at least successfully minifies it.

├─┬ [email protected]
│ ├── [email protected] deduped
│ ├── [email protected]
│ └── [email protected]

[boopathi]

Interesting. Thanks for digging into the issue and finding out that mangler is causing this. I'll try to debug this sometime soon.

[brandonmp]

I had the same problem, I used : node --max-old-space-size=4076 ... and it finished.

A+, thx.

this worked for me as well (but boy does the build take awhile to finish)

[yocontra]

Still getting the same issue as of July/August - disabling mangle didn't help.

<--- Last few GCs --->

[24150:0x102801600]   126688 ms: Mark-sweep 1403.4 (1490.7) -> 1403.2 (1459.7) MB, 2206.9 / 0.0 ms  (+ 0.0 ms in 0 steps since start of marking, biggest step 0.0 ms, walltime since start of marking 2207 ms) last resort
[24150:0x102801600]   128938 ms: Mark-sweep 1403.2 (1459.7) -> 1403.2 (1459.7) MB, 2249.9 / 0.0 ms  last resort


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0xd18d03a66a1 <JS Object>
    1: getBindingIdentifiers [/Users/contra/Projects/staeco/node_modules/babili-webpack-plugin/node_modules/babel-types/lib/retrievers.js:~20] [pc=0x10c73d68fe4e](this=0x26c47a909239 <an Object with map 0x2405980dddd1>,node=0x2c5b756e2ad9 <a Node with map 0x52ecb9d1f61>,duplicates=0x3363d09023b1 <true>,outerOnly=0x3363d0902311 <undefined>)
    2: arguments adaptor frame: 2->3
    3: registerBind...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node::Abort() [/Users/contra/.nvm/versions/node/v7.10.0/bin/node]
 2: node::FatalException(v8::Isolate*, v8::Local<v8::Value>, v8::Local<v8::Message>) [/Users/contra/.nvm/versions/node/v7.10.0/bin/node]
 3: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [/Users/contra/.nvm/versions/node/v7.10.0/bin/node]
 4: v8::internal::Factory::NewTransitionArray(int) [/Users/contra/.nvm/versions/node/v7.10.0/bin/node]
 5: v8::internal::TransitionArray::Insert(v8::internal::Handle<v8::internal::Map>, v8::internal::Handle<v8::internal::Name>, v8::internal::Handle<v8::internal::Map>, v8::internal::SimpleTransitionFlag) [/Users/contra/.nvm/versions/node/v7.10.0/bin/node]
 6: v8::internal::Map::CopyReplaceDescriptors(v8::internal::Handle<v8::internal::Map>, v8::internal::Handle<v8::internal::DescriptorArray>, v8::internal::Handle<v8::internal::LayoutDescriptor>, v8::internal::TransitionFlag, v8::internal::MaybeHandle<v8::internal::Name>, char const*, v8::internal::SimpleTransitionFlag) [/Users/contra/.nvm/versions/node/v7.10.0/bin/node]
 7: v8::internal::Map::CopyAddDescriptor(v8::internal::Handle<v8::internal::Map>, v8::internal::Descriptor*, v8::internal::TransitionFlag) [/Users/contra/.nvm/versions/node/v7.10.0/bin/node]
 8: v8::internal::Map::CopyWithField(v8::internal::Handle<v8::internal::Map>, v8::internal::Handle<v8::internal::Name>, v8::internal::Handle<v8::internal::FieldType>, v8::internal::PropertyAttributes, v8::internal::Representation, v8::internal::TransitionFlag) [/Users/contra/.nvm/versions/node/v7.10.0/bin/node]
 9: v8::internal::Map::TransitionToDataProperty(v8::internal::Handle<v8::internal::Map>, v8::internal::Handle<v8::internal::Name>, v8::internal::Handle<v8::internal::Object>, v8::internal::PropertyAttributes, v8::internal::Object::StoreFromKeyed) [/Users/contra/.nvm/versions/node/v7.10.0/bin/node]
10: v8::internal::LookupIterator::PrepareTransitionToDataProperty(v8::internal::Handle<v8::internal::JSObject>, v8::internal::Handle<v8::internal::Object>, v8::internal::PropertyAttributes, v8::internal::Object::StoreFromKeyed) [/Users/contra/.nvm/versions/node/v7.10.0/bin/node]
11: v8::internal::Object::AddDataProperty(v8::internal::LookupIterator*, v8::internal::Handle<v8::internal::Object>, v8::internal::PropertyAttributes, v8::internal::Object::ShouldThrow, v8::internal::Object::StoreFromKeyed) [/Users/contra/.nvm/versions/node/v7.10.0/bin/node]
12: v8::internal::Runtime::SetObjectProperty(v8::internal::Isolate*, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>, v8::internal::LanguageMode) [/Users/contra/.nvm/versions/node/v7.10.0/bin/node]
13: v8::internal::Runtime_SetProperty(int, v8::internal::Object**, v8::internal::Isolate*) [/Users/contra/.nvm/versions/node/v7.10.0/bin/node]
14: 0x10c73c5043a7

[iamacup]

Hey team

I am experiencing this issue - but i have a working before and after with a specific library if it helps you debug:

https://github.com/iamacup/sliips-ui/commit/c437975c837a1b05476f3c36f716e59c5f4a756b

This commit works fine - if you do npm install then yarn build:local you will see it takes awhile to get through 91% asset optimisation, but it completes after a couple of minutes.

https://github.com/iamacup/sliips-ui/commit/c1d28a946d2e868baca7bae9c656339bd4744fb3

This commit is totally broken and yarn build:local will hang on the 91% for an hour or so and then run out of heap - even at 16 GB.

The only difference between the two commits is that i added a 571 KB dependency to the vendors.js file - specifically the pdf.js project's pdf.js file pdfjs-dist/build/pdf.js

I am wondering if its something to do with the fact that pdf.js contains webpack markup that is confusing things? anyway I will leave it with you - hope this helps.

[tkharuk]

Hi there,
not sure what is the real root cause for this but had the same issue which was a total blocker for CI. Neither allocating more memory nor turning off some flags didn't help us much and it was either OOM or build would take over an hour.
Our project is probably on a medium scale with minified bundle weighing somewhere near 4MB.

In the end had to switch to UglifyJS. It is a pity that such problem occurs recurrently and it is not clear at first glance what is the true root cause.

node: 7.9
npm: 5.0
used: react, mobx, decorators, babel-polyfill - targeting IE10+

[awkaiser]

I also ran into this problem on a larger React project and ended up moving from babel-minify-webpack-plugin (791 seconds before running out of memory) to uglifyjs-webpack-plugin (85 seconds, completed minification with source maps).

[damianobarbati]

@awkaiser what about bundle size? Did you compare them after moving to uglify? I actually didn't manage to get it working, weird errors and weird options (I'm talking about parse/output options that makes me think uglify attempt to trasform again es5<=>es8 code which is already done by babel). On the otherside babel-minify is flawless to me, just... slow :P

[awkaiser]

@damianobarbati Since the Uglify output size was roughly within expectations, I did not bother noting a size comparison with Babel Minify (which wasn't completing its build at all; would have had to roll back to an earlier version of the project).

[tkharuk]

@damianobarbati @awkaiser as for me, I don't recall exact numbers, but the results were roughly the same.

[caesarsol]

Same problem here, with multiple projects... Seems to work well until a certain size limit is reached, sometimes the addition of a dependency. Is that your case @iamacup ?

Is there some investigation going on this? Do you need any insight?

Thanks, hooray for Babel team! ;)

[taoabc]

Mark, same issue here. I use babel-minify-webpack-plugin to minify pdfjs-dist/build/pdf.worker.js