microbundle

microbundle copied to clipboard

How to reproduce:

1. Clone https://github.com/daangn/urlpack
2. cd urlpack/packages/msgpack (https://github.com/daangn/urlpack/tree/main/packages/msgpack)
3. yarn install
4. yarn _build (microbundle --format esm,cjs)

This always fails with an out-of-memory issue. on Node v14, v16, Mac OS and Ubuntu 20.04

It is not reproduced in other packages with similar code and settings.

yarn run v1.22.5
$ microbundle --format esm,cjs

<--- Last few GCs --->

[44348:0x5519880]    13088 ms: Scavenge 3273.1 (3307.3) -> 3273.1 (3307.3) MB, 36.7 / 0.0 ms  (average mu = 0.906, current mu = 0.894) allocation failure
[44348:0x5519880]    13243 ms: Scavenge 3536.7 (3570.9) -> 3536.7 (3570.9) MB, 36.9 / 0.0 ms  (average mu = 0.906, current mu = 0.894) allocation failure
[44348:0x5519880]    13643 ms: Scavenge 3574.7 (3608.9) -> 3574.7 (3608.9) MB, 5.6 / 0.0 ms  (average mu = 0.906, current mu = 0.894) allocation failure


<--- JS stacktrace --->

FATAL ERROR: MarkCompactCollector: young object promotion failed Allocation failed - JavaScript heap out of memory
 1: 0xa222f0 node::Abort() [/home/cometkim/.asdf/installs/nodejs/14.17.0/bin/node]
 2: 0x96411f node::FatalError(char const*, char const*) [/home/cometkim/.asdf/installs/nodejs/14.17.0/bin/node]
 3: 0xb97f1e v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/home/cometkim/.asdf/installs/nodejs/14.17.0/bin/node]
 4: 0xb98297 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/home/cometkim/.asdf/installs/nodejs/14.17.0/bin/node]
 5: 0xd52fd5  [/home/cometkim/.asdf/installs/nodejs/14.17.0/bin/node]
 6: 0xd83bbe v8::internal::EvacuateNewSpaceVisitor::Visit(v8::internal::HeapObject, int) [/home/cometkim/.asdf/installs/nodejs/14.17.0/bin/node]
 7: 0xd8fbf6 v8::internal::FullEvacuator::RawEvacuatePage(v8::internal::MemoryChunk*, long*) [/home/cometkim/.asdf/installs/nodejs/14.17.0/bin/node]
 8: 0xd7bd8f v8::internal::Evacuator::EvacuatePage(v8::internal::MemoryChunk*) [/home/cometkim/.asdf/installs/nodejs/14.17.0/bin/node]
 9: 0xd7c008 v8::internal::PageEvacuationTask::RunInParallel(v8::internal::ItemParallelJob::Task::Runner) [/home/cometkim/.asdf/installs/nodejs/14.17.0/bin/node]
10: 0xd6e8e9 v8::internal::ItemParallelJob::Run() [/home/cometkim/.asdf/installs/nodejs/14.17.0/bin/node]
11: 0xd91b50 void v8::internal::MarkCompactCollectorBase::CreateAndExecuteEvacuationTasks<v8::internal::FullEvacuator, v8::internal::MarkCompactCollector>(v8::internal::MarkCompactCollector*, v8::internal::ItemParallelJob*, v8::internal::MigrationObserver*, long) [/home/cometkim/.asdf/installs/nodejs/14.17.0/bin/node]
12: 0xd923ec v8::internal::MarkCompactCollector::EvacuatePagesInParallel() [/home/cometkim/.asdf/installs/nodejs/14.17.0/bin/node]
13: 0xd925b5 v8::internal::MarkCompactCollector::Evacuate() [/home/cometkim/.asdf/installs/nodejs/14.17.0/bin/node]
14: 0xda45b1 v8::internal::MarkCompactCollector::CollectGarbage() [/home/cometkim/.asdf/installs/nodejs/14.17.0/bin/node]
15: 0xd60878 v8::internal::Heap::MarkCompact() [/home/cometkim/.asdf/installs/nodejs/14.17.0/bin/node]
16: 0xd62368 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/home/cometkim/.asdf/installs/nodejs/14.17.0/bin/node]
17: 0xd642a5 v8::internal::Heap::HandleGCRequest() [/home/cometkim/.asdf/installs/nodejs/14.17.0/bin/node]
18: 0xd0a787 v8::internal::StackGuard::HandleInterrupts() [/home/cometkim/.asdf/installs/nodejs/14.17.0/bin/node]
19: 0x107c20a v8::internal::Runtime_StackGuardWithGap(int, unsigned long*, v8::internal::Isolate*) [/home/cometkim/.asdf/installs/nodejs/14.17.0/bin/node]
20: 0x1423279  [/home/cometkim/.asdf/installs/nodejs/14.17.0/bin/node]
Aborted (core dumped)
error Command failed with exit code 134.

I had this issue as well, the cause was that I was inadvertently bundling a ton of dependencies on accident.

I also tried this as an intermediary step to see how large the bundle was

export NODE_OPTIONS=--max_old_space_size=4096 && microbundle  --target node

The result was 430kb for a simple helper lib. From there I was able to narrow down which dependencies were casuing issues by adding them to peerDependencies which are externalized by default. In the end, my lib ended up being 2.3k after externalizing them

Thanks @christopher-caldwell , but, my library has zero dependencies 😅

I'm running in to a similar issue. It might be related to an external typescript import. Was there any resolution?

npm run build:widget                                                                                                             53s

> [email protected] build:widget
> NODE_OPTIONS=--max_old_space_size=4096 microbundle build --target node


<--- Last few GCs --->

[22261:0x7fb990040000]    41038 ms: Scavenge 4058.2 (4133.0) -> 4057.4 (4133.5) MB, 6.5 / 0.0 ms  (average mu = 0.836, current mu = 0.584) allocation failure; 
[22261:0x7fb990040000]    41055 ms: Scavenge 4058.7 (4133.5) -> 4058.1 (4143.5) MB, 11.5 / 0.0 ms  (average mu = 0.836, current mu = 0.584) allocation failure; 
[22261:0x7fb990040000]    43136 ms: Mark-sweep 4062.1 (4143.5) -> 4060.5 (4147.3) MB, 2065.3 / 0.0 ms  (average mu = 0.541, current mu = 0.044) allocation failure; GC in old space requested


<--- JS stacktrace --->

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
 1: 0x106a08d55 node::Abort() (.cold.1) [/Users/m.salpeter/.nvm/versions/node/v18.12.0/bin/node]
 2: 0x1054d1259 node::Abort() [/Users/m.salpeter/.nvm/versions/node/v18.12.0/bin/node]
 3: 0x1054d143e node::OOMErrorHandler(char const*, bool) [/Users/m.salpeter/.nvm/versions/node/v18.12.0/bin/node]
 4: 0x10564c0a3 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/Users/m.salpeter/.nvm/versions/node/v18.12.0/bin/node]
 5: 0x105814ae5 v8::internal::Heap::FatalProcessOutOfMemory(char const*) [/Users/m.salpeter/.nvm/versions/node/v18.12.0/bin/node]
 6: 0x1058134c2 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/Users/m.salpeter/.nvm/versions/node/v18.12.0/bin/node]
 7: 0x1058057fa v8::internal::HeapAllocator::AllocateRawWithLightRetrySlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/Users/m.salpeter/.nvm/versions/node/v18.12.0/bin/node]
 8: 0x105806175 v8::internal::HeapAllocator::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/Users/m.salpeter/.nvm/versions/node/v18.12.0/bin/node]
 9: 0x1057e7dba v8::internal::Factory::AllocateRaw(int, v8::internal::AllocationType, v8::internal::AllocationAlignment) [/Users/m.salpeter/.nvm/versions/node/v18.12.0/bin/node]
10: 0x1057df7e4 v8::internal::FactoryBase<v8::internal::Factory>::NewFixedArrayWithFiller(v8::internal::Handle<v8::internal::Map>, int, v8::internal::Handle<v8::internal::Oddball>, v8::internal::AllocationType) [/Users/m.salpeter/.nvm/versions/node/v18.12.0/bin/node]
11: 0x105b11b9d v8::internal::MaybeHandle<v8::internal::OrderedHashMap> v8::internal::OrderedHashTable<v8::internal::OrderedHashMap, 2>::Rehash<v8::internal::Isolate>(v8::internal::Isolate*, v8::internal::Handle<v8::internal::OrderedHashMap>, int) [/Users/m.salpeter/.nvm/versions/node/v18.12.0/bin/node]
12: 0x105c16875 v8::internal::Runtime_MapGrow(int, unsigned long*, v8::internal::Isolate*) [/Users/m.salpeter/.nvm/versions/node/v18.12.0/bin/node]
13: 0x106018cf9 Builtins_CEntry_Return1_DontSaveFPRegs_ArgvOnStack_NoBuiltinExit [/Users/m.salpeter/.nvm/versions/node/v18.12.0/bin/node]
14: 0x105fecdf3 Builtins_MapPrototypeSet [/Users/m.salpeter/.nvm/versions/node/v18.12.0/bin/node]
15: 0x10b0b1ded 
16: 0x10b00538c 
17: 0x10af51b95 
18: 0x10aacd61a 
[1]    22260 abort      npm run build:widget
```                                                                                                                             44s