firebase-import icon indicating copy to clipboard operation
firebase-import copied to clipboard

Error Javascript heap out of memory

Open kingekartik opened this issue 6 years ago • 4 comments

Brief: While uploading a 400mb file using this tool, I get Javascript heap overflow error.

Command used:

firebase-import --database_url https://database.firebaseio.com/ --path / --json C:\User\Desktop\file.json --service_account C:\Users\service_account.json

Complete Error log:

<--- Last few GCs ---> [4052:000002065D718290] 155635 ms: Mark-sweep 1375.3 (1399.6) -> 1375.3 (1399.6) MB, 3254.4 / 0.0 ms (average mu = 0.091, current mu = 0.000) last resort GC in old space requested [4052:000002065D718290] 158828 ms: Mark-sweep 1375.3 (1399.6) -> 1375.3 (1399.6) MB, 3192.5 / 0.0 ms (average mu = 0.047, current mu = 0.000) last resort GC in old space requested <--- JS stacktrace ---> ==== JS stack trace ========================================= 0: ExitFrame [pc: 000000079E2DC5C1] Security context: 0x00d110a08ce9 <JSObject> 1: push [000002490F19CC39](this=0x031861c9b181 <JSArray[1956331]>,0x02b158d904a1 <Object map = 000002D7FA950751>) 2: chunkInternal [000000C0F32CEA41] [C:\Users\firebase-import.js:~179] [pc=000000079E5D300A](this=0x001a0279ad11 <JSGlobal Object>,ref=0x031861c9b1a1 <Reference map = 000002D... FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory 1: 00007FF78778121A v8::internal::GCIdleTimeHandler::GCIdleTimeHandler+4810 2: 00007FF78775A5B6 node::MakeCallback+4518 3: 00007FF78775AFA0 node_module_register+2160 4: 00007FF7879EB3EE v8::internal::FatalProcessOutOfMemory+846 5: 00007FF7879EB31F v8::internal::FatalProcessOutOfMemory+639 6: 00007FF787F29304 v8::internal::Heap::MaxHeapGrowingFactor+11476 7: 00007FF787F27098 v8::internal::Heap::MaxHeapGrowingFactor+2664 8: 00007FF787AC9C08 v8::internal::Factory::AllocateRawArray+56 9: 00007FF787ACA582 v8::internal::Factory::NewFixedArrayWithFiller+66 10: 00007FF787B3F91C v8::internal::HashTablev8::internal::NumberDictionary,v8::internal::NumberDictionaryShape::KeyAt+43820 11: 00007FF787B3F59D v8::internal::HashTablev8::internal::NumberDictionary,v8::internal::NumberDictionaryShape::KeyAt+42925 12: 00007FF787B33396 v8::internal::HashTablev8::internal::NumberDictionary,v8::internal::NumberDictionaryShape::EntryToIndex+71382 13: 00007FF787D061CE v8::internal::SharedFunctionInfo::SetScript+23662 14: 00007FF787CE8CA3 v8::internal::JSReceiver::class_name+20931 15: 00007FF787AB9FD1 v8::internal::wasm::WasmCodeManager::LookupCode+18929 16: 00007FF787ABD068 v8::internal::wasm::WasmCodeManager::LookupCode+31368 17: 000000079E2DC5C1

kingekartik avatar Jan 12 '19 18:01 kingekartik

Did you find a solution for this?

PedroMazarini avatar Jan 21 '19 13:01 PedroMazarini

Hi Pedro Mazarini de Oliveira,

Not actually, but I managed to do my task the other way. I decreased the database size (as it was flexible in my case to nearly about 200Mb), disabled my cloud functions for a while and then uploaded the database and then enabled it again. This worked for me.

Regards, Kartik

On Mon 21 Jan, 2019, 6:46 PM Pedro Mazarini de Oliveira < [email protected] wrote:

Did you find a solution for this?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/FirebaseExtended/firebase-import/issues/43#issuecomment-456070550, or mute the thread https://github.com/notifications/unsubscribe-auth/AZvKoV3OpIcq5_eD3snGzzeELTKEdwlBks5vFb2zgaJpZM4Z8xcW .

kingekartik avatar Jan 21 '19 13:01 kingekartik

For me, file size 913M :

node --max-old-space-size=8192 /usr/local/bin/firebase-import ...

and solve.

banghasan avatar Aug 25 '19 10:08 banghasan

I am just in the process of importing a 2.82 GB JSON file... and it seems to work 😎

firebase-import-3gb

qbasic16 avatar Oct 25 '20 14:10 qbasic16