node-twitter icon indicating copy to clipboard operation
node-twitter copied to clipboard

"JavaScript heap out of memory" when media/upload with 60mb video

Open GeoffreyPlitt opened this issue 6 years ago • 2 comments

Getting an OOM error from client makePost while uploading a twitter video. It worked before, and I'm not sure what changed to cause this. How to debug?

  twitter created twitter client with opts:  { consumer_key: '<SNIPPED>',
  consumer_secret: '<SNIPPED>',
  access_token_key: '<SNIPPED>',
  access_token_secret: '<SNIPPED>' } +0ms
  twitter download first time +4s
  twitter download second time +6ms
  twitter { video_url: 'https://res.cloudinary.com/vista-stardust/video/upload/w_375,h_667,c_fill,q_100/l_text:Roboto_20_bold:bob,co_white/e_shadow:4,co_black,x_1,y_1/fl_layer_apply,g_north,y_20/l_bottom-gradient,g_south,w_1.0/l_text:Roboto_16_medium:The%20Matrix,co_white,g_south_west,y_65,x_100/l_text:Roboto_12:1999,co_white,o_70,g_south_west,x_100,y_48/l_stars-5,g_south_west,y_15,x_100/l_fetch:aHR0cHM6Ly9pbWFnZS50bWRiLm9yZy90L3AvdzkyLy9sWnBXcHJKcWJJRnBFVjV1b0hmb0swS0NuVFcuanBn,w_68,c_fill,g_south_west,y_15,x_15,e_sharpen,q_100/l_heart-icon,g_south_east,y_15,x_15,q_100/l_comment-icon,g_south_east,y_57,x_15,q_100/v1493145624/eceafc75zqi74suayvkm.mov',
  twitter   dest_file_path: 'tmp//eceafc75zqi74suayvkm.mov',
  twitter   tweet_text: 'This is a test',
  twitter   media_size: 67963644 } +6s
  twitter FILE DATA - first 100: AAAAFGZ0eXBxdCAgAAACAHF0ICAAAD+7bW9vdgAAAGxtdmhkAAAAAAAAAAAAAAAAAAAD6AAAZDsAAQAAAQAAAAAAAAAAAAAAAAEA +2ms
  twitter FILE DATA - last 100: Aj03SUwPXuc0fHPoZM3nymPn/HwYDNDNnf3TG+M+Q7r+6FsnPTmzTDmMDxYQ8PiYsh2tqvYqtgap11snOgtgpoaiaGvY1Cydd3fg +0ms
  twitter makePost { endpoint: 'media/upload',
  media_id: undefined,
  command: 'INIT',
  status: undefined } +1ms
  twitter makePost: success +133ms
  twitter initUpload: data: { media_id: 997551703141462000,
  media_id_string: '997551703141462016',
  expires_after_secs: 86400,
  media_key: '7_997551703141462016' } 

 +0ms
  twitter makePost { endpoint: 'media/upload',
  media_id: '997551703141462016',
  command: 'APPEND',
  status: undefined } +3ms

<--- Last few GCs --->

   18753 ms: Mark-sweep 1382.7 (1428.6) -> 1382.6 (1435.6) MB, 868.2 / 0.0 ms [allocation failure] [scavenge might not succeed].
   20148 ms: Mark-sweep 1389.2 (1435.6) -> 1389.2 (1435.6) MB, 1389.5 / 0.0 ms (+ 0.2 ms in 104 steps since start of marking, biggest step 0.0 ms) [allocation failure] [scavenge might not succeed].


<--- JS stacktrace --->
Cannot get stack trace in GC.
FATAL ERROR: MarkCompactCollector: semi-space copy, fallback in old gen Allocation failed - JavaScript heap out of memory
 1: node::Abort() [/Users/giro/.nvm/versions/node/v6.11.1/bin/node]
 2: node::FatalException(v8::Isolate*, v8::Local<v8::Value>, v8::Local<v8::Message>) [/Users/giro/.nvm/versions/node/v6.11.1/bin/node]
 3: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [/Users/giro/.nvm/versions/node/v6.11.1/bin/node]
 4: v8::internal::MarkCompactCollector::EvacuateNewSpaceVisitor::AllocateTargetObject(v8::internal::HeapObject*, v8::internal::HeapObject**) [/Users/giro/.nvm/versions/node/v6.11.1/bin/node]
 5: v8::internal::MarkCompactCollector::EvacuateNewSpaceVisitor::Visit(v8::internal::HeapObject*) [/Users/giro/.nvm/versions/node/v6.11.1/bin/node]
 6: v8::internal::MarkCompactCollector::VisitLiveObjects(v8::internal::MemoryChunk*, v8::internal::MarkCompactCollector::HeapObjectVisitor*, v8::internal::MarkCompactCollector::IterationMode) [/Users/giro/.nvm/versions/node/v6.11.1/bin/node]
 7: v8::internal::MarkCompactCollector::Evacuator::EvacuatePage(v8::internal::MemoryChunk*) [/Users/giro/.nvm/versions/node/v6.11.1/bin/node]
 8: v8::internal::PageParallelJob<v8::internal::EvacuationJobTraits>::Task::RunInternal() [/Users/giro/.nvm/versions/node/v6.11.1/bin/node]
 9: v8::internal::MarkCompactCollector::EvacuatePagesInParallel() [/Users/giro/.nvm/versions/node/v6.11.1/bin/node]
10: v8::internal::MarkCompactCollector::EvacuateNewSpaceAndCandidates() [/Users/giro/.nvm/versions/node/v6.11.1/bin/node]
11: v8::internal::MarkCompactCollector::CollectGarbage() [/Users/giro/.nvm/versions/node/v6.11.1/bin/node]
12: v8::internal::Heap::MarkCompact() [/Users/giro/.nvm/versions/node/v6.11.1/bin/node]
13: v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/Users/giro/.nvm/versions/node/v6.11.1/bin/node]
14: v8::internal::Heap::CollectGarbage(v8::internal::GarbageCollector, char const*, char const*, v8::GCCallbackFlags) [/Users/giro/.nvm/versions/node/v6.11.1/bin/node]
15: v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationSpace) [/Users/giro/.nvm/versions/node/v6.11.1/bin/node]
16: v8::internal::Runtime_AllocateInTargetSpace(int, v8::internal::Object**, v8::internal::Isolate*) [/Users/giro/.nvm/versions/node/v6.11.1/bin/node]
17: 0x1c3854062bf
Abort trap: 6

GeoffreyPlitt avatar May 18 '18 18:05 GeoffreyPlitt

Mac 10.13.4, Node 6.11.1

GeoffreyPlitt avatar May 18 '18 19:05 GeoffreyPlitt

@desmondmorris Any ideas

GeoffreyPlitt avatar Jul 12 '18 00:07 GeoffreyPlitt