plex-sync icon indicating copy to clipboard operation
plex-sync copied to clipboard

Out of memory Large Library

Open segator opened this issue 6 years ago • 2 comments

Out of memory on a 32GB system (30 GB not used)

plex-sync serverA/2,r serverB/2,rw Large Library(80K episodes on 2 servers) <--- Last few GCs --->

[4332:0x55a7b4ce31d0] 638448 ms: Mark-sweep 1655.4 (1720.0) -> 1655.4 (1688.5) MB, 2072.7 / 0.0 ms last resort GC in old space requested [4332:0x55a7b4ce31d0] 640695 ms: Mark-sweep 1655.4 (1688.5) -> 1655.4 (1688.5) MB, 2246.6 / 0.0 ms last resort GC in old space requested

<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x72c21a18fe1 <JSObject> 1: /* anonymous /(aka / anonymous /) [/usr/local/lib/node_modules/plex-sync/lib/plex.js:~124] [pc=0x200bebd43a3e](this=0x72c21a02241 ,video=0x64a08ede011 <Object map = 0x80fe66b9361>) 2: arguments adaptor frame: 3->1 3: forEach(this=0x234fd89ec6f9 <JSArray[32]>) 4: / anonymous /(aka / anonymous */) [/usr/local/lib/node_modules/plex-sync/lib/plex.js:124] [bytecode=...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory 1: node::Abort() [node] 2: 0x55a7b3305011 [node] 3: v8::Utils::ReportOOMFailure(char const*, bool) [node] 4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [node] 5: v8::internal::Factory::NewUninitializedFixedArray(int) [node] 6: 0x55a7b2ef1add [node] 7: v8::internal::Runtime_GrowArrayElements(int, v8::internal::Object**, v8::internal::Isolate*) [node] 8: 0x200beba840bd

segator avatar Dec 12 '18 18:12 segator

At some point I'd like to optimize plex-sync so that it's not holding so much information in memory, but as a temporary workaround, you can increase the node heap size. Node limits itself to around 1.7GB of memory on 64-bit machines, so for instance to run with 4GB of memory:

$ node --max-old-space-size=4096 `which plex-sync` [...]

jacobwgillespie avatar Dec 12 '18 21:12 jacobwgillespie

Sorry to bump an old thread, I am still getting memory errors with even as much as 32gb. Does anyone know if there is a improved fork of this or a better solution that exists now? Need to sync watched states in two libraries and don't have file access to one server nor do I want to risk messing directly with .db files. Any help is greatly appreciated.

RyanWor avatar Oct 27 '22 15:10 RyanWor