Driver-Anomaly-Detection icon indicating copy to clipboard operation
Driver-Anomaly-Detection copied to clipboard

More RAM is consumed as training epoch increases

Open hongvin opened this issue 2 years ago • 2 comments

I have repeated the experiment for many times and eventually at the 14th epoch, the process is killed, mainly due to my CPU RAM is exhausted. My current configuration is 8 core CPUs, 30GB CPU RAM, P5000 16GB GPU.

I have tried to reduce memory bank size, but it's still the same.

hongvin avatar Aug 17 '21 05:08 hongvin

reduce num_worker thread and batch size.

LEE-SEON-WOO avatar Apr 01 '22 11:04 LEE-SEON-WOO

I have the same problem when I try to train this model, and anything can I do?

CreatedTRYNA avatar Sep 06 '22 12:09 CreatedTRYNA