Driver-Anomaly-Detection
Driver-Anomaly-Detection copied to clipboard
More RAM is consumed as training epoch increases
I have repeated the experiment for many times and eventually at the 14th epoch, the process is killed, mainly due to my CPU RAM is exhausted. My current configuration is 8 core CPUs, 30GB CPU RAM, P5000 16GB GPU.
I have tried to reduce memory bank size, but it's still the same.
reduce num_worker thread and batch size.
I have the same problem when I try to train this model, and anything can I do?