White-box-Cartoonization icon indicating copy to clipboard operation
White-box-Cartoonization copied to clipboard

Training error

Open yxt132 opened this issue 4 years ago • 2 comments

We ran into an error in the middle of the training process. We used 1 1080ti gpu with batch size of 16 (image size 256x256). But the training got killed and the error message is like:

pid = os.fork() OSError: [Errno 12] Cannot allocate memory File "train.py", line 192, in train superpixel_batch = utils.selective_adacolor(inter_out, power=1.2)

We tried to decrease the batch size to 9 and the same error occurred.

How can we fix this problem?

Thanks

yxt132 avatar Dec 12 '20 09:12 yxt132

@yxt132 gtx 1080ti has 11gb gpu memory and i think it's enough to train this network. In fact I managed to train this network in my gtx1070 laptop. The out of memory problem maybe RAM rather than gpu memory?

SystemErrorWang avatar Feb 05 '21 17:02 SystemErrorWang

@SystemErrorWang sir what was your complete system configuration, like ram, gpu vram, environment, if you share please ?

| and can you enable discussion too ?|

maifeeulasad avatar Mar 22 '21 10:03 maifeeulasad