DRAEM
DRAEM copied to clipboard
CUDA out of memory(test_DREAM.py)
Hello.
I tried to run test_DREAM.py, but CUDA out of memory occur. (I use 15.90 GiB memory)
I don't know well about all of your codes, but when i check GPU memory, memory usage continue to increase.
I think some variables need to be allocated at CPU. (by using .cpu().detach())
After many attempts, I know where i need to change.
https://github.com/VitjanZ/DRAEM/blob/main/test_DRAEM.py#L96:L99
Before(Original codes)
display_images[cnt_display] = gray_rec[0] display_gt_images[cnt_display] = gray_batch[0] display_out_masks[cnt_display] = t_mask[0] display_in_masks[cnt_display] = true_mask[0]
After(changed codes)
display_images[cnt_display] = gray_rec[0].cpu().detach() display_gt_images[cnt_display] = gray_batch[0].cpu().detach() display_out_masks[cnt_display] = t_mask[0].cpu().detach() display_in_masks[cnt_display] = true_mask[0].cpu().detach()
Thanks, this helps a lot.
or adding with torch.no_grad(): before testing (line 77) also works. and i removed the "display" part cause it's not used in the code, i guess it's for visualization.