Memory of RAM

[hubery4]

Hi JiaRen,

I have met a question that when I run 'finetune.py' in my computer with RAM = 32G. It always will be killed after one epoch. The error is this. I just use the KITTI_2015 training data for finetune based on pretrained ScenFlow model.

Is the problem that the RAM=32G is not enough? How much the memory of RAM you used during training? Or is there anything in 'finetune.py' I need to change to run in RAM <= 32G?

Thank you

[JiaRenChang]

Hi, @hubery4 You could reduce the batch size both of trainset and valset. In validation, a pair of KITTI image takes around ~4.xGB memory. It seems that 32GB is not enough for batch size 8 in validation.

[hubery4]

@JiaRenChang Thank you for the suggestion. I have tried batch-size=1, it still had this problem. I also tried in Google Colab. Each time the process will stop after 1 training epoch.

[hubery4]

@JiaRenChang Thank you I have figured out this problem. It is the dimension problem of true_disp and pred_disp.

[tyleryzhu]

@hubery4 Could you explain what you mean by dimension problem? I fixed this already in finetune.py by squeezing the extra dimension in axis=1, but I'm still getting this out of memory error with a 32G gpu. Thanks!