FgSegNet_v2 icon indicating copy to clipboard operation
FgSegNet_v2 copied to clipboard

Memory leak and how to train using a gpu ?

Open AriouatI opened this issue 4 years ago • 3 comments

how can i enable gpu training cause i'm trying to train on colab but it always goes beyond the 12Gb RAM limit is it even normal ? i'm using keras 2.0.6 and tensorflow 1.1.0 as advised but the training process is extremely long and only works with batch_size= 1 otherwise there's a RAM memory problem !

Thanks for your help. @lim-anggun

AriouatI avatar Aug 24 '20 14:08 AriouatI

@AriouatI. If you enable gpu runtime the scripts will automatically use gpu. Are you getting this error while extracting masks ???

Akhilesh64 avatar Aug 26 '20 15:08 Akhilesh64

@Akhilesh64 thanks for your comment.

The gpu is enabled but unused (maybe due to tf version used in this code ?) The RAM problem happens either :

  • during the model.fit() when the batch size > 1 (looks like a memory leak)
  • during the call to getData.

@lim-anggun

AriouatI avatar Aug 26 '20 16:08 AriouatI

how can i enable gpu training cause i'm trying to train on colab but it always goes beyond the 12Gb RAM limit is it even normal ? i'm using keras 2.0.6 and tensorflow 1.1.0 as advised but the training process is extremely long and only works with batch_size= 1 otherwise there's a RAM memory problem !

Thanks for your help. @lim-anggun

Could you please tell me how to download tensorflow-gpu 1.1.0? Why can't I find it with conda command?

helenypzhang avatar Feb 08 '21 02:02 helenypzhang