keras-applications icon indicating copy to clipboard operation
keras-applications copied to clipboard

There is a bug using GPU in MobileNetV3Small

Open asparsa opened this issue 2 years ago • 0 comments

I wanted to change the Implementation of MobilebetV3Small for my use case. so I cloned the keras-applications and tried to run the mobileNetV3 directly from the source. but it had raised errors when I was using GPU tensorflow Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try to see if a warning log message was printed above. I tried TPU and CPU and everything was right. so I tried to limit the GPU memory usage manually to solve it as GPU memory was almost full when I got the error. as a result, I had to pick a smaller batch and it didn't work well so I gave up and trained my network with TPU but it took so long.

Summary

there is a bug when you are training your network with GPU

Environment

  • Keras version: 2.9
  • Keras-applications version:1.0.8
  • Keras backend with version:2.9

Logs or source codes for reproduction

model5=models.Sequential() model5.add(MobileNetV3(stack_fn,1000,input_shape=(480,640,3),model_type='small',alpha=0.75,minimalistic=False,include_top=False,backend=keras.backend)) model5.add(layers.GlobalMaxPool2D()) model5.add(layers.Flatten()) model5.add(layers.Dense(9, activation='tanh')) model5.add(layers.Dense(3)) model5.build(input_shape=(1,480, 640, 3))

asparsa avatar May 18 '22 09:05 asparsa