Execute training on multiple gpus

[mirkozaff]

Hi,

I have a question, is it possible to train the models on multiple gpus?

[lborke]

It is an issue of the Tensorflow backend and your CUDA/CuDNN configuration. Keras and this package are only a high level API. Normally, it should work well if everything in the backend is configured properly.

[mirkozaff]

Thank you for your reply! Sadly I have to work with CUDA 9.1 and Tensorflow 1.8, so I have manged to modify the function train in order to work with more than one gpu. Thank you for your info, it will be usefull when I'll upgrade my machine!

[lborke]

see also issue #133 for more information about docker containers. I have already tested keras_segmentation on tensorflow:1.14.0-gpu-py3 (with GPU support) and tensorflow:1.14.0-py3 (only CPU). On tensorflow:latest-gpu-py3 (TF version 2.0) not everything works. Starting with TF 2.0 Keras is a full part of TF and some code adaptation on the Keras level is necessary.

By the way, you don't need to worry about CUDA/CuDNN using Docker containers (as tensorflow:1.14.0-gpu-py3) because the appropriate CUDA/CuDNN drivers are already installed within the Docker image/container.

[GGDRriedel]

If you are using the non-dockerized version you can wrap your model in a multi_gpu model like so:

from keras.utils import multi_gpu_model
parallel_model = multi_gpu_model(model, gpus=2)
parallel_model.compile(optimizer="adam", loss="binary_crossentropy", metrics=["binary_accuracy"])