Speed of training and related computer specs

[JulioEndara]

When training each step it's taking a little over 320 seconds. At that rate for 5000 steps it'd take over 18 days. I'm using a VM in google cloud with the following specs: 8vCPUs, 52Gb RAM, 1GPU (NVIDIA Tesla P100).

Is that the expected speed for that hardware? is there anything to tune in the code to take full advantage of the hardware? Can you share your specs and the speed you are getting?

thanks!

[Mohamad73]

I am doing it on two GTX 1070 and it is taking around 1.5 second per step

[Nortinwell]

Maybe the TensorFlow you installed is not the GPU version, try pip install tensorflow-gpu insteal of tensorflow. And maybe you should change the "gpu_num" in train_c3d_ucf101.py