mnist_dpsgd_tutorial_vectorized Training slower on GPU

[paperblack]

I'm running the vectorized tutorial (tf-gpu 1.14) and the per epoch training time on GPU (Nvidia Tesla M10) is slower than the non-vectorized implementation:

GPU Vectorized : 203.55 seconds/epoch Non-vectorized : 115 seconds/epoch

I was expecting to see better runtimes on GPU, but I am not sure why it's slower. Any help would be greatly appreciated.