Punctuate with GPU

[MeteorBurn]

Hello! I try to use GPU with CUDA with cuDNN for punctuating on Windows. My Theano tests show that CPU (i7 7700) spend 11 sec and GPU (GTX 1060 6 GB) 0,2 sec.

But I noticed interesting thing, when I punctuate a big text with more 75 000 symbols, GPU punctuates texts faster than CPU, but when text has less 50 000 symbols, then CPU punctuates texts faster than GPU.

How can this be? And how to make the GPU punctuate texts faster regardless of their size?

[robinsax]

There's some overhead involved in loading to and from the GPU so it makes sense you would see a threshold like that. Not sure how to get around that, it's more of a hardware thing.

[MeteorBurn]

Thank you!