lingvo icon indicating copy to clipboard operation
lingvo copied to clipboard

Published 20 hours ago • verified publisher icon tensorflow

low GPU utilization rate

Open boji123 opened this issue 6 years ago • 2 comments

In locally sync mode, the GPU utilization is unstable (sometime 100%, sometime 0%), and the average GPU utilization is about 60%. I think this question may caused by IO when data feeding. Any ways to solve it?

boji123 avatar Aug 09 '19 06:08 boji123

How the input generator work? Will it prefetching data for next batch when GPU is caculating current batch?

boji123 avatar Aug 09 '19 06:08 boji123

The input generator does not prefetch data so that might indeed be the cause.

jonathanasdf avatar Aug 09 '19 18:08 jonathanasdf