benchmarks icon indicating copy to clipboard operation
benchmarks copied to clipboard

Published 20 hours ago • verified publisher icon tensorflow

Mismatched BROADCAST CPU/GPU

Open x666633 opened this issue 6 years ago • 1 comments

when i run horovod in tf_cnn_benchmarks, i got the error

(0) Failed precondition: Mismatched BROADCAST CPU/GPU device selection: One rank specified device CPU, but another rank specified device GPU. [[node HorovodBroadcast_v0_cg_affine2_weights_0 (defined at :263) ]] [[HorovodBroadcast_v0_cg_affine0_biases_0/_37]] (1) Failed precondition: Mismatched BROADCAST CPU/GPU device selection: One rank specified device CPU, but another rank specified device GPU. [[node HorovodBroadcast_v0_cg_affine2_weights_0 (defined at :263) ]]

x666633 avatar Oct 27 '19 07:10 x666633

Unfortunately, tf_cnn_benchmarks is unmaintained and I am not very familiar with Horovod. So it is unlikely you'll get answers to Horovod issues :(

reedwm avatar Jan 17 '20 01:01 reedwm