chaox

The AMP is mainly target on training instead of serving.(https://www.tensorflow.org/guide/keras/mixed_precision) Have you observed the significant performance difference for serving as well? If so, could you share the benchmark and related...

Thanks for the experiments and numbers! Based on the number, we could add the option. I will also follow up with our GPU team.