sejunkwonme comments

Repositories
Issues
Comments

Results 1 comments of


                                            sejunkwonme

[Performance] cudaMemcpyAsync dominates runtime for batch inference with FP32 inputs

I've experienced exact same thing like you. If you bind your input tensor and output tensor with Ort::IOBinding, the cudamemcopyasync block is changed to cudaStreamSynchronize. This means, the long-time consuming...