Multiple stream share memory allocation

[Joeyzhouqihui]

Hi, sorry for bothering. I am wondering if it is possible to let multiple models running on different cuda streams to share one single memory allocator?

[zerollzeng]

I think it's feasible, see https://docs.nvidia.com/deeplearning/tensorrt/api/c_api/classnvinfer1_1_1_i_gpu_allocator.html @nvpohanh correct me if I'm wrong.

[zerollzeng]

From what I see TF-TRT uses a custom GPU allocator and there may be multiple streams running in parallel.

[nvpohanh]

Yes, you can use it across multiple streams

[zerollzeng]

closed