onnxruntime icon indicating copy to clipboard operation
onnxruntime copied to clipboard

Published 20 hours ago • verified publisher icon microsoft

[TensorRT EP] Update TRT10.0 deprecated api

Open yf711 opened this issue 1 year ago • 6 comments
trafficstars

Description

Note:

  • This PR would remove C4996 suppression in tensorrt_execution_provider.cc only (according to Nvidia, places with nvinfer.h included need C4996 suppression, when /Zc:__cplusplus is enabled in ORT win build)
  • A follow-up PR will be raised to update deprecated TRT Plugin api usage.

Here are deprecated apis to be updated in this PR:

deprecated api Update
kCUBLAS /
kCUBLAS_LT /
kCUDNN /
reallocateOutput Superseded by reallocateOutputAsync with cudaStream_t argument
createExecutionContextWithoutDeviceMemory Superseded by createExecutionContext() with parameter.
Check ExecutionContextAllocationStrategy::kUSER_MANAGED for more detail

Motivation and Context

TRT deprecated api list: https://docs.nvidia.com/deeplearning/tensorrt/api/c_api/deprecated.html

yf711 avatar Jun 10 '24 22:06 yf711

Please remove lines with 4996 from the following files.

  • onnxruntime/core/providers/tensorrt/nv_includes.h
  • onnxruntime/test/unittest_main/test_main.cc

tianleiwu avatar Jun 12 '24 05:06 tianleiwu

Please remove lines with 4996 from the following files.

  • onnxruntime/core/providers/tensorrt/nv_includes.h
  • onnxruntime/test/unittest_main/test_main.cc

Thanks for the context. These 4996 usage have been reverted

yf711 avatar Jun 12 '24 17:06 yf711

there are build errors?

D:\a_work_temp\TensorRT-10.0.1.6.Windows10.x86_64.cuda-11.8\include\NvInferRuntimeBase.h#L593 D:\a_work_temp\TensorRT-10.0.1.6.Windows10.x86_64.cuda-11.8\include\NvInferRuntimeBase.h(593,16): Error C4996: 'nvinfer1::v_1_0::IGpuAllocator::allocate': was declared deprecated

jywu-msft avatar Jun 16 '24 22:06 jywu-msft

Including nvinfer.h would cause C4996 errors during Windows ORT build, when /Zc:__cplusplus is enabled: nvinfer1::v_1_0::IGpuAllocator::allocate nvinfer1::v_1_0::IGpuAllocator::deallocate nvinfer1::IPluginV2 nvinfer1::IPluginV2Ext It seems that there are indirect usage of these api by TRT internally. I'm adding C4996 suppression back temporarily and checking with Nvidia for better solution on that.

yf711 avatar Jun 17 '24 19:06 yf711

was the rellocateOutput->reallocateOutputAsync change tested on Faster-RCNN with oss parser?

jywu-msft avatar Jun 21 '24 02:06 jywu-msft

was the rellocateOutput->reallocateOutputAsync change tested on Faster-RCNN with oss parser?

That has been tested with no issue

yf711 avatar Jun 28 '24 19:06 yf711