onnxruntime
onnxruntime copied to clipboard
[Build] Propagate build option for CUDA minimal to TRT
Description
Extend cuda minimal option to TRT provider, as with TRT 10 no linking to cuDNN is required anymore . Besides that with the new engine dump feature it is also possible to embed an engine in to an ONNX and not ship a builder lib. In addition to that this has roughly the same deserialization time/session setup time that using TRT standalone has.
Motivation and Context
exe_builder_lib\onnxruntime_perf_test.exe -I -e tensorrt -r 5 -i 'trt_engine_cache_enable|1 trt_timing_cache_enable|1 trt_dump_ep_context_model|1 trt_weightless_engine_enable|1' model.onnx
exe_no_builder_lib\onnxruntime_perf_test.exe -I -e tensorrt -r 5 -i 'trt_engine_cache_enable|1 trt_timing_cache_enable|1 trt_dump_ep_context_model|1 trt_weightless_engine_enable|1' model_ctx.onnx
@chilo-ms and @yf711 for review
Can you rebase it to main? main now has all the CI settings to run TRT 10
@chilo-ms Sure, done.
/azp run Linux CPU CI Pipeline, Linux CPU Minimal Build E2E CI Pipeline, Linux GPU CI Pipeline, Linux GPU TensorRT CI Pipeline, Linux OpenVINO CI Pipeline, MacOS CI Pipeline, ONNX Runtime Web CI Pipeline, onnxruntime-binary-size-checks-ci-pipeline, Linux QNN CI Pipeline
/azp run Windows CPU CI Pipeline, Windows GPU CI Pipeline, Windows GPU TensorRT CI Pipeline, Windows ARM64 QNN CI Pipeline, orttraining-linux-ci-pipeline, orttraining-linux-gpu-ci-pipeline, orttraining-ortmodule-distributed, ONNX Runtime React Native CI Pipeline, Windows x64 QNN CI Pipeline
/azp run Linux MIGraphX CI Pipeline, orttraining-amd-gpu-ci-pipeline
Azure Pipelines successfully started running 2 pipeline(s).
Azure Pipelines successfully started running 9 pipeline(s).
Azure Pipelines successfully started running 9 pipeline(s).
@jywu-msft Could you please review these build changes ? They enable a compile of the TRT EP without any other cuda dependencies other than cudart.
/azp run Big Models, Linux Android Emulator QNN CI Pipeline
Azure Pipelines successfully started running 2 pipeline(s).
@jywu-msft Could you please review these build changes ? They enable a compile of the TRT EP without any other cuda dependencies other than cudart.
sorry I missed this. LGTM.
Related: #21285 #21204