onnxruntime icon indicating copy to clipboard operation
onnxruntime copied to clipboard

[Build] Propagate build option for CUDA minimal to TRT

Open gedoensmax opened this issue 1 year ago • 10 comments

Description

Extend cuda minimal option to TRT provider, as with TRT 10 no linking to cuDNN is required anymore . Besides that with the new engine dump feature it is also possible to embed an engine in to an ONNX and not ship a builder lib. In addition to that this has roughly the same deserialization time/session setup time that using TRT standalone has.

Motivation and Context

exe_builder_lib\onnxruntime_perf_test.exe -I -e tensorrt -r 5 -i 'trt_engine_cache_enable|1 trt_timing_cache_enable|1 trt_dump_ep_context_model|1 trt_weightless_engine_enable|1' model.onnx


exe_no_builder_lib\onnxruntime_perf_test.exe -I -e tensorrt -r 5 -i 'trt_engine_cache_enable|1 trt_timing_cache_enable|1 trt_dump_ep_context_model|1 trt_weightless_engine_enable|1' model_ctx.onnx

gedoensmax avatar May 16 '24 09:05 gedoensmax

@chilo-ms and @yf711 for review

gedoensmax avatar May 16 '24 09:05 gedoensmax

Can you rebase it to main? main now has all the CI settings to run TRT 10

chilo-ms avatar May 16 '24 18:05 chilo-ms

@chilo-ms Sure, done.

gedoensmax avatar May 17 '24 10:05 gedoensmax

/azp run Linux CPU CI Pipeline, Linux CPU Minimal Build E2E CI Pipeline, Linux GPU CI Pipeline, Linux GPU TensorRT CI Pipeline, Linux OpenVINO CI Pipeline, MacOS CI Pipeline, ONNX Runtime Web CI Pipeline, onnxruntime-binary-size-checks-ci-pipeline, Linux QNN CI Pipeline

chilo-ms avatar May 17 '24 15:05 chilo-ms

/azp run Windows CPU CI Pipeline, Windows GPU CI Pipeline, Windows GPU TensorRT CI Pipeline, Windows ARM64 QNN CI Pipeline, orttraining-linux-ci-pipeline, orttraining-linux-gpu-ci-pipeline, orttraining-ortmodule-distributed, ONNX Runtime React Native CI Pipeline, Windows x64 QNN CI Pipeline

chilo-ms avatar May 17 '24 15:05 chilo-ms

/azp run Linux MIGraphX CI Pipeline, orttraining-amd-gpu-ci-pipeline

chilo-ms avatar May 17 '24 15:05 chilo-ms

Azure Pipelines successfully started running 2 pipeline(s).

azure-pipelines[bot] avatar May 17 '24 15:05 azure-pipelines[bot]

Azure Pipelines successfully started running 9 pipeline(s).

azure-pipelines[bot] avatar May 17 '24 15:05 azure-pipelines[bot]

Azure Pipelines successfully started running 9 pipeline(s).

azure-pipelines[bot] avatar May 17 '24 15:05 azure-pipelines[bot]

@jywu-msft Could you please review these build changes ? They enable a compile of the TRT EP without any other cuda dependencies other than cudart.

gedoensmax avatar Jun 25 '24 12:06 gedoensmax

/azp run Big Models, Linux Android Emulator QNN CI Pipeline

snnn avatar Jul 09 '24 15:07 snnn

Azure Pipelines successfully started running 2 pipeline(s).

azure-pipelines[bot] avatar Jul 09 '24 15:07 azure-pipelines[bot]

@jywu-msft Could you please review these build changes ? They enable a compile of the TRT EP without any other cuda dependencies other than cudart.

sorry I missed this. LGTM.

jywu-msft avatar Jul 09 '24 15:07 jywu-msft

Related: #21285 #21204

snnn avatar Jul 12 '24 16:07 snnn