TensorRT icon indicating copy to clipboard operation
TensorRT copied to clipboard

Question: Disable Optimizations for TensorRT

Open YixuanSeanZhou opened this issue 1 year ago • 5 comments

Question

Because there are so many optimizations that TRT performs, sometimes it is very hard to isolate the issue if we see regression in model accuracy. I know we have the builder_optimization_level flag, but it seems to only control which kernel is used when executing the model.

I wonder if there is more fine-grained control? For example, I want to prevent fusions, or prevent removing dead code.

To give more context: In my specific use case, I am interested in isolating whether resolving Q/DQ nodes can causes regression in model. What I am interested to achieve is to only enable Q/DQ resolution and disable all other optimizations. Is this achievable?

Thanks in advance

YixuanSeanZhou avatar Aug 12 '24 22:08 YixuanSeanZhou