Lilium comments

Results 7 comments of


                                            Lilium

量化后的paddle模型转TensorRT模型的问题

> @ceci3 @cryoco 感谢回复，其实我的本意是希望能够把量化后的paddle模型转化为独立的trt模型，脱离paddle框架，仅使用trt去做推理，但是目前我还没有想到实现的途径。我先尝试您们提供的建议测试下性能，再次感谢！老哥找到方法了吗，我也是相同的问题

使用自动压缩resnet18后，得到的量化模型，不能单独使用tensorRT部署

老哥找到解决办法没，我也是相同的问题

Transformer-like model hurts accuracy converted from onnx(opset16) when using fp16 in TensorRT-8.6, and TensorRT-8.6 cannot parse onnx(opset17) because of LayerNormalization/LayerNorm

I found my onnx/onnx-tensorrt version is wrong, and convert engines from onnx(opset17) successfully with the right version, but the accurary is still bad: | model | trt-op17-fp32 | trt-op17-fp16 |...

Transformer-like model hurts accuracy converted from onnx(opset16) when using fp16 in TensorRT-8.6, and TensorRT-8.6 cannot parse onnx(opset17) because of LayerNormalization/LayerNorm

> > I found my onnx/onnx-tensorrt version is wrong, and convert engines from onnx(opset17) successfully with the right version, but the accurary is still bad: | model | trt-op17-fp32 |...

Transformer-like model hurts accuracy converted from onnx(opset16) when using fp16 in TensorRT-8.6, and TensorRT-8.6 cannot parse onnx(opset17) because of LayerNormalization/LayerNorm

> Could you please try TRT 9.3/9.2? Thanks! I have tried TRT 9.2, the results are still bad, but TRT 8.5(without flash attention) has good performance, as below: | model...

Transformer-like model hurts accuracy converted from onnx(opset16) when using fp16 in TensorRT-8.6, and TensorRT-8.6 cannot parse onnx(opset17) because of LayerNormalization/LayerNorm

> Does it pass with polygraphy? e.g. polygraphy run model.onnx --trt --onnxrt The problem of LayerNormalization in opset17 cannot be converted to engine has been solved and I found that...

Transformer-like model hurts accuracy converted from onnx(opset16) when using fp16 in TensorRT-8.6, and TensorRT-8.6 cannot parse onnx(opset17) because of LayerNormalization/LayerNorm

> > > Does it pass with polygraphy? e.g. polygraphy run model.onnx --trt --onnxrt > > > > > > The problem of LayerNormalization in opset17 cannot be converted to...