Paddle2ONNX int8量化后的模型导出onnx时，有两个op不支持

fake_quantize_dequantize_moving_average_abs_max fake_channel_wise_quantize_dequantize_abs_max

Nov 22 '21 02:11 zhangxianjie1314

您好，请拉这个PR测试：https://github.com/PaddlePaddle/Paddle2ONNX/pull/399

Nov 22 '21 03:11 yeliang2258

另外，用户您好，请问你转换为ONNX模型的目的是？用于部署的话，请问部署引擎是？

Nov 23 '21 03:11 yeliang2258

Hi, @zhangxianjie1314

我们准备了一份简单的问题调研（半分钟可完成），如有时间可帮忙完成调研，帮助Paddle2ONNX做得更好 https://iwenjuan.baidu.com/?code=r8hu2s

Nov 25 '21 06:11 jiangjiajun

另外，用户您好，请问你转换为ONNX模型的目的是？用于部署的话，请问部署引擎是？

想用onnx来进行部署，但是用tensorrt对量化后的onnx进行部署时，发现onnx2trt似乎不支持int8的onnx，所以想请教一下，如果对int8模型进行部署，有什么推荐的方法吗？谢谢。

Nov 29 '21 06:11 zhangxianjie1314

可以直接使用paddleinference进行部署，paddleinference可以直接部署。另外使用TensorRT是可以直接加载我们转换出来的ONNX量化模型的，具体可以参考TensorRT的例子，直接加载ONNX模型。

Nov 29 '21 07:11 yeliang2258

好的，谢谢了，我尝试一下

Nov 30 '21 01:11 zhangxianjie1314

可以直接使用paddleinference进行部署，paddleinference可以直接部署。另外使用TensorRT是可以直接加载我们转换出来的ONNX量化模型的，具体可以参考TensorRT的例子，直接加载ONNX模型。

为什么TensorRT官网上说不支持int8的onnx进行部署？

Nov 30 '21 01:11 zhangxianjie1314

请安装TensorRT8+，具体链接：https://docs.nvidia.com/deeplearning/tensorrt/developer-guide/index.html#working-with-int8

Nov 30 '21 01:11 yeliang2258

你也可以使用如下类似的命令进行转换： trtexec --onnx=quant_model.onnx --workspace=2048 --explicitBatch --int8

Dec 03 '21 02:12 yeliang2258

你也可以使用如下类似的命令进行转换： trtexec --onnx=quant_model.onnx --workspace=2048 --explicitBatch --int8

好的好的，谢谢

Dec 06 '21 02:12 zhangxianjie1314

Dear @yeliang2258 , I have been trying your work (#399), but it failed. Do you have any suggestion? Thanks for your time. Here is the log:

2022-01-13 08:41:22 [WARNING]
======================

 Not Find Name : hardswish_38.tmp_0

======================

2022-01-13 08:41:22 [WARNING]
======================

 Not Find Name : hardswish_42.tmp_0

======================

2022-01-13 08:41:22 [WARNING]
======================

 Not Find Name : hardswish_46.tmp_0

======================

2022-01-13 08:41:22 [WARNING]
======================

 Not Find Name : hardswish_50.tmp_0

======================

2022-01-13 08:41:23 [INFO]      ONNX model generated is valid.
Traceback (most recent call last):
  File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddle2onnx-0.9.0-py3.7.egg/paddle2onnx/utils.py", line 43, in check_model
  File "/usr/local/python3.7.0/lib/python3.7/site-packages/onnx/checker.py", line 104, in check_model
    C.check_model(protobuf_string)
onnx.onnx_cpp2py_export.checker.ValidationError: Unrecognized attribute: split for operator Split

==> Context: Bad node spec for node. Name: Split_0 OpType: Split

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/python3.7.0/bin/paddle2onnx", line 33, in <module>
    sys.exit(load_entry_point('paddle2onnx==0.9.0', 'console_scripts', 'paddle2onnx')())
  File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddle2onnx-0.9.0-py3.7.egg/paddle2onnx/command.py", line 195, in main
  File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddle2onnx-0.9.0-py3.7.egg/paddle2onnx/command.py", line 159, in program2onnx
  File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddle2onnx-0.9.0-py3.7.egg/paddle2onnx/convert.py", line 85, in program2onnx
  File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddle2onnx-0.9.0-py3.7.egg/paddle2onnx/convert.py", line 38, in export_onnx
  File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddle2onnx-0.9.0-py3.7.egg/paddle2onnx/graph/onnx_graph.py", line 367, in export_proto
  File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddle2onnx-0.9.0-py3.7.egg/paddle2onnx/utils.py", line 45, in check_model
Exception: ONNX model is not valid.

Jan 13 '22 08:01 edwardnguyen1705

@edwardnguyen1705 hello, please pull this PR to test: https://github.com/PaddlePaddle/Paddle2ONNX/pull/399

Jan 13 '22 11:01 yeliang2258

@edwardnguyen1705 hello, please pull this PR to test: #399

Hi @yeliang2258 , Thanks for your time. I checked that you have already fixed the split issue. However, I have just tried to pull PR #399, but the result is the same.

$ git clone https://github.com/yeliang2258/Paddle2ONNX
$ cd Paddle2ONNX
$ git checkout quantize_dev
$ python setup.py install

onnx: 1.9.0
onnxruntime : 1.10.0
paddle2onnx: 0.9.0

2022-01-14 02:15:17 [WARNING]
======================

 Not Find Name : hardswish_38.tmp_0

======================

2022-01-14 02:15:17 [WARNING]
======================

 Not Find Name : hardswish_42.tmp_0

======================

2022-01-14 02:15:18 [WARNING]
======================

 Not Find Name : hardswish_46.tmp_0

======================

2022-01-14 02:15:18 [WARNING]
======================

 Not Find Name : hardswish_50.tmp_0

======================

2022-01-14 02:15:18 [INFO]      ONNX model generated is valid.
Traceback (most recent call last):
  File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddle2onnx-0.9.0-py3.7.egg/paddle2onnx/utils.py", line 43, in check_model
  File "/usr/local/python3.7.0/lib/python3.7/site-packages/onnx/checker.py", line 104, in check_model
    C.check_model(protobuf_string)
onnx.onnx_cpp2py_export.checker.ValidationError: Unrecognized attribute: split for operator Split

==> Context: Bad node spec for node. Name: Split_0 OpType: Split

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/python3.7.0/bin/paddle2onnx", line 33, in <module>
    sys.exit(load_entry_point('paddle2onnx==0.9.0', 'console_scripts', 'paddle2onnx')())
  File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddle2onnx-0.9.0-py3.7.egg/paddle2onnx/command.py", line 195, in main
  File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddle2onnx-0.9.0-py3.7.egg/paddle2onnx/command.py", line 159, in program2onnx
  File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddle2onnx-0.9.0-py3.7.egg/paddle2onnx/convert.py", line 85, in program2onnx
  File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddle2onnx-0.9.0-py3.7.egg/paddle2onnx/convert.py", line 38, in export_onnx
  File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddle2onnx-0.9.0-py3.7.egg/paddle2onnx/graph/onnx_graph.py", line 367, in export_proto
  File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddle2onnx-0.9.0-py3.7.egg/paddle2onnx/utils.py", line 45, in check_model
Exception: ONNX model is not valid.

Jan 14 '22 02:01 edwardnguyen1705

Hi @yeliang2258 , Regarding the hardwish activation layer, could you take a look: https://github.com/ultralytics/yolov5/issues/825

Jan 14 '22 02:01 edwardnguyen1705

@yeliang2258 请问，这两个op支持了么，试过了好多分支了，仍然解决不了，目前的需求是：将paddle模型量化后转为onnx =========== fake_channel_wise_quantize_dequantize_abs_max =========== =========== fake_quantize_dequantize_moving_average_abs_max ===========

Jun 29 '22 11:06 nongwoluanlai666

@nongwoluanlai666 用这个PR来转吧：https://github.com/PaddlePaddle/Paddle2ONNX/pull/651 另外，请问你导出为ONNX用什么推理引擎部署呢？

Jun 29 '22 11:06 yeliang2258

@yeliang2258 您好，我是打算用在MNN部署的。目前MNN可以用protobuffer 格式作为量化参数的输入，因为不熟悉protobuffer，我用#651导出的txt转换仍有困难; MNN官网的量化参数定义如下： MNN_compression.proto

Jun 30 '22 06:06 nongwoluanlai666

Paddle2ONNX Paddle2ONNX copied to clipboard

int8量化后的模型导出onnx时，有两个op不支持

Paddle2ONNX
Paddle2ONNX copied to clipboard