testMNNFromOnnx.py 验证Qwen-1_8B-Chat ONNX 出错

[sunnyzhaohui]

平台(如果交叉编译请再附上交叉编译目标平台):

Linux

Github版本:

参考文档 https://mnn-docs.readthedocs.io/en/latest/tools/convert.html#id4 验证 testMNNFromOnnx.py

编译方式:

1）魔搭下载Qwen-1_8B-Chat 模型文件 2）使用如下脚本生成 onnx 和 mnn python3 llm_export.py --embed_bin
--embed_bf16
--onnx_path onnx
--type Qwen-1_8B-Chat
--path ./Qwen-1_8B-Chat
--export_embed
--export_token
--mnn_path mnn
--export

3）使用命令检查 onnx 文件 python ./tools/script/testMNNFromOnnx.py ./transformers/llm/export/onnx/llm.onnx

4）输出结果日志：

(wangenv) cmdc2023@fxzs-ai-gpu:~/zhwang_workSpace/MNN/MNN$ python ./tools/script/testMNNFromOnnx.py ./transformers/llm/export/onnx/llm.onnx Dir exist onnx/test.onnx tensor(float) tensor(bool) tensor(int64) tensor(float) ['logits', 'presents'] inputs: input_ids onnx/ attention_mask onnx/ position_ids onnx/ past_key_values onnx/ 2024-06-26 09:06:12.759355532 [E:onnxruntime:, sequential_executor.cc:516 ExecuteKernel] Non-zero status code returned while running Concat node. Name:'/h.0/attn/Concat_9' Status Message: /onnxruntime_src/onnxruntime/core/framework/op_kernel.cc:83 virtual OrtValue* onnxruntime::OpKernelContext::OutputMLValue(int, const onnxruntime::TensorShape&) status.IsOK() was false. Shape mismatch attempting to re-use buffer. {1,1,16,128} != {1,2,16,128}. Validate usage of dim_value (values should be > 0) and dim_param (all values with the same string should equate to the same size) in shapes in the model.

Traceback (most recent call last): File "./tools/script/testMNNFromOnnx.py", line 287, in t.Test() File "./tools/script/testMNNFromOnnx.py", line 263, in Test self.__run_onnx() File "./tools/script/testMNNFromOnnx.py", line 172, in __run_onnx outputs = ort_session.run(None, inputs) File "/home/cmdc2023/miniconda3/envs/wangenv/lib/python3.8/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 220, in run return self._sess.run(output_names, input_feed, run_options) onnxruntime.capi.onnxruntime_pybind11_state.RuntimeException: [ONNXRuntimeError] : 6 : RUNTIME_EXCEPTION : Non-zero status code returned while running Concat node. Name:'/h.0/attn/Concat_9' Status Message: /onnxruntime_src/onnxruntime/core/framework/op_kernel.cc:83 virtual OrtValue* onnxruntime::OpKernelContext::OutputMLValue(int, const onnxruntime::TensorShape&) status.IsOK() was false. Shape mismatch attempting to re-use buffer. {1,1,16,128} != {1,2,16,128}. Validate usage of dim_value (values should be > 0) and dim_param (all values with the same string should equate to the same size) in shapes in the model.

[jxt1234]

这个是默认输入不支持，可以自行修改校验脚本。也可以跳过这一步直接转mnn 使用，这个模型一般没有问题

[github-actions[bot]]

Marking as stale. No activity in 60 days.