【Mac上运行llm_demo报错：./llm_demo model/qwen-1.8b-int4

[luocf]

model name : Qwen_1.8b

hw.cpufamily: 458787763 , size = 4 The device support i8sdot:1, support fp16:1, support i8mm: 0 load tokenizer load tokenizer Done [ 10% ] load model/qwen-1.8b-int4/lm.mnn model ... Done! [ 13% ] load model/qwen-1.8b-int4/embedding.mnn model ... Done! [ 20% ] load model/qwen-1.8b-int4/block_0.mnn model ... Done! [ 24% ] load model/qwen-1.8b-int4/block_1.mnn model ... Done! [ 27% ] load model/qwen-1.8b-int4/block_2.mnn model ... Done! [ 31% ] load model/qwen-1.8b-int4/block_3.mnn model ... Done! [ 34% ] load model/qwen-1.8b-int4/block_4.mnn model ... Done! [ 38% ] load model/qwen-1.8b-int4/block_5.mnn model ... Done! [ 41% ] load model/qwen-1.8b-int4/block_6.mnn model ... Done! [ 45% ] load model/qwen-1.8b-int4/block_7.mnn model ... Done! [ 48% ] load model/qwen-1.8b-int4/block_8.mnn model ... Done! [ 52% ] load model/qwen-1.8b-int4/block_9.mnn model ... Done! [ 55% ] load model/qwen-1.8b-int4/block_10.mnn model ... Done! [ 58% ] load model/qwen-1.8b-int4/block_11.mnn model ... Done! [ 62% ] load model/qwen-1.8b-int4/block_12.mnn model ... Done! [ 65% ] load model/qwen-1.8b-int4/block_13.mnn model ... Done! [ 69% ] load model/qwen-1.8b-int4/block_14.mnn model ... Done! [ 72% ] load model/qwen-1.8b-int4/block_15.mnn model ... Done! [ 76% ] load model/qwen-1.8b-int4/block_16.mnn model ... Done! [ 79% ] load model/qwen-1.8b-int4/block_17.mnn model ... Done! [ 83% ] load model/qwen-1.8b-int4/block_18.mnn model ... Done! [ 86% ] load model/qwen-1.8b-int4/block_19.mnn model ... Done! [ 90% ] load model/qwen-1.8b-int4/block_20.mnn model ... Done! [ 93% ] load model/qwen-1.8b-int4/block_21.mnn model ... Done! [ 97% ] load model/qwen-1.8b-int4/block_22.mnn model ... Done! [100% ] load model/qwen-1.8b-int4/block_23.mnn model ... Done!

Q: 你好！

A: Reshape error: 28160 -> 26624 Compute Shape Error for onnx::Cast_30 code=3 in onForward, 372 Segmentation fault: 11

[jxt1234]

1. 现在建议用完整模型，即合并为同一模型的用法
2. mnn 版本是什么？

[luocf]

模型用的是这个：qwen-1.8b-int4 mnn版本用是github上最新版本： commit 976d1d7c0f916ea8a7acc3d31352789590f00b18 (HEAD -> master, origin/master, origin/HEAD) Merge: 2d926e4a ed68e167 Author: jxt1234 [email protected] Date: Mon Apr 8 16:17:33 2024 +0800

Merge pull request #2812 from juju812/patch-2

[bugfix] add return value to lambda of std::async

[luocf]

》〉现在建议用完整模型，即合并为同一模型的用法 请问，如何合并为一个模型？

[github-actions[bot]]

Marking as stale. No activity in 60 days.