hariji

请问和llama.cpp 相比有什么优化的地方吗？因为我看大部分代码都是和他重合的

8

虽然有点冒犯，但是如题

benchmark 测试的时候会卡住，如何解决呢？

1

./benchmark -p /opt/Convert/flm/qwen-14b-chart-int4.flm -f ../example/benchmark/prompts/beijing.txt -b 1 Load (323 / 323) Warmup... finish. AVX: ON AVX2: ON AARCH64: OFF Neon FP16: OFF Neon DOT: OFF

sherpa onnx 的 jnilibs 好了，但是 其他想用onnx 推理的 模型无法找到onnx环境

9

1、这个是日志 FATAL EXCEPTION: textTranslation Process: com.example.aartest, PID: 13235 java.lang.NoClassDefFoundError: Failed resolution of: Lai/onnxruntime/OrtEnvironment; at android.yan.kernel.translate.Translator$1.run(Translator.java:98) Caused by: java.lang.ClassNotFoundException: Didn't find class "ai.onnxruntime.OrtEnvironment" on path: DexPathList[[zip file "/data/app/~~JCTw255ZQ2sW71_agJPkQA==/com.example.aartest-cGT4KKr4Z9wQMCGedTZ5FQ==/base.apk"],nativeLibraryDirectories=[/data/app/~~JCTw255ZQ2sW71_agJPkQA==/com.example.aartest-cGT4KKr4Z9wQMCGedTZ5FQ==/lib/arm64, /data/app/~~JCTw255ZQ2sW71_agJPkQA==/com.example.aartest-cGT4KKr4Z9wQMCGedTZ5FQ==/base.apk!/lib/arm64-v8a, /system/lib64, /system_ext/lib64]]...

Prebuilding tokenizers for Windows arm

1

I can't using this on my X elite PC , platform is arm ,but os is windows.

U7 155H can't use Npu , U5 ,U9 it's be OK

1

Failed to infer model with device NPU: Exception from src\inference\src\cpp\infer_request.cpp:223: Exception from src\plugins\intel_npu\src\compiler_adapter\src\ze_graph_ext_wrappers.cpp:114: L0 zeGraphSetArgumentValue result: ZE_RESULT_ERROR_INVALID_ARGUMENT, code 0x78000004 - generic error code for invalid arguments . how to solve...