InferLLM

InferLLM copied to clipboard

在树莓派3b+上部署，出现无法打开模型的问题

4

我在树莓派3b+上运行chatglm2，无法打开模型，请问这种情况怎么解决？ `main: seed = 1715851844 Assert ' m_file ' failed at file : /home/dominic/project/InferLLM-main/src/file.cpp line 10 : inferllm::InputFile::InputFile(const std::string &, bool), extra message: Failed to open model file.Aborted `

Dominic23331

官方的部署教程在make那一步一直显示“make: *** No targets specified and no makefile found. Stop.”

ASHLEYDX

在微调过的llama2模型上能够完成转换和量化，但在运行模型的时候会报错 ``` # ./llama -m ../llama2-13b-sft-filterd-v17/llama2-13b-sft-filterd-v17-inferllm-fp32.bin -g GPU --version 2 main: seed = 1709878763 total vocab length = 68419 weight tok_embeddings.weight is not match. Assert ' weight->length() == nr_number '...

Wowoho

python API support

4

Hi, any plan on support python API?

lucasjinreal

如果我想要将该项目移植到NPU上需要实现哪些函数接口

yinghuo302

请问可以在不支持V扩展的RISC-V CPU上运行吗

1

TheKiteRunner24

arm 平台输出乱码

3

tanjatang

I got the error on centos 7: failed to tokenize string!

2

I got the error like this on centos 7: failed to tokenize string! [root@VM-0-15-centos build]# ./llama -m chinese-alpaca-7b-q4.bin -t 2 main: seed = 1700961097 model is new , version =...

Mignet

make报错

3

在centos7.8环境中编译，cmake使用的版本是 ： cmake version 3.23.1 ，make过程报错： make版本： GNU Make 3.82 [root@node02 build]# make [ 2%] Building CXX object CMakeFiles/InferLLM.dir/src/core/device.cpp.o In file included from /root/InferLLM-main/src/utils.h:13:0, from /root/InferLLM-main/src/kern/kernel.h:6, from /root/InferLLM-main/src/core/device.h:6, from /root/InferLLM-main/src/core/device.cpp:2:...

hoyin2013

chatglm3有计划支持吗？

3

sunzhe09