fastllm

fastllm copied to clipboard

ANDROID速度很慢 1.3t/s

3

我按照README完成ANDROID编译，拷贝benchmark到termux, 跑chatglm2-int4.flm 速度只有1.3t/s 我也试了直接在termux里面编译，直接cmake, 速度也是一样 我的手机是Snapdragon 8 gen 2, t=4， NDK r24 r25c都试过了

JianbangZ

substract -> subtract

eltociear

make -j 出现报错 _mm256_setm128i was not declared in this scope

1

cmake 3.24.3 gcc 9.5.0 cuda 10.2 cmake .. -DUSE_CUDA=ON ok，有warning： CUDA_ARCHITECTURE is empty for target benchmark/fastllm-tool/... make -j出现报错： fastllm-master/src/fastllm.cpp: In member function ‘void fastllm::Data::CalcWeightSum()’: fastllm-master/src/fastllm.cpp:495:31: error: ‘_mm256_set_m128i’ was not declared...

Joris-Fu

很难用啊，真的支持cuda吗？试了好几次都没有跑在cuda上

7

zhaoying9105

按readme的快速加载方法chatglm int4在cpu上报错

3

FastLLM Error: Linear's weight's shape error 请问这是什么问题呢？fp16的可以正常运行。

cherishtttz

White-Friday

chatglm2运行llm.from_hf报错Error: cublas error

25

cuda11.7 cudnn8 运行llm.from_hf转换chatglm2-6B模型报 convert ( 200 / 200 ) Warmup... status = 15 1 16 128 Error: cublas error. Aborted

shancq

python batch推理接口是不是还没实现

4

wqh17101

cmake .. -DUSE_CUDA=ON出现报错

3

-- USE_CUDA: ON -- PYTHON_API: OFF -- CMAKE_CXX_FLAGS -pthread --std=c++17 -O2 -march=native CMake Error: Could not find cmake module file: CMakeDetermineCUDACompiler.cmake CMake Error: Error required internal CMake variable not set,...

741075810

Segmentation fault

1

(pytorch1.13) user@user:~/zx/ChatGLM2-6B$ CUDA_VISIBLE_DEVICES=3 python api_jiasu.py -p ./model.flm llm model: chatglm Load (200 / 200) Warmup... Segmentation fault (core dumped) 模型导出成功，导出后加载时报错

neverstoplearn