PowerInfer
PowerInfer copied to clipboard
Which version of falcon-40b model used in llama.cpp reference in the demo?
Prerequisites
Before submitting your question, please ensure the following:
- [x] I am running the latest version of PowerInfer. Development is rapid, and as of now, there are no tagged versions.
- [x] I have carefully read and followed the instructions in the README.md.
- [x] I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
Question Details
I want to try the demo in READMD using falcon-40b model to compare with llama.cpp.But I don't know which model version should I use in llama.cpp. Relu version(PowerInfer/ReluFalcon-40B-PowerInfer-GGUF) or the original model(tiiuae/falcon-40b)?
Additional Context
Thank you very much!