PowerInfer icon indicating copy to clipboard operation
PowerInfer copied to clipboard

Which version of falcon-40b model used in llama.cpp reference in the demo?

Open wuooo339 opened this issue 1 year ago • 0 comments

Prerequisites

Before submitting your question, please ensure the following:

  • [x] I am running the latest version of PowerInfer. Development is rapid, and as of now, there are no tagged versions.
  • [x] I have carefully read and followed the instructions in the README.md.
  • [x] I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).

Question Details

I want to try the demo in READMD using falcon-40b model to compare with llama.cpp.But I don't know which model version should I use in llama.cpp. Relu version(PowerInfer/ReluFalcon-40B-PowerInfer-GGUF) or the original model(tiiuae/falcon-40b)?

Additional Context

Thank you very much!

wuooo339 avatar Dec 25 '24 14:12 wuooo339