Llama-2-Open-Source-LLM-CPU-Inference icon indicating copy to clipboard operation
Llama-2-Open-Source-LLM-CPU-Inference copied to clipboard

Published 20 hours ago • verified publisher icon kennethleungty

Any ideas what "Illegal Instruction" indicates?

Open jediwarpraptor opened this issue 1 year ago • 1 comments

  • Using llama-2-7b-chat.ggmlv3.q8_0.bin downloaded today
  • I've setup everything running on a Dell PowerEdge 16-processor machine w/ 128GB of RAM.
  • Pointed to a directory that only had 4 PDFs in it.
  • Every question just using your standard poetry run python main.py "question text?" returns just one line:

"Illegal instruction"

Some questions take longer to return the same.

Any clues?

jediwarpraptor avatar Jul 29 '23 17:07 jediwarpraptor

Maybe you are running on a VM environment, and cat /proc/cpuinfo | grep flags has no avx or certain required flag.

In such case, try to adjust VM CPU settings.

randalhsu avatar Aug 11 '23 09:08 randalhsu