Add LlamaCppInferenceEngine

Open vs4vijay opened this issue 1 year ago • 5 comments

Related to #167

Dec 21 '24 15:12 vs4vijay

Hi Vijay, this would be a great addition - we need better support for CPU in exo!

Did you forget to commit some files?

Dec 22 '24 23:12 AlexCheema

@vs4vijay , any updates?

Jan 23 '25 22:01 lexasub

:eyes: @vs4vijay definitely interested in this PR, though based on prev comments some files are missing?

Jan 29 '25 14:01 pcfreak30

Hey @AlexCheema , @lexasub , @pcfreak30 - I got carried away with other work. I plan to pick this up this week.

Jan 29 '25 17:01 vs4vijay

@vs4vijay sorry to bug :), but I have an interest in this as well. Would love to look at using this to extend support to Intel iGPU/dGPU

https://github.com/intel/ipex-llm/blob/main/docs/mddocs/Quickstart/llama_cpp_quickstart.md

Feb 21 '25 01:02 shellfyred