jan icon indicating copy to clipboard operation
jan copied to clipboard

epic: Support Intel as Inference Engine

Open dan-menlo opened this issue 2 years ago • 0 comments

WIP Spec

  • Need to figure out if BigDL or Intel Extensions are separate
  • Have Extensions for each inference engine
  • model.json should have an engine: intel-bigdl or engine: intel-transformers

Resources

Intel BigDL https://github.com/intel-analytics/BigDL

Intel Extension for Transformers https://medium.com/intel-analytics-software/efficient-streaming-llm-with-intel-extension-for-transformers-runtime-31ee24577d26

dan-menlo avatar Nov 28 '23 14:11 dan-menlo