jan
jan copied to clipboard
epic: Support Intel as Inference Engine
WIP Spec
- Need to figure out if BigDL or Intel Extensions are separate
- Have Extensions for each inference engine
model.jsonshould have anengine: intel-bigdlorengine: intel-transformers
Resources
Intel BigDL https://github.com/intel-analytics/BigDL
Intel Extension for Transformers https://medium.com/intel-analytics-software/efficient-streaming-llm-with-intel-extension-for-transformers-runtime-31ee24577d26