ipex-llm icon indicating copy to clipboard operation
ipex-llm copied to clipboard
verified publisher icon intel-analytics

Support for MultiGPU Mixtral Inference

Open sriraman2020 opened this issue 1 year ago • 1 comments

Hi, Could you pls add support to MultiGPU Mixtral Inference.

https://github.com/jason-dai/BigDL/tree/main/python/llm/example/GPU/Deepspeed-AutoTP

sriraman2020 avatar Feb 01 '24 07:02 sriraman2020

Hi @sriraman2020

Sorry that Mixtral multi-gpu inference is currently not supported. We will update in this issue if this is supported in the future.

hkvision avatar Feb 05 '24 02:02 hkvision