text-generation-inference icon indicating copy to clipboard operation
text-generation-inference copied to clipboard

Published 20 hours ago • verified publisher icon huggingface

Mixtral 8x22b

Open sa- opened this issue 2 months ago • 3 comments

Model description

Mixtral's most capable model yet https://huggingface.co/mistralai/Mixtral-8x22B-Instruct-v0.1

I see that it's not explicitly mentioned here: https://github.com/huggingface/text-generation-inference/blob/2d0a7173d4891e7cd5f9b77f8e0987b82a339e51/docs/source/supported_models.md?plain=1#L23 And I wonder if it's already works

Open source status

  • [X] The model implementation is available
  • [X] The model weights are available

Provide useful links for the implementation

No response

sa- avatar Apr 19 '24 13:04 sa-

Also have the same question.

By checking their release log, I assume TGI hasn't supported Mixtral 8x22b yet.

ponshane avatar Apr 22 '24 14:04 ponshane

Any updates on this? Thanks

nrepesh avatar Apr 27 '24 21:04 nrepesh

I am currently unable to run the model as well. I could be having another issue unrelated issues though.

mallorbc avatar May 16 '24 05:05 mallorbc