PowerInfer icon indicating copy to clipboard operation
PowerInfer copied to clipboard
verified publisher icon SJTU-IPADS

Where is the TurboSparse-Mixtral mlp_predictor?

Open MatthewCroughan opened this issue 1 year ago • 1 comments

Predictors for all other models are provided, but not this one https://huggingface.co/PowerInfer?search_models=predictor

I was wanting to convert TurboSparse-Mixtral into a quantized GGUF to reproduce the claims made about usage on smartphones.

MatthewCroughan avatar Jun 27 '24 10:06 MatthewCroughan

Similar situation here. Since it is basically Bamboo with an extra MLP as predictor within BambooMLP, I suspect we need standalone pretrained predictor weights s.t. they can be combined with the TurboSparse model.

yichen0104 avatar Jul 01 '24 13:07 yichen0104