phi 3.5 moe usage

[GlasslessPizza]

trafficstars

The model_downloader.py script doesn't list the recently supported phi 3.5 moe. I'd like to also know if it's ok to use the v0.3 release from Jul 6 as-is to run it. Thanks!

[ayttop]

microsoft/Phi-3.5-MoE-instruct/??????????????????????????

[ayttop]

how to convert microsoft/Phi-3.5-MoE-instruct to gguf?

[foldl]

@GlasslessPizza You can try the latest v0.4 or build it yourself.

You can use -m :phi3.5:moe in Python binding.

[foldl]

@ayttop Sorry, GGUF is not supported. This project uses something like the old GGML file format.

[ayttop]

Can the form be converted to ggml and any body convert it to gguf>?

[foldl]

No.