ThiloteE
ThiloteE
Nowadays, lots of datasets for llms are published at huggingface.
Phi-3.5 MoE seems to be based on https://huggingface.co/microsoft/GRIN-MoE/tree/main. Maybe their technical report at https://arxiv.org/abs/2409.12136 can help at identifying differences to other MoE architectures, which should ease adoption in llama.cpp.
The PR in the transformers repo to support Phi-3.5 MoE has been merged and is featured in release [v4.46.0](https://github.com/huggingface/transformers/releases/tag/v4.46.0), so maybe finally llama.cpp can add this model architecture? Oh and...
Can you still reproduce?
My first thought would be to check, if Tex Studio is triggering Windows to lock the file (which would be an external issue and better be reported to Tex Studio),...
@cosmic-snow did a great job at the community contributed wiki page: https://github.com/nomic-ai/gpt4all/wiki/Local-API-Server. Are you aware?
Can be closed, I guess? If still doesn't work, you can re-open.
Unfortunately I am not fluent in Chinese. The original model card does not specify, if mainland or taiwanese.
If this model is not good enough, I can also try to find a finetune of it, but it is hard to find good finetunes nowadays, since the huggingface open...
I will add a PR for Qwen2 as well. Maybe one of its finetunes as well. I think there are more finetunes for Qwen2