CTranslate2 icon indicating copy to clipboard operation
CTranslate2 copied to clipboard

Support for Phi3-Small, Medium, and Vision

Open BBC-Esq opened this issue 1 month ago • 1 comments

Does ctranslate2 to have plans to support the recently released small, medium, and vision. I've tried running them with transformers (on Windows) and can't get past a Triton and compiler not found kind of errors). I've not had the same issues with phi3-4k so it stands to reason that the phi3 converter in ctranslate2 might need to be modified to handle these models...even though they're in the same family.

ALSO, as far as I know ctranslate2 has never supported any vision models. Phi-3-Vision might be a good place to start or maybe you guys don't plan to support that general category due to time constraints.

Here are the models. Let me know if I can help in any way:

https://huggingface.co/microsoft/Phi-3-small-8k-instruct https://huggingface.co/microsoft/Phi-3-small-128k-instruct https://huggingface.co/microsoft/Phi-3-medium-4k-instruct https://huggingface.co/microsoft/Phi-3-medium-128k-instruct https://huggingface.co/microsoft/Phi-3-vision-128k-instruct

BBC-Esq avatar May 23 '24 12:05 BBC-Esq