CTranslate2
CTranslate2 copied to clipboard
OpenNMT
Fast inference engine for Transformer models
```python try: import transformers except ImportError: pass from ctranslate2.specs import ( transformer_spec, ) from ctranslate2.converters.transformers import TransformersConverter, register_loader, ModelLoader, _SUPPORTED_ACTIVATIONS @register_loader("GPTBigCodeConfig") class GPTBigCodeMHALoader(ModelLoader): @property def architecture_name(self): return "GPTBigCodeForCausalLM" def get_model_spec(self,...
Can ctranslate2 support convert yarn 128k model Link to the model: https://huggingface.co/NousResearch/Yarn-Llama-2-13b-128k
hi, is the current version not supporting the blip2 model? Do you have plans to support it ？
Any plans to support the Llava architecture as in the following shiny new models? See here on huggingface.co: liuhaotian/llava-v1.5-7b liuhaotian/llava-v1.5-13b And any of the clip models by chance?
Hi, may I know how is the prefix implemented for faster-whisper? I tried looking at the code, it seems like the tokens will be generated as usual (from the start...
Hi! First and foremost, thanks for the great package! I converted the model `ehartford/WizardLM-Uncensored-Falcon-7b` from HuggingFace with the following command: ```bash ct2-transformers-converter --model ehartford/WizardLM-Uncensored-Falcon-7b --output_dir ./falcon_7b_fp32 --trust_remote_code ``` After that,...
I keep running into a ValueError when using `score_batch` on a model architecture I've just converted. I'm using the `score_batch` function to filter some translation data to continue training the...
Have faced issues related to the generation of docker image for the generation of vmap. please check this [https://github.com/OpenNMT/papers/issues/6](issue) for more details. Thanks in advance
ctranslate2.models.Whisper() Can we support mps device ? currently, I got error, ValueError: unsupported device mps refer to https://developer.apple.com/metal/pytorch/ https://opennmt.net/CTranslate2/installation.html Can `WITH_ACCELERATE` be enabled by default? current is `off` env info...
Recently, a lot of benchmarks point to the fact that if you want to serve your models behind an API, continuous batching grants higher throughput and lower latency compared to...
