CTranslate2
CTranslate2 copied to clipboard
OpenNMT
Fast inference engine for Transformer models
Is there any way to inference using CT2/OpenNMT engines on GPU/NPU of Android devices?
HI All, I'm trying to build CTranslate2 with oneDNN support enabled which internally uses the Intel GPU. When I execute the benchmark tests, I don't see any output, when gpu...
Hello, I'm currently using Ctranslate2 to quickly obtain attention scores for a downstream task, and I am interested obtaining the optimal "alignment_heads" and "alignment_layer" for HuggingFace Seq2Seq converted models. However,...
Hi, I was running Flan-t5 XXL with ctranslate2 and observed completely different results when run with tensor parallelism. **To convert from HF to CT2:** ```bash ct2-transformers-converter --model google/flan-t5-xxl --output_dir flan-t5-xxl...
``` #0 0x00007bc0622c6554 in std::_Rb_tree_increment(std::_Rb_tree_node_base const*) () from /lib/x86_64-linux-gnu/libstdc++.so.6 No symbol table info available. #1 0x00007bc05573e59a in cub::CachingDeviceAllocator::DeviceAllocate(int, void**, unsigned long, CUstream_st*) () from /home/.local/lib/python3.10/site-packages/ctranslate2.libs/libctranslate2.so.4 No symbol table info available....
Currently, the Pypi hub of Ctranslate2 exceeds 20GB, the release 4.3.0 is missing for python 3.8 and 3.9. Request is sent to pypi to upgrade the limit. Some points to...
Kind of a niche feature, I think. The idea behind the change is that CUDA compiler should be separate from CXX compiler, simply because then the user will be able...
Does ctranslate2 to have plans to support the recently released small, medium, and vision. I've tried running them with transformers (on Windows) and can't get past a Triton and compiler...
Obviously the encoding time is almost a non-issue, only when you are working on very small audio chunks it could even hope to shave off some meaningful total percentage of...
When installing `ctranslate2` as part of a python app, it's dependency library `ctranslate2.libs/libctranslate2-81fc0d88.so.4.2.1` has executable stack flag. This breaks the build process if we try to build the `snap` package...
