Michael Feil

icon indicating a website link michaelfeil.eu

icon company @basetenlabs icon location San Francisco icon location @basetenlabs | building infinity | M.Sc. ML/Robotics@TU-Munich

Results 125 comments of Michael Feil

Continuous batching

#1317

Stop words (sequence)

Sounds useful for me. Also Instuction Tuned Models have often Stop sequences instead of a stop token. Just for clarification, this is on a sequence of valid tokens in the...

Support for CodeT5pConfig

@ferboz For this reason trying to add "CodeGen" (v1) the decoder part of codet5p-2B https://github.com/OpenNMT/CTranslate2/pull/1230 The smaller codet5p-770M and 220M are fully supported.

Exception: Download failed for English → Spanish

Maybe related to: https://github.com/LibreTranslate/LibreTranslate/issues/441

LLamaTokenizer with `use_fast=True` / and `use_fast=False` causing memory leak when used with multiprocessing / `dataset.map(num_proc)`

No, not stale!

LLamaTokenizer with `use_fast=True` / and `use_fast=False` causing memory leak when used with multiprocessing / `dataset.map(num_proc)`

Not stale

LLamaTokenizer with `use_fast=True` / and `use_fast=False` causing memory leak when used with multiprocessing / `dataset.map(num_proc)`

Not stale

Add Azure and AWS

Approved

Script for reproducing the Benchmark

@OlivierDehaene I compared TEI on batch inference. Takeaways: - embedding 2048 random sentences is (Tokenization / Queing is fast -> *7.10x* speedup on my system -> great work. - TEI...

Script for reproducing the Benchmark

Thanks for the details. Probably this precision I would suggest adding a PR that validates, that e.g. `Cosine-Distance` of `TEI` vs `sentence-transformers` is `>0.999` - compare to https://github.com/qdrant/fastembed/pull/54/files