André Pankraz

Batch Sizes not used anywhere? Out of mem...

2

**Describe the bug** I have some out of mems with 35 GB processes, stanze could be tracked down as reason. **To Reproduce** Steps to reproduce the behavior: 1. Take e.g....

BGE-M3 Sparse

5

Currently i cannot really use the "Sparse mode" of BGE-M3. Even with 8 GB VRAM and small batch sizes I get CUDA out of mem. Why does this mode need...

Support for ELECTRA-based Re-Ranking Cross-encoder

5

### Feature request Currently, BGE rerankers are limited in multilingual support, mainly in English and Chinese. Many XML RoBERTa Cross-Encoders are also English-focused. Could you please add support for other...

Future of Chainlit as fully functional Open Source project

4

Hello, I have more a question than a feature request (or may be a feature request for a clean roadmap). I have seen that the Prompt Playground has now completely...

Playground Template Recognitiion doesn't work anymore in current version 1.0.300

2

See title. When I update 1.0.200 to 300, Playground cannot find template formats anymore.

[Bug]: Docker vLLM 0.9.1 CUDA error: an illegal memory access, sampled_token_ids.tolist()

11

### Your current environment Docker on 4 x A100 SMX. BTW: vLLM 0.8.4 worked stable with same setup. 0.9.01 was already unstable (restarted few time a day), now even more....