Sebastian Schoennenbeck

Installing eli5 in a blank environment installs jinja2 in version 3.1.1 which leads to the following error: ``` >>> import eli5 Traceback (most recent call last): File "", line 1,...

Is there a reason why the output (both for the column and the table mask) has 3 channels? The training is basically doing binary classification (which also means the mask...

Fix/async chat serving

6

This fixes [Issue 2683](https://github.com/vllm-project/vllm/issues/2683) Changes: - Added `OpenAIServingChat._load_chat_template_async` which waits for the tokenizer to become available. - According changes to `OpenAIServingChat.__init__` to accomodate this (analogous implementation to `ServingEngine.__init__`. - Added...

Does `acquire` wait longer than needed?

4

In the `acquire`-method the future waiting for drips has a timeout of `1 / self._rate_per_sec * amount` which waits "enough seconds" for "amount" to drip out of the bucket so...

Faster acquire

5

Acquire now only waits the minimum amount of time needed for enough drip to occur. See: https://github.com/mjpieters/aiolimiter/issues/217

[Feature]: Unblock LLM while handling long sequences / Handling multiple prefills at the same time

8

### 🚀 The feature, motivation and pitch ## Motivation If an engine is currently handling a single long sequence in the prefill stage any other incoming sequence has to wait...