Nadav Shmayovits

[Misc] Logits processor plugins

11

This pull request adds support for Logits processor plugins. This makes implementing custom Logits processors very easy, and eliminates the need to change vLLM directly to implement it. For example...

[Feature]: Tensor Parallelism with non divisble amount of attention heads

7

### 🚀 The feature, motivation and pitch I am trying to run a 70B model on a node with 3XA100-80Gi. 2XA100-80Gi does not contain enough VRAM to run the model,...