text-generation-inference topic

🏋️ A unified multi-backend utility for benchmarking Transformers, Timm, PEFT, Diffusers and Sentence-Transformers with full support of Optimum's hardware optimizations & quantization schemes.

huggingface

Effortlessly run LLM backends, APIs, frontends, and services with one command.

av

☸️ Easy, advanced inference platform for large language models on Kubernetes

InftyAI