vLLM

A high-throughput and memory-efficient inference and serving engine for LLMs

vllm-project

Cost-efficient and pluggable Infrastructure components for GenAI inference

vllm-project

A framework for efficient model inference with omni-modality models

vllm-project

System Level Intelligent Router for Mixture-of-Models

vllm-project