vllm topic

Private Open AI on Kubernetes

substratusai

[EMNLP 2024 & AAAI 2026] A powerful toolkit for compressing large models including LLM, VLM, and video generation models.

ModelTC

A large-scale simulation framework for LLM inference

microsoft

Effortlessly run LLM backends, APIs, frontends, and services with one command.

av

大模型推理框架加速，让 LLM 飞起来

zRzRzRzRzRzRzR

Evaluate your LLM's response with Prometheus and GPT4 💯

prometheus-eval

The goal of ramalama is to make working with AI boring.

containers

☸️ Easy, advanced inference platform for large language models on Kubernetes. 🌟 Star to support our work!

InftyAI

Fully-featured, beautiful web interface for vLLM - built with NextJS.

yoziru

【深度学习模型部署框架】支持tf/torch/trt/trtllm/vllm以及更多nn框架，支持dynamic batching、streaming模式，支持python/c++双语言，可限制，可拓展，高性能。帮助用户快速地将模型部署到线上，并通过http/rpc接...

NetEase-Media