llm-serving topic
runbooks
Finetune LLMs on K8s by using Runbooks
lorax
Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs
SwiftInfer
Efficient AI Inference & Serving
sugarcane-ai.github.io
rtp-llm
RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.
LLM-FineTuning-Large-Language-Models
LLM (Large Language Model) FineTuning
ray-educational-materials
This is suite of the hands-on training materials that shows how to scale CV, NLP, time-series forecasting workloads with Ray.
ray_vllm_inference
A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving.
llms-in-prod-workshop-2023
Deploy and Scale LLM-based applications