model-serving topic
Deep-Learning-in-Production
In this repository, I will share some useful notes and references about deploying deep learning-based models in production.
kserve
Standardized Serverless ML Inference Platform on Kubernetes
BentoML
The easiest way to serve AI apps and models - Build reliable Inference APIs, LLM apps, Multi-model chains, RAG service, and much more!
chitra
A multi-functional library for full-stack Deep Learning. Simplifies Model Building, API development, and Model Deployment.
Yatai
Model Deployment at Scale on Kubernetes 🦄️
pinferencia
Python + Inference - Model Deployment library in Python. Simplest model inference server ever.
gallery
BentoML Example Projects 🎨
kafka-with-akka-streams-kafka-streams-tutorial
Code samples for the Lightbend tutorial on writing microservices with Akka Streams, Kafka Streams, and Kafka
FATE-Serving
A scalable, high-performance serving system for federated learning models
hopsworks
Hopsworks - Data-Intensive AI platform with a Feature Store