llm-ops topic

Run any open-source LLMs, such as Llama 3.1, Gemma, as OpenAI compatible API endpoint in the cloud.

bentoml

AIConfig is a config-based framework to build generative AI applications.

lastmile-ai

Friendli: the fastest serving engine for generative AI

friendliai

An end-to-end LLM reference implementation providing a Q&A interface for Airflow and Astronomer

astronomer

Python SDK for running evaluations on LLM generated responses

athina-ai

RAG (Retrieval Augmented Generation) Framework for building modular, open source applications for production by TrueFoundry

truefoundry

A new DSL and server for AI agents and multi-step tasks

julep-ai

The collaborative spreadsheet for AI. Chain cells into powerful pipelines, experiment with prompts and models, and evaluate LLM responses in real-time. Work together seamlessly to build and iterate on...

EmbeddedLLM

cluster/scheduler health monitoring for GPU jobs on k8s

Trainy-ai