llm-eval topic

🐢 Open-Source Evaluation & Testing for ML & LLM systems

Giskard-AI

AI Observability & Evaluation

Arize-ai

UpTrain is an open-source unified platform to evaluate and improve Generative AI applications. We provide grades for 20+ preconfigured checks (covering language, code, embedding use-cases), perform ro...

uptrain-ai

Test your prompts, agents, and RAGs. Red teaming, pentesting, and vulnerability scanning for LLMs. Compare performance of GPT, Claude, Gemini, Llama, and more. Simple declarative configs with command...

promptfoo

Python SDK for running evaluations on LLM generated responses

athina-ai

A simple GPT-based evaluation tool for multi-aspect, interpretable assessment of LLMs.

Re-Align

Бенчмарк сравнивает русские аналоги ChatGPT: Saiga, YandexGPT, Gigachat

kuk

Python SDK for experimenting, testing, evaluating & monitoring LLM-powered applications - Parea AI (YC S23)

parea-ai

🎯 Your free LLM evaluation toolkit helps you assess the accuracy of facts, how well it understands context, its tone, and more. This helps you see how good your LLM applications are.

Auto-Playground

An open source library for asynchronous querying of LLM endpoints

alan-turing-institute