llm-evaluation-framework topic

Test your prompts, agents, and RAGs. Red teaming, pentesting, and vulnerability scanning for LLMs. Compare performance of GPT, Claude, Gemini, Llama, and more. Simple declarative configs with command...

promptfoo

The LLM Evaluation Framework

confident-ai

Python SDK for experimenting, testing, evaluating & monitoring LLM-powered applications - Parea AI (YC S23)

parea-ai

Agentic LLM Vulnerability Scanner / AI red teaming kit 🧪

msoedov

The official evaluation suite and dynamic data release for MixEval.

Psycoy

[ACL'24] A Knowledge-grounded Interactive Evaluation Framework for Large Language Models

zhuohaoyu

FM-Leaderboard-er allows you to create leaderboard to find the best LLM/prompt for your own business use case based on your data, task, prompts

aws-samples

Realign is a testing and simulation framework for AI applications.

honeyhiveai