ai-evaluation topic

Vivaria is METR's tool for running evaluations and conducting agent elicitation research.

METR

Hallucinations (Confabulations) Document-Based Benchmark for RAG. Includes human-verified questions and answers.

lechmazur

Benchmark evaluating LLMs on their ability to create and resist disinformation. Includes comprehensive testing across major models (Claude, GPT-4, Gemini, Llama, etc.) with standardized evaluation met...

lechmazur

Code scanner to check for issues in prompts and LLM calls

kereva-dev

UQLM: Uncertainty Quantification for Language Models, is a Python package for UQ-based LLM hallucination detection

cvs-health

Example Projects integrated with Future AGI Tech Stack for easy AI development

future-agi

Ranking LLMs on agentic tasks

rungalileo

☑️ A curated list of tools, methods & platforms for evaluating AI reliability in real applications.

Vvkmnn

benchmark and evaluate generative research synthesis

guestrin-lab