llm-as-a-judge topic

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.

Agenta-AI

CodeUltraFeedback: aligning large language models to coding preferences (TOSEM 2025)

martin-wey

This is the repo for the survey of Bias and Fairness in IR with LLMs.

KID-22

Code and data for "Timo: Towards Better Temporal Reasoning for Language Models" (COLM 2024)

zhaochen0110

Evaluate your LLM's response with Prometheus and GPT4 💯

prometheus-eval

Code and data for Koo et al's ACL 2024 paper "Benchmarking Cognitive Biases in Large Language Models as Evaluators"

minnesotanlp

Official implementation for "MJ-Bench: Is Your Multimodal Reward Model Really a Good Judge for Text-to-Image Generation?"

MJ-Bench

👩‍⚖️ Coding Agent-as-a-Judge

metauto-ai

Dingo: A Comprehensive AI Data Quality Evaluation Tool

MigoXLab

Inference-time scaling for LLMs-as-a-judge.

haizelabs