continuous-eval
continuous-eval copied to clipboard
Data-Driven Evaluation for LLM-Powered Applications
# Pull Request Description ## Summary This pull request introduces a new SQL AST comparison metric to the `continuous-eval` repository. The new metric, `SQLASTSimilarity`, compares SQL queries using Abstract Syntax...
_This change addresses [review comments](https://github.com/relari-ai/continuous-eval/pull/59#pullrequestreview-2061835523) left by [@yisz](https://github.com/yisz) on PR #59: **Add SQL Metrics Implementation**_ > :warning: We couldn't build/test your project to verify our changes. [Add a Dockerfile](https://docs.ellipsis.dev/build) to...
Issue: reduce the need for inference providers with continuous-eval. Advantage: Allow users to use multiple LLMs with fallbacks/caching without building the core infra.