RFC: Refactor metrics module - Create metrics.core for better separation of abstractions

[jjmachan]

📋 Proposal

I'm proposing to refactor the metrics module to create a clearer separation between core abstractions and specific metric implementations.

🎯 Motivation

Currently, the src/ragas/metrics/ directory mixes core abstractions with specific implementations, making it harder to:

• Understand which files are foundational vs. implementations
• Navigate the codebase for new contributors
• Maintain clear architectural boundaries

📁 Current Structure

The metrics directory contains ~40 files with two categories:

Core abstractions (files without '_' prefix):

• base.py - Base metric classes and interfaces
• result.py - MetricResult class
• decorator.py - Metric decorator factory
• validators.py - Validation utilities
• discrete.py - DiscreteMetric class
• llm_based.py - LLM-based metric primitives
• numeric.py - NumericMetric class
• ranking.py - RankingMetric class
• utils.py - Utility functions
• quoted_spans.py - Quoted span utilities

Specific implementations (files with '_' prefix):

• 30+ files like _answer_correctness.py, _faithfulness.py, etc.

🚀 Proposed Solution

Create a metrics.core submodule:

src/ragas/metrics/
├── core/
│   ├── __init__.py
│   ├── base.py
│   ├── result.py
│   ├── decorator.py
│   ├── validators.py
│   ├── discrete.py
│   ├── llm_based.py
│   ├── numeric.py
│   ├── ranking.py
│   ├── utils.py
│   └── quoted_spans.py
├── _answer_correctness.py
├── _faithfulness.py
├── ... (other implementations)
└── __init__.py

✅ Benefits

1. Clearer separation: Explicit distinction between abstractions and implementations
2. Better organization: Core functionality grouped logically
3. Improved discoverability: Developers can easily find foundational components
4. Reduced cognitive load: Clear hierarchy between core and specific metrics

⚠️ Considerations

1. Breaking changes: Existing imports would need updating
2. Backward compatibility: We should maintain compatibility with:

   # Old way (should still work)
   from ragas.metrics.base import Metric
   from ragas.metrics.result import MetricResult
   
   # New way
   from ragas.metrics.core import Metric, MetricResult
   

💭 Questions for Discussion

1. Is this separation valuable enough to justify the refactoring effort?
2. Should we use a different name than "core"? (alternatives: "base", "abstractions", "primitives")
3. How should we handle the migration? Phase it over multiple releases?
4. Are there other files that should be considered "core" that I've missed?

🔄 Migration Strategy (if approved)

1. Create the metrics.core module with all core files
2. Update internal imports throughout the codebase
3. Add backward compatibility imports in metrics.__init__.py
4. Deprecation warnings for old import paths (remove in future major version)
5. Update all documentation and examples

Looking forward to feedback from the maintainers and community!

cc: @jjmachan