foundry icon indicating copy to clipboard operation
foundry copied to clipboard

fix(forge test): ensure --rerun only runs failing tests from correct contracts

Open dghelm opened this issue 6 months ago • 0 comments

Motivation

Fixes issue #10693 where forge test --rerun incorrectly runs passing tests that have the same name as failing tests in different contracts.

Solution

Implemented qualified test matching with contract-aware filtering while maintaining full backward compatibility:

Key Changes

  1. Extended TestFilter trait with matches_qualified_test(contract_name, test_name) method for context-aware filtering
  2. Enhanced FilterArgs system with qualified_failures field to store specific contract/test combinations
  3. Smart failure persistence that intelligently chooses between qualified format (ContractName_testName) and legacy format (testName1|testName2) based on whether multiple contracts are involved
  4. Updated test runner logic in is_matching_test_in_context to use qualified matching when appropriate

Architecture

  • Intelligent format detection: Uses qualified format only when test names are ambiguous across multiple contracts
  • Graceful fallback: Maintains legacy behavior for single-contract scenarios
  • Conservative approach: Minimizes format changes to preserve existing workflows

Files Modified

  • crates/common/src/traits.rs - Extended TestFilter trait
  • crates/forge/src/cmd/test/filter.rs - Added qualified matching logic and fixed test name format consistency
  • crates/forge/src/cmd/test/mod.rs - Enhanced failure persistence and loading with intelligent format selection
  • crates/forge/src/multi_runner.rs - Updated test matching logic to use qualified context
  • crates/forge/tests/cli/test_cmd.rs - Added comprehensive regression test

Backward Compatibility

All existing functionality is preserved:

  • Single-contract scenarios continue using legacy format
  • Existing test failure files work unchanged
  • All CLI arguments and behavior remain the same

Side Effects and Considerations

Potential Side Effects

  1. Test failure file format changes: Files may now contain qualified names (ContractTest_testName) in multi-contract scenarios
  2. Filter matching behavior: Slightly different matching logic when qualified failures are present
  3. Performance impact: Minimal additional processing to detect ambiguous test names

Mitigation

  • Backward compatibility preserved: Legacy format still used for single-contract scenarios
  • Graceful fallback: System falls back to legacy behavior when qualified parsing fails
  • Conservative approach: Only uses qualified format when absolutely necessary

PR Checklist

  • [x] Added Tests
    • Added should_replay_failures_only_correct_contract regression test
    • Verified existing should_replay_failures_only test continues to pass
  • [x] Added Documentation
    • Comprehensive inline code comments explaining the qualified matching logic
    • Updated function documentation for new matches_qualified_test method
  • [x] Breaking changes

This contribution was made to explore the viability of bug-fixing work via Claude Code assistance, as prompted by this Tweet. If it's a mess, please don't spend too much time helping me get it over the line.

dghelm avatar Jun 10 '25 19:06 dghelm