Liger-Kernel icon indicating copy to clipboard operation
Liger-Kernel copied to clipboard

test: Add test for ref_input parameter in fused linear preference

Open xingyaoww opened this issue 11 months ago • 0 comments

This PR adds a test for the ref_input parameter that was introduced in #467.

Changes

  • Add test_ref_input.py to verify the ref_input parameter works correctly in LigerFusedLinearPreferenceBase
  • Test ensures that:
    • Policy model outputs (chosen_logps, rejected_logps) are identical when using ref_input vs not using it
    • Final loss and aux outputs differ when using ref_input vs using input_chunk for reference model
  • Uses same parametrization as other tests for consistency

Testing

The test verifies that:

  1. When ref_input is not provided, the reference model uses input_chunk
  2. When ref_input is provided, the reference model uses it instead of input_chunk
  3. The policy model outputs remain unchanged regardless of ref_input
  4. The final loss differs when using different inputs for the reference model

Follows up on #467 which added ref_input parameter support.

xingyaoww avatar Dec 11 '24 21:12 xingyaoww