Liger-Kernel
Liger-Kernel copied to clipboard
test: Add test for ref_input parameter in fused linear preference
This PR adds a test for the ref_input parameter that was introduced in #467.
Changes
- Add
test_ref_input.pyto verify theref_inputparameter works correctly inLigerFusedLinearPreferenceBase - Test ensures that:
- Policy model outputs (chosen_logps, rejected_logps) are identical when using
ref_inputvs not using it - Final loss and aux outputs differ when using
ref_inputvs usinginput_chunkfor reference model
- Policy model outputs (chosen_logps, rejected_logps) are identical when using
- Uses same parametrization as other tests for consistency
Testing
The test verifies that:
- When
ref_inputis not provided, the reference model usesinput_chunk - When
ref_inputis provided, the reference model uses it instead ofinput_chunk - The policy model outputs remain unchanged regardless of
ref_input - The final loss differs when using different inputs for the reference model
Follows up on #467 which added ref_input parameter support.