Increased test coverage for lora/qlora

[ved1beta]

Increased test coverage for lora/qlora

Motivation and Context

#3145

Test for lora includes

TEST config validation : test_qlora_4bit_validation test_basic_configuration_validation

TEST Freeze : test_parameter_freezing_gradient_behavior test_parameter_freezing_with_real_lora_layer test_multiple_adapters_active_adapter_selection test_quantization_state_handling test_parameter_shapes_consistency test_parameter_active_adapters_enabled test_parameter_freezing_no_adapters test_parameter_freezing_adapters_merged test_parameter_freezing_adapters_disabled

TEST Merge/UnMergeing: test_basic_lora_merge_unmerge_cycle test_merge_weight_calculation_accuracy test_cli_do_merge_functionality test_quantized_model_merge_compatibility

NOT included QLoRA 4-bit + LoRA Integration (already exist) Target Module Selection Validation (SKIPPING too big can be done in a separate PR)

Summary by CodeRabbit

• Tests
  • Added unit tests for LoRA/QLoRA configuration validation, including 4-bit requirements and error conditions.
  • Introduced comprehensive tests for LoRA parameter freezing, covering disabled/merged adapters, no-adapter scenarios, active adapters, shape/dtype consistency, quantization state, multiple adapters, and gradient behavior.
  • Added tests for adapter merge/unmerge ensuring weight accuracy and reversibility.
  • Covered CLI-driven merge flows, including save/load interactions.
  • Validated compatibility with quantized models and memory-efficient merge using CPU offload.

[coderabbitai[bot]]

[!IMPORTANT]

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

📝 Walkthrough

Walkthrough

Adds three new test modules under tests/utils/lora covering configuration validation, parameter freezing behavior, and adapter merge/unmerge operations for LoRA/QLoRA, including mocked components and conditional integration tests.

Changes

Cohort / File(s)	Summary
LoRA/QLoRA config validation tests tests/utils/lora/test_config_validation_lora.py	Adds tests for validate_config with valid/invalid LoRA and QLoRA setups, asserting adapter selection, 4-bit requirements, and error messages.
LoRA parameter freezing tests tests/utils/lora/test_freeze_lora.py	Introduces unit and conditional integration tests for freezing behavior: adapters enabled/disabled/merged, parameter presence (A/B/s), shapes/dtypes, quantization propagation, multiple adapters selection, and gradient checks (when PEFT available).
LoRA merge/unmerge tests tests/utils/lora/test_merge_lora.py	Adds tests for merge/unmerge correctness using mocked models: delta computation ((B @ A) * scaling), weight restoration, CLI merge flow with stubs, quantized compatibility, and CPU offload path (lora_on_cpu).

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 74.07% which is insufficient. The required threshold is 80.00%.	You can run @coderabbitai generate docstrings to improve docstring coverage.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title Check	✅ Passed	The title "Increased test coverage for lora/qlora" clearly and concisely summarizes the primary change: adding tests for LoRA and QLoRA behavior (config validation, freezing, merge/unmerge). It is specific, short, and directly reflects the changes in the PR without extraneous detail. A reviewer scanning history will understand the main intent from the title.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[codecov[bot]]

Codecov Report

:white_check_mark: All modified and coverable lines are covered by tests.

:loudspeaker: Thoughts on this report? Let us know!

[winglian]

@ved1beta see nitpicks from coderabbit, then should be good to go