verl icon indicating copy to clipboard operation
verl copied to clipboard

[bugfix] fix megatron model merger

Open ShareLer opened this issue 6 months ago • 0 comments

Checklist Before Starting

  • [x] Search for similar PR(s).

What does this PR do?

Fix megatron model merger.

High-Level Design

Demonstrate the high-level design if this PR is complex.

Specific Changes

  • Fix get rank method to support just TP.
  • Fix state_dict keys after convert.
  • Add mla/moe convert support.

API

Demonstrate how the API changes if any.

Usage Example

Provide usage example(s) for easier usage.

# Add code snippet or script demonstrating how to use this 

Test

Test with Qwen3-8B and Qwen2.5-7B.

Additional Info.

  • Issue Number: Fixes issue #1757
  • Training: [Note which backend this PR will affect: FSDP, Megatron, both, or none]
  • Inference: [Note which backend this PR will affect: vLLM, SGLang, both, or none]

Checklist Before Submitting

  • [ ] Read the Contribute Guide.
  • [ ] Apply pre-commit checks.
  • [ ] Add [BREAKING] to the PR title if it breaks any API.
  • [ ] Update the documentation about your changes in the docs.
  • [ ] Add CI test(s) if necessary.

ShareLer avatar May 30 '25 08:05 ShareLer