verl
verl copied to clipboard
[megatron] fix: Pass optimizer config betas to Megatron optimizer config
What does this PR do?
This PR fixes a bug where the value of the beta parameters for the Adam optimizer actor_rollout_ref.(actor|critic).optim.betas are ignored when using the Megatron backend.
Checklist Before Starting
- [X] Search for similar PRs. Paste at least one query link here: beta; optimizer
- [X] Format the PR title as
[{modules}] {type}: {description}(This will be checked by the CI)-
{modules}includefsdp,megatron,sglang,vllm,rollout,trainer,ci,training_utils,recipe,hardware,deployment,ray,worker,single_controller,misc,perf,model,algo,env,tool,ckpt,doc,data - If this PR involves multiple modules, separate them with
,like[megatron, fsdp, doc] -
{type}is infeat,fix,refactor,chore,test - If this PR breaks any API (CLI arguments, config, function signature, etc.), add
[BREAKING]to the beginning of the title. - Example:
[BREAKING][fsdp, megatron] feat: dynamic batching
-
Test
No changes
API and Usage Example
No changes
Design & Code Changes
No design choices or changes
Checklist Before Submitting
[!IMPORTANT] Please check all the following items before requesting a review, otherwise the reviewer might deprioritize this PR for review.
- [X] Read the Contribute Guide.
- [X] Apply pre-commit checks:
pre-commit install && pre-commit run --all-files --show-diff-on-failure --color=always - [X] Add / Update the documentation.
- [X] Add unit or end-to-end test(s) to the CI workflow to cover all the code. If not feasible, explain why: ...
- [ ] Once your PR is ready for CI, send a message in the
ci-requestchannel in theverlSlack workspace. (If not accessible, please try the Feishu group (飞书群).)