Fix bugs in LLaVA NeXT model with padding and CPU initialization options
[!IMPORTANT]
TheUpdate branchbutton must only be pressed in very rare occassions. An outdated branch is never blocking the merge of a PR. Please reach out to the automation team before pressing that button.
What does this PR do ?
This PR addresses bug fixes in the LLaVA NeXT model, specifically adding padding functionality and introducing a use_cpu_init argument to export nemo model to HF format without requiring a GPU or avoiding OOM error
Collection: vlm/llava_next
Changelog
- Added pad_to_multiple_of to support training without packed sequences.
- Introduced use_cpu_init argument to enable HFExporter functionality without requiring a GPU.
Usage
- You can potentially add a usage example below
# Add a code snippet demonstrating how to use this
GitHub Actions CI
The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.
The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR. To re-run CI remove and add the label again. To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".
Before your PR is "Ready for review"
Pre checks:
- [ ] Make sure you read and followed Contributor guidelines
- [ ] Did you write any new necessary tests?
- [ ] Did you add or update any necessary documentation?
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
- [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
PR Type:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation
If you haven't finished some of the above items you can still open "Draft" PR.
Who can review?
@abhinavg4
Anyone in the community is free to review the PR once the checks have passed. Contributor guidelines contains specific people who can review PRs to various areas.
Additional Information
- Related to # (issue)