metaseq
metaseq copied to clipboard
facebookresearch
Repo for external large-scale work
To avoid errors such as "RuntimeError: Building apex kernels was requested, but nvcc was not found.", ask that user check availability of minimum hardware requirements. **Patch Description** Describe your changes...
**Patch Description** Describe your changes **Testing steps** Describe how you tested your changes
## 🚀 Feature Request Congratulations on this great work - Scaling Autoregressive Multi-Modal Models:Pretraining and Instruction Tuning. The paper mentions that `metaseq` was used for the model training. Just wondering...
Thanks for releasing the intermediate checkpoints. It is though not clear to me how they were created and hence how to load them properly. To be concrete: How can I...
⚠️ This PR is not intended to be merged directly. Purpose to share features that may be useful for Metaseq ⚠️ # Background One of the main goals for our...
## 🐛 Bug ### To Reproduce My data process is fine ,but when i come to train the data, it broken during some steps.I used colossalai to train this ,an...
**Patch Description** Broadcast jsonl index instead of loading on all ranks, to prevent slowdown by multiple ranks reading the same file **Testing steps** Tested on 64 nodes, loaded in 2...
With the upcoming major release of Aim (https://github.com/aimhubio/aim) Aim SDK will not be backwards compatible. To prevent any failures on metaseq side, setting upper limit for the install Aim version
**Patch Description** Describe your changes **Testing steps** Describe how you tested your changes
