metaseq
metaseq copied to clipboard
facebookresearch
Repo for external large-scale work
This would greatly help with readability and debugging!
As noted in #689, convert_to_singleton doesn't produce statedicts with compatible keys (for some unknown reason). Since reshard_mp can do the same job, without the GPU node requirement of convert_to_singleton, we...
## ❓ Questions and Help #### What is your question? I've downloaded the weights for OPT-175B using the URL I got after filling out the Google form. I've also got...
1. Take a 125m pretrained checkpoint. 2. Consolidate the checkpoint using convert_to_singleton.py 3. Try loading the model behind the metaseq API. RuntimeError: Error(s) in loading state_dict for TransformerLanguageModel: Missing key(s)...
Adds confusion to what can be configured / where these values are derived from. Remove this flag for now.
When i use metaseq-api-local to launch bb3_30B model, i meet the error: hydra.errors.MissingConfigException: In 'config': Could not load 'base_config'.
Follow up for #672 - Right now it's not implemented https://github.com/facebookresearch/metaseq/blob/main/metaseq/tasks/streaming_language_modeling.py#L180
## 🐛 Bug The convert_to_singleton.py script fails for the 1.3B checkpoint ### To Reproduce ``` ls 1.3b/ dict.txt gpt2-merges.txt gpt2-vocab.json reshard-model_part-0.pt reshard-model_part-1.pt ``` ``` Loading extension module fused_mix_prec_layer_norm_cuda... 2022-07-19 03:15:10...
**Patch Description** Update CircleCI image **Testing steps** Describe how you tested your changes
