ModelCenter
ModelCenter copied to clipboard
[FEATURE] support model.from_pretrained without the need of init distributed
from model_center.layer import CPM1
CPM1.from_pretrained("cpm1-large")
currently could not work since the function check_web_and_convert_path
calls bmt.rank()
or bmt.print_rank()
to prevent every process downloads the checkpoint in a multi-gpu scenario.
While ModelCenter is mainly designed to support distributed training, I think it is still important to support such a common code snippet.