ModelCenter icon indicating copy to clipboard operation
ModelCenter copied to clipboard

[FEATURE] support model.from_pretrained without the need of init distributed

Open Jiaxin-Wen opened this issue 2 years ago • 0 comments

from model_center.layer import CPM1
CPM1.from_pretrained("cpm1-large")

currently could not work since the function check_web_and_convert_path calls bmt.rank() or bmt.print_rank() to prevent every process downloads the checkpoint in a multi-gpu scenario.

While ModelCenter is mainly designed to support distributed training, I think it is still important to support such a common code snippet.

Jiaxin-Wen avatar Jun 11 '22 14:06 Jiaxin-Wen