ChatLM-mini-Chinese icon indicating copy to clipboard operation
ChatLM-mini-Chinese copied to clipboard

Published 20 hours ago • verified publisher icon charent

tokenizer训练OOM 。内存60G

Open musexiaoluo opened this issue 6 months ago • 1 comments

调用 train_my_huggingface_wiki_tokenizer 方法时，OOM 。 超过了 60G 的内存

image

musexiaoluo avatar Aug 28 '24 07:08 musexiaoluo