transformers add low_cpu_mem_usage option in run_clm.py example which will benefit…

add low_cpu_mem_usage option in run_clm.py example which will benefit…

Open sywangyi opened this issue 1 year ago • 2 comments

… LLM loading

add low_cpu_mem_usage option in run clm example, set the option True will help reduce peak memory and loading time in LLM finetune and inference.

Mar 21 '23 08:03 sywangyi

@sgugger please help review

Mar 21 '23 08:03 sywangyi

The documentation is not available anymore as the PR was closed or merged.

Mar 21 '23 08:03 HuggingFaceDocBuilderDev