ColossalAI icon indicating copy to clipboard operation
ColossalAI copied to clipboard

Published 20 hours ago • verified publisher icon hpcaitech

Optimize the memory usage

Open heshuguo opened this issue 1 year ago • 4 comments

I am trying to fine-tune the Llama 13B model using Colossalai. However, the memory usage is quite high, exceeding 270B, and causing an OOM error directly. Is there any way to optimize the memory usage?

heshuguo avatar Apr 17 '23 11:04 heshuguo

Hi, you may test out our strategies such as GeminiDDP as illustrated in the examples.

JThh avatar Apr 18 '23 05:04 JThh

Hi, you may test out our strategies such as GeminiDDP as illustrated in the examples.

This is the training method we used. The memory usage is too large, unusable. Any suggestion?

heshuguo avatar Apr 21 '23 06:04 heshuguo

What are your environment settings?

JThh avatar Apr 21 '23 10:04 JThh

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

What are your environment settings?

Issues-translate-bot avatar Apr 21 '23 10:04 Issues-translate-bot