ChaoyuHuang

i only modify the adpter_v2.py like that devices = 1 to devices = 4 AND batch_size = 64 / devices to batch_size = 128 / devices

nope，i tried reinstall cuda118，but it doesn't work ---Original--- From: "Han ***@***.***> Date: Fri, Jun 23, 2023 04:11 AM To: ***@***.***>; Cc: ***@***.******@***.***>; Subject: Re: [Lightning-AI/lit-llama] use adpater_v2.py fine-tuning llama 13Berror...

> nope，i guess at least 8*A100-80GB needed for fine-tuning llama65B, i have tried 4*A100-80GB just configured the hyperparameters but OOM errors happend

thks for your answer，i‘ve downloaded llama checkpoint from huggyllama 发自我的iPhone ------------------ Original ------------------ From: weilong ***@***.***> Date: Thu,Jun 29,2023 11:43 PM To: Lightning-AI/lit-llama ***@***.***> Cc: ChaoyuHuang ***@***.***>, Comment ***@***.***> Subject:...