PoseGPT icon indicating copy to clipboard operation
PoseGPT copied to clipboard

Published 20 hours ago • verified publisher icon yfeng95

python main_chat.py <> CUDA out of memory

Open affromero opened this issue 8 months ago • 9 comments

Hello,

  • When running the main_chat.py with bf16 precision I get an OOM error. I am using a 24GB GPU. Is this expected? Can't find info about the minimal gpu requirement.
  • If I enable fp16 precision, I get AttributeError: 'LlamaAttention' object has no attribute 'rope_theta'. I think this is related to deepspeed, which was not listed in the requirements, so should I install a specific version?

Thanks!

affromero avatar Jun 21 '24 08:06 affromero