memit icon indicating copy to clipboard operation
memit copied to clipboard

CUDA out of memory

Open simba0626 opened this issue 1 year ago • 0 comments

Hi authors,

MEMIT is an interesting work. When I run: python -m experiments.evaluate --alg_name=MEMIT --model_name=EleutherAI/gpt-j-6B --hparams_fname=EleutherAI_gpt-j-6B.json --num_edits=10 --use_cache

There is error about CUDA out of memory: File "\memit-main\memit\memit_main.py", line 97, in weights_copy = {k: v.detach().clone() for k, v in weights.items()} RuntimeError: CUDA out of memory. Tried to allocate 256.00 MiB (GPU 0; 24.00 GiB total capacity; 23.15 GiB already allocated; 0 bytes free; 23.16 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

My local is 24GB 3090 GPU. Would you help me how to run the MEMIT code ? How to revise the configure file (EleutherAI_gpt-j-6B.json) , in order to reduce memory?

Thank you very much. MEMIT is very nice.

simba0626 avatar May 08 '23 17:05 simba0626