llama-stack-apps icon indicating copy to clipboard operation
llama-stack-apps copied to clipboard

Published 20 hours ago • verified publisher icon meta-llama

what is the minimum requirements to run in local?

Open Amrosx opened this issue 4 months ago • 5 comments 

  torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 20.00 MiB. GPU 0 has a total capacity of 7.69 GiB of which 13.94 MiB is free. Including non-PyTorch memory, this process has 7.64 GiB memory in use. Of the allocated memory 7.34 GiB is allocated by PyTorch, and 121.77 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. ```

Amrosx avatar Oct 01 '24 07:10 Amrosx