llm icon indicating copy to clipboard operation
llm copied to clipboard
verified publisher icon rustformers

Free tensors from RAM if they are offloaded to an Accelerator

Open LLukas22 opened this issue 2 years ago • 0 comments

Right now the data of a tensor isn't freed if it is offloaded to a GPU. We should fix that to enable users to run bigger models which are split between CPU and GPU.

LLukas22 avatar Jul 27 '23 10:07 LLukas22