OpenChatKit icon indicating copy to clipboard operation
OpenChatKit copied to clipboard

Published 20 hours ago • verified publisher icon togethercomputer

prevent prompt tensors from accumulating in GPU

Open MichaMucha opened this issue 1 year ago • 0 comments

this could help with CUDA OOM errors especially on consumer grade hardware.

prompt and output tensors will be erased from VRAM

MichaMucha avatar Apr 02 '23 22:04 MichaMucha