keepsake icon indicating copy to clipboard operation
keepsake copied to clipboard

checkpoint.open() shouldn't read entire file into memory

Open andreasjansson opened this issue 4 years ago • 0 comments

Problem

Model files can be huge, but checkpoint.open() currently works by returning io.BytesIO(f.read()). This is a bodge since it allows us to immediately delete the temporarily downloaded experiment files.

Solution

open() should stream files instead of downloading to a temporary file.

Blocked by

This is currently not possible due to our jsonrpc shared library interface. See #344

andreasjansson avatar Nov 12 '20 22:11 andreasjansson