keepsake
keepsake copied to clipboard
checkpoint.open() shouldn't read entire file into memory
Problem
Model files can be huge, but checkpoint.open()
currently works by returning io.BytesIO(f.read())
. This is a bodge since it allows us to immediately delete the temporarily downloaded experiment files.
Solution
open()
should stream files instead of downloading to a temporary file.
Blocked by
This is currently not possible due to our jsonrpc shared library interface. See #344