imp
imp copied to clipboard
Add Python pickle buffer support
Currently when pickling an IMP C++ object we serialize into a C++ std::string
and then pass that back to Python as a bytes
object, which Python can then pickle. This creates several copies of the underlying data. Consider adding support for pickle protocol version 5 if available (Python >= 3.8, or the backported pickle5
module in Python >= 3.5) as per PEP 574 to store the data in a Python buffer. This should in principle use less memory and be faster, particularly for large objects such as IMP::em::DensityMap
or IMP::Model
. For example, see the work done for NumPy arrays at numpy/numpy#12011.