Gael Varoquaux
Gael Varoquaux
I have been thinking about this problem. Maybe it can be fixed by special casing the KeyboardInterrupt in the part of the code that retrieves the results to kill the...
> I am afraid that there is little we can do about it. Would it be possible to write in the stored file whether or not there are numpy objects...
> Walking the graph structure of such complex objects is actually quite costly. But once the storage is finished, it's done, so after storing, we should know.
> Actually thinking about it, maybe the cleanest thing to do is to add a > use_joblib_pickling argument to Memory, which should be True by default. That would work, but...
> it is Awesome, so if we know that there are no numpy arrays in it, we can use the fast loading route, right?
> Yes but it can be costly to determine whether the arbitrary object contains a > numpy array or not (if we use the python implementation of pickle). At write...
> What do you mean ? In our NumpyPickler, we can change a flag (on the pickler for instance) when we hit an array.
> So if I understand correctly, at the end of the dump, this flag will be written > at the beginning of the pickle file ? I think that we...