hdf5storage
hdf5storage copied to clipboard
Add marshaller/s for scipy.sparse matrices
Due to fixing Issue #49 , this can now be done without adding scipy as a package dependency/requirement.
Sparse matrices are useful and are additionally one of the ndarray/matrix types built into the MAT v7.3 format without resorting to classes that this package still does not support.
I'm trying to load a struct array that contains a bunch of sparse matrices. It's taking a very long time, probably because it is converting each of them to a dense matrix? Would it be worth it to try to dig into this and open a PR or is this a tricky enhancement? Thanks for your effort on this package - it looks real nice!
I can see it being tricky given that the Marshallers are in a bit of flux being that they are mid overhaul, but then again it may just be tricky to me and not a new set of eyes (a PR would be very welcome). I can look more in depth this week. Do know that a Marshaller for sparse matrices is one of the main priorities in order to work with the vast majority of v7.3 MAT files.
Any update on this? I would still love to have this feature... (I just re-encountered this problem today for a new dataset)
I haven't worked on it in a while. I actually ran into a major problem that I have been trying to think about that is in the way. That is, str(some_type)
produces different output on different python version. For str(float)
, it is producing "<class 'float'>"
on 3.5.3 and "<type 'float'>"
. This makes the current way that having marshallers can lazy load other modules brittle. I tried digging around more in the inspect module but still haven't found anything that works that well yet.