switchy
switchy copied to clipboard
pandas/CSV storage breaks on Py3.5
The py35-pandas test run contains a bunch of failures currently because of:
-
shmarrydoen't support unicode as per the docs onmultiprocessing.RawArrayso if we want to keep the sharednumpyarray stuff it would seem we have to useBytes. -
pandas"encoding" (type coercing?) problems are stemming from pydata/pandas#9712 where when a csv data store is written it keeps thebbytes prefix. This ends up causing problems with round tripping (which is done implicitly when reading the entire contents of aDataStorerin mem + on disk) sincepd.read_csvthen parses thebas part of the data point.
I have a feeling there might be an some way to always hack around this either using str.encode directly or somehow with to_csv although it seems the latter is being battled in the issue above to little avail.
I made the decision to bring in #44 since I don't expect many people to jump onto py3.5 immediately anyway.