tablib icon indicating copy to clipboard operation
tablib copied to clipboard

Published 20 hours ago • verified publisher icon jazzband

Memory usage of Dataset

Open mingdongt opened this issue 7 years ago • 2 comments

We have to build and load the dataset in memory every time, which seems to occupy a lot of memory if data is really huge. Is there any way of improving this?

mingdongt avatar Mar 09 '17 15:03 mingdongt

No, for now tablib is not intended to large datasets.

I agree it is an issue. I'd like to support it for a subset of tablib API, mostly format conversions.

#207 is somewhat related to this issue.

iurisilvio avatar Mar 10 '17 15:03 iurisilvio

With the respect for tablib, the streaming feature is pending. For now, I would suggest giving it a go with pyexcel. Here is the tutorial on streaming data and the information on the choice of plugins.

chfw avatar Aug 08 '18 12:08 chfw