facets icon indicating copy to clipboard operation
facets copied to clipboard

Published 20 hours ago • verified publisher icon PAIR-code

why facets dive consume so much memory

Open wuzhipeng2014 opened this issue 8 years ago • 3 comments

I loaded a 80MB file and found that my computer was in a hight load, so i have to close the jupter notebook. i also want to know what's the maximum amount of data the facets dive can handle?

wuzhipeng2014 avatar Jul 24 '17 07:07 wuzhipeng2014

image

image1

wuzhipeng2014 avatar Jul 24 '17 07:07 wuzhipeng2014

Hi wuzhipeng2014,

Thanks for taking the time to post this. There are a couple of things that could be going on. What kind of file are you loading?

jimbojw avatar Jul 24 '17 14:07 jimbojw

I experienced similar issue. My dataframe is about 3.7KB, but the output of Dive is about 15M. Is it because the JSON representation of the data is not efficient enough? My dataframe contains only 3000 instances (150 columns). If it includes more than 10000 instances my chrome just dies.

Would be the protobuf string be more efficient?

qimingj avatar Jul 31 '17 18:07 qimingj