facets icon indicating copy to clipboard operation
facets copied to clipboard

Can you please elaborate on what is considered "large amount of data" for facets?

Open anshbansal opened this issue 7 years ago • 5 comments
trafficstars

I tried loading a 65.2 MB csv file into facets on Google Chrome on Ubuntu 16.04 LTS following same code as in the demo jupyter notebook. When I started loading the data I had 13 GB RAM available. During the load it used up 4 GB RAM. But it took a few minutes. Google Chrome asked me whether I want to wait or exit the page. I said wait. That happened another time. I said wait. Then the page crashed finally crashed. So I just wanted clarification what is considered "large amount of data" for facets?

anshbansal avatar Jul 05 '18 15:07 anshbansal

Thanks for the feedback. Some questions to help clarify: Which Facets visualization were you trying to use (Overview or Dive)? And at what step did the RAM usage and freeze/crash occur? How many examples (rows) are in the csv file and how many features (columns)? It would help if you would could share the csv if possible.

jameswex avatar Jul 06 '18 13:07 jameswex

In my case I load the data below and got the same problem:

<class 'pandas.core.frame.DataFrame'> RangeIndex: 8461352 entries, 0 to 8461351 Data columns (total 10 columns): UC int64 PCR object ENDERECO object MUNICIPIO object LOCALIDADE object COORDENADA_X float64 COORDENADA_Y float64 ESTADO_FORNECIMENTO object ESTADO_CLIENTE object TRAFO object dtypes: float64(2), int64(1), object(7) memory usage: 645.6+ MB

everardogentil avatar Jul 27 '18 19:07 everardogentil

Can you share a notebook with the problem?

jameswex avatar Jul 27 '18 19:07 jameswex

The code is here: Data Description.ipynb.zip

everardogentil avatar Jul 27 '18 20:07 everardogentil

Could you also provide the csv so I can repro? Thanks

jameswex avatar Aug 06 '18 14:08 jameswex