marimo
marimo copied to clipboard
Dataframe viewer's column chart are aggregated on the backend
Description
Currently we send up to 20k rows of data to the frontend to aggregate columns summaries. This aggregation happens in vega-lite and can cause memory issues and slowdowns.
Suggested solution
We would like to:
- move this aggregation to the backend. either using
vega-fusion(if installed) ornarwhals.Expr.hist(if/when implemented) - lazily make these requests based on content visibility on the frontend
Alternative
No response
Additional context
This could help: https://github.com/narwhals-dev/narwhals/issues/1561