datoviz
datoviz copied to clipboard
datoviz
Support for aggregations
This looks like a cool project! I'm aware of vispy, pyqtgraph and other precursors, I still keep wondering if there will be support for data aggregation. What I mean by that is that instead of all points in a data series, you plot a band with summaries like average, min/max and variance. This greatly reduces the amount of polygons required to draw. Say for example a 1 minute view of a 10 kHz signal would take 600000 points. But one can reduce the amount of data required to draw by plotting a min/max band and an average line for this data. When zooming in and out, one would switch between the two modes.
My own implementation of this idea lives here: https://github.com/windelbouwman/lognplot
Maybe this could be of some use. I use this, and this feature is especially useful for live streaming of data.
Great idea! I would definitely like to see something like that in Datoviz. The simplest would be to compute aggregations on the CPU. If one were to use the GPU, it could be done separately in a compute shader, for example, or even in the fragment shader, although the latter might not make much sense with non-dynamic data.
Where would something like this be implemented? As a custom visual? Note that I created some sort of tree with intermediate nodes. A nice touch would be to persist some nodes to disk, to reduce memory footprint.
I would say the first step would be to implement a proof of concept as a standalone example, using either C or Python. Then we could think about how to wrap this into a custom visual.