plotly.js icon indicating copy to clipboard operation
plotly.js copied to clipboard

Performance Issues with large data sets since 2.x.x

Open sean-mcl opened this issue 4 years ago • 11 comments

Hey, we found some performance issues regarding large data sets.

Here is an example with 100k datapoints. When we use the plotly-2.2.0.min.js, the whole plot freezes and has a very bad performance. When we use the plotly-1.58.4.min.js, everything works fine.

LayTec-AG/Plotly.Blazor#99

sean-mcl avatar Jun 30 '21 15:06 sean-mcl

Please note that the default hovermode in v1 is x and in v2 is closest. So with v2 you could simply set layout.hovermode to 'x' for this graph.

archmoj avatar Jun 30 '21 21:06 archmoj

Let's investigate this one a bit more... 100k points shouldn't slow down this much in any hovermode, right?

nicolaskruchten avatar Jun 30 '21 23:06 nicolaskruchten

The plots here are much faster and have hovermode=closest (and are rendered with 2.2.0) https://plotly.com/python/webgl-vs-svg/ ... what's the difference? Is closest hovermode much more efficient with a gaussian distribution of points in x/y or something?

nicolaskruchten avatar Jun 30 '21 23:06 nicolaskruchten

Just tested it with sin data here. The performance impact is huge! When I use linear data, its almost unusable.

sean-mcl avatar Jul 01 '21 12:07 sean-mcl

I can confirm it is definitely a hovermode problem. I was generating a WebGL scatter of circa 300,000 points where there was small Y variability over X, and the perf was awful If I generate a synthetic scatter with a larger distribution over Y the perf improves. Changing it to a full continuous Y distribution the perf is just fine. Coming back to my original scatter, if I set HoverMode=false, no perf issues HoverMode.x or HoverMode=xUnified = not too bad Hovermode.y, HoverMode.yUnified or Hovermode.Closest = awful perf So as @nicolaskruchten indicates it is related to the distribution of data for a given axis. Anything over about 50,000 points starts to lag if there is no distribution of data Running v2.6.3 here

ordinaryorange avatar Mar 15 '22 10:03 ordinaryorange

Has anyone found any solutions to handling the issue? Setting the hovermode to 'x' has improved the hover performance slightly, however as soon as I hover over an area on the screen with many zeros, my app crashes. I'm using react-plotly 2.5.1. This only occurs once I plot more than 100k points. I'm already using scattergl as well.

JudeShamsi avatar Aug 21 '22 00:08 JudeShamsi

+1 I went back to matplotlib because I need to be able to look at large data sets. Currently attempting a dataframe with 105,983 rows, and anything over 4 columns is just awful.

As others have stated, forcing webgl doesn't help (I think it's default for plotly express anyway). Turning hovermode off works pretty well but drawing the canvas is still a chore. Plus, hovermode is half the reason I use plotly.

If this is truly unfixable due to JS limitations, then I think plotly should feature an automatic subsampling option to reduce the number of points shown.

hobob0t avatar Mar 20 '23 15:03 hobob0t