Slow performance when loading large JSONs and toggling collapse/expand

[gurupras]

I think the entire JSON tree is being re-rendered every time one item is toggled. This does not scale well for large JSONs. Additionally, the component struggles to display a JSON of size ~21MB :slightly_frowning_face:

One thing I did notice was that most of the time is being spent in jsonFlatten and/or flatData computation. Is there anything we can do to improve both of these? Given that jsonFlatten is not touching the DOM in any way, can we parallelize the work using WebWorkers? I'm happy to work on implementing it, but I don't fully understand the codebase yet.

Branch: 1.x

https://user-images.githubusercontent.com/2933207/133361231-db2438e0-bf5d-45a3-b5e9-741c3f60b16d.mp4

https://user-images.githubusercontent.com/2933207/133361242-346f64c8-b8c7-4ff1-bdff-bf3ebc738ed4.mp4

[leezng]

Have you turned on the virtual list mode?

[gurupras]

Yes, sorry, I've configured it as follows:

 <vue-json-pretty :data="filteredData"
    ref="results"
    style="height: inherit; max-height: inherit;"
    :deep="expandDepth"
    :deep-collapse-children="true"
    :show-double-quotes="false"
    :virtual="true"
    :virtual-lines="100"
    :select-on-click-node="false"
    :highlight-selected-node="false"/>

[fredrivett]

I'm also seeing something similar, some JSON loads are taking over 10 seconds and are blocking the rest of the UI from loading.

I've not looked into this but what are your thoughts on @gurupras investigation about jsonFlatten and/or flatData being the bottleneck @leezng? Intrigued if you have any ideas about what the issue might be and where we might be able to improve things here.

One option in my mind is if deep isn't infinity then to prioritise calculating the collapsed view, display that but read only and then swap it out to the interactive version once that has finished loading.

If we could find a way to increase efficiency that would be ideal.

[leezng]

Sorry, I have been busy recently and did not reply in time. I found this problem. In the demo, when the array reaches 10,000, it took 5s to calculate. My thought is whether it is possible to use the ability on the worker to calculate it?

There is also a feature here, which is the number of rows of current data, so we have to ensure that we can calculate the number of each row of data.

[simake2017]

the same to me

[leezng]

In the latest version(1.9.0 & 2.2.0), the efficiency of JSON parsing has been improved by 40 times compared with the previous version, maybe you can have a try.