Handling of paging/scrolling in large data sets via AJAX/REST

[don-p]

In the "Large Collection" example, this shows how react-virtualized can work with a large list to manage the DOM and render only the visible subset of list items. This works great for the presentation of the data. However, in a real-world application we don't generate 250,000 items in the client/browser, and we don't load a large set like this from a server in one request/response.

So, we need a way to manage subsets of data via scrolling/paging.

What's the support and suggested approach for handling this, in this react-virtualized-tree component? Is there any built-in support to allow us to do something like this?

• Load a root-level list of tree nodes.
• onExpand of a node, load a partial set (a page) of children of the node, a set that overflows the current viewport height.
• On scrolling near the boundary of the viewport, load a subsequent partial set (a page) of children of the node, which are concatenated onto the children[] list.
• Subsequent scrolling continues to load/concatenate more children.

This is basically "Infinite Scrolling". In our application, we have customers that may have millions of children in nodes, so we need an approach like this to handle the presentation of the nested child nodes. Do you have any advice on how to provide this in the component?

Thanks.

[diogofcunha]

Hi @don-p.

At the moment there is no infinite scroll functionality. I can look into adding that in the near future.

The onExpand part I don't see why you can't do that now, is there something I'm missing?

[diogofcunha]

Hi @don-p.

Started a POC on this at https://github.com/diogofcunha/react-virtualized-tree/pull/79. So far the PR has 0 breaking changes.

The next part will be to figure out how to easily check if something a certain row is loaded or not, this can be challenging to do in a performant manner. Need to think about it a little bit more

[don-p]

@diogofcunha

I would suggest looking at the implementations of existing infinite list/ree packages, rather than re-inventing.

I suspect that there is a combination of approaches being used:

• storing node data in a flat list (Array), with a depth property to indicate the node's level in the tree.
• applying a unique index or key property to each node/row, and storing the index only (not the entire node) in an indexed data structure (Object or Map) for easy checking.
• using the rowHeight and scroll data to determine when to load rows and how many.

Some packages you could look at to examine the implementations: https://github.com/venasolutions/react-lazy-paginated-tree https://github.com/storybooks/react-treebeard

don

[diogofcunha]

Hey Don.

That is not the problem I do have in that PR.

Exposing a callback that fires when rows are rendered and a ref to register the children is enough and empowers the library users to do pagination in whatever manner they see fit. The easiest thing they can do is plug in react-virtualized infinite loader and with a few lines of code you will have pagination.

It's important to minimize the amount of state inside this library, knowing if a row is loaded or not should not the handled by this library, if you think about it. Http requests are very prone to slowness and failure, writing the a state object is not a trivial operation as many things can go wrong, when someone tries to handle that inside an open source core 90% of the time it either does a bad job handling errors or is very prone to race conditions that can affect usability.

What I want to figure out is a way to easily empower users to know if a row is loaded or not, all the rest is pretty much done.

[diogofcunha]

I don't think there is any reliable way to know if a row is loaded or not inside this source code, because all apps have their own business logic.

Image that for example you have a websocket connection that signals to the client that a change occurred and you need to reload a certain batch of rows that were previously loaded, at this moment in time any internal state held by the component becomes obsolete and that is when hacks start to emerge