[data grid] server side aggregation and grouping workarounds

[mike-paretos]

The problem in depth

Our application has successfully leveraged DataGridPremium for data aggregations and groupings on datasets of up to 100,000 rows.

Recently, we have encountered the necessity to extend it to handle significantly larger datasets, exceeding 3 million rows. To provide the same level of data manipulation flexibility, we require server-side processing for grouping and aggregation functions. This approach needs to support pagination to ensure optimal performance and usability.

Upon reviewing the current documentation and features, it appears that DataGridPremium does not have support for server-side aggregation and grouping.

Therefore, I have the following questions:

1. Are there alternative methods or best practices within DataGridPremium to facilitate server-side grouping and aggregation for large datasets, possibly through server-side paginated Tree Data (back-end would return structure with the metadata for path & aggregation for each row and total) or other mechanisms?
2. Could you suggest any workarounds or third-party integrations that may complement DataGridPremium's capabilities in this context?
3. Is there an estimated timeline for when out-of-the-box support for these functionalities might be introduced in DataGridPremium?

Your environment

`npx @mui/envinfo`

  Don't forget to mention which browser you used.
  Output from `npx @mui/envinfo` goes here.

Search keywords: server side aggregation, back-end aggregation, server side tree data pagination Order ID: 82474

[michelengelen]

@MBilalShafi could you help out here please? This somewhat falls into your recent explorations in server-side data provisioning. Thanks a lot! 🙇🏼

[MBilalShafi]

Hey @mike-paretos, thank you for reaching out.

Are there alternative methods or best practices within DataGridPremium to facilitate server-side grouping and aggregation for large datasets, possibly through server-side paginated Tree Data (back-end would return structure with the metadata for path & aggregation for each row and total) or other mechanisms?

I don't think there's some performant ootb way to achieve this use-case. You should be able to do some workaround by controlling the models and wrapping the Grid component in some parent component which handles data fetching and passing along to the grid. But since you'll be frequently updating rows prop, possibly columns prop too, and other controlled models very frequently, there is a possibility of the unnecessary re-renders, resulting in computations which may impact performance of the application.

I have tried a similar use-case on very simple dummy data in this stackblitz example. Since the row grouping and tree data only support pagination for the root level, we have to make sure the data being passed after server side pagination only refers to the currently active page. Could you relate this with your use-case somehow, or by pagination you mean lazy-loading of children rows?

Is there an estimated timeline for when out-of-the-box support for these functionalities might be introduced in DataGridPremium?

Providing a performant, out of the box solution for server-side data fetching is a set of enhancements we are actively working on atm, I can not provide an exact timeline, but it is currently one of the top priority areas for the team. We have a pre-documentation for the concept already live here. Please feel free to hop in and provide some feedback if you want.

[mike-paretos]

Thank you @MBilalShafi, we will give it a try!

[github-actions[bot]]

:warning: This issue has been closed. If you have a similar problem, please open a new issue and provide details about your specific problem. If you can provide additional information related to this topic that could help future readers, please feel free to leave a comment.

How did we do @mike-paretos? Your experience with our support team matters to us. If you have a moment, please share your thoughts through our brief survey.