DataTable "view" of a larger data set

[multimeric]

The DataTable seems to have performance issues with about a million cells. If we reduce the number of cells in memory at one time to a fraction of that, and then lazily load new rows as the user scrolls down, plus delete old rows, it seems like we might solve these performance issues. Not entirely sure though because I'm not sure how the implementation works.

The simplest implementation of this might be via a pagination mechanism where the user has to implement get_chunk() -> Iterable[Row] and the class handles the loading as the user approaches the end of the current chunk . This would also support viewing external data such as in a database or a remote API and avoid having to load everything into memory.

[github-actions[bot]]

Thank you for your issue. Give us a little time to review it.

PS. You might want to check the FAQ if you haven't done so already.

This is an automated reply, generated by FAQtory

[multimeric]

I note that "Lazy loading API" for DataTable is listed in the Roadmap. I might be able to give implementation a go if one of the maintainers is interested.

[kdkavanagh]

Anyone come up with an implementation/workaround for this? Very interested as well

[max-arnold]

Could this be used as an interim solution? https://github.com/tconbeer/textual-fastdatatable

[willmcgugan]

This may happen, but not in the short term. A community solution would be great, and we would be happy to help.

[github-actions[bot]]

Don't forget to star the repository!

Follow @textualizeio for Textual updates.