electric-sql
Windowed incremental live queries
First of all, thanks for PGlite! I'm currently trying it with Electric Sync, and it works incredibly well.
When I looked at the code of the live extension, I saw that live.incrementalQuery already returns a refresh function. However, unlike live.query, it doesn't accept any parameters. So I guess I have to choose between:
- Using
live.query, which can be used in windowed mode but is less efficient with updates (especially in React). - Using
live.incrementalQuery, which is more efficient on updates but doesn't support windowing out of the box.
For now I'll just add offsets and limits to my queries manually, and replace the existing incrementalQuery with a new one whenever they change. But it would be amazing if the incrementalQuery would support windowed mode as well. This would be helpful for infinite scroll use cases to expand, move, or limit the current window in an efficient, incremental way.
Cheers and thanks again!
It's worth carefully benchmarking the difference in performance been the two versions for each use case. The overhead of calculating the diff inside the live.incrementalQuery can sometimes outweigh the gains you get from it. It's ideally suited to either very wide tables (lots of columns) where only a small number get updates, or very long result sets where you want to load all of them but only a small number change.
The main use case for the offset/limit on the live.query is for windowed scrolling where you are already limiting the number of rows that are being returned, and want the query to execute very fast. I would expect a live.query to probably be better than a live.incrementalQuery in this case.
Thanks for the details, @samwillis! Having built some infinite scroll UIs in the past, I usually opt for larger windows (about 150 items) and only load additional items when the user gets somewhat close to the edge of a window, because this often results in a pretty stable and performant UX in my experience, especially if each item has a lot of data. In this case, I'm planning to do the same, because each item contains data of multiple joined tables, and I expect individual rows or cells to change very infrequently. I figured this would be the perfect use case for windowed incremental live queries because I worry that updating about 150 items in the UI every time one of them has changed would result in quite a lot of unnecessary work.
@gian-reto thank you for your kind words and for the input!
I'll close this issue now, but please feel free to open a new one if you have any other feedback - or praises ;)
