electric icon indicating copy to clipboard operation
electric copied to clipboard

Can we count the rows expected for an initial sync and put it in some metadata?

Open thruflo opened this issue 10 months ago • 1 comments

I appreciate this may not work with caching but …

When developing against a larger initial sync, it’s natural to want to know / display progress.

When we do an initial sync at Electric, we query Postgres. Is there any way of threading the count of rows through to a client? Perhaps the count per frontier.

That would simplify monitoring progress towards a large initial sync.

thruflo avatar Feb 10 '25 10:02 thruflo

I suspect we should be able to include some sort of progress header for initial sync requests, not sure how this would play with compaction though. Do you envision something like an "electric-initial-sync-progress": "1425/2150" where the fraction is the number of rows (including all the ones in the request that carries this header)?

msfstef avatar Feb 10 '25 11:02 msfstef