pgsync icon indicating copy to clipboard operation
pgsync copied to clipboard

Existing delay in sync

Open hookenful opened this issue 2 years ago • 3 comments

PGSync version: 2.2.1

Postgres version: 14.1

Elasticsearch version: 7.11.2

Python version: 3.10-slim

Problem Description:

Our friends-frontends reporting a delay between sync, for e.g they delete some record (only one) and after they re-request data, it still persists until force reload. We tried a setTimeout delay between deletion and re-requesting and it's values were: 250-800 msec and between 350-800 all was okay.

So our goal is to lower that delay to at maximum 200msec. Can you please share a knowledge about environment values, which can take effect to lower that delay?

We have decent configuration machines, so we can try some higher settings, but need to check.

Thanks in advance for any help!

hookenful avatar Jul 11 '22 10:07 hookenful

I think the delay depends on a large number of factors including how your data is laid out. You could try the async mode to see if it gets you the numbers you are looking for. You do this by setting the env var:

USE_ASYNC=True

toluaina avatar Jul 11 '22 14:07 toluaina

I think the delay depends on a large number of factors including how your data is laid out.

What do you mean by 'how data is laid out'? We are working with .NET ORM - EF Core. Nothing special here.

USE_ASYNC=True

Thanks, I'll try this. And what about other parameters? I saw poll interval, threads count.. Are they vital for my case?

hookenful avatar Jul 11 '22 20:07 hookenful

  • You can reduce POLL_TIMEOUT which might increase cpu usage
  • increasing NTHREADS_POLLDB will help. Increasing threads can be counter productive. You need to find the optimal value

toluaina avatar Jul 18 '22 12:07 toluaina