Initial mempool sync may take a lot of time

[romanz]

All the above is strongly suggesting to me that the rpc.sync() call on this line is blocking for too long, probably because the mempool.sync() call on this line is blocking. This results in JSON-RPC client calls not being handled because the electrs code relies on syncing being paused before handling incoming client requests.

@conduition Good catch, thanks! It seems that the initial mempool sync takes quite a while... I'll open a new issue.

Originally posted by @romanz in https://github.com/romanz/electrs/issues/961#issuecomment-1843453480

[conduition]

I'd be happy to take a look at fixing this.

@romanz could you please clarify, is blocking on mempool sync is a desired behavior? My suggestion would be to block until the initial blockchain and mempool synchronization is complete, but then after that, handle incoming client requests asynchronously.

Alternatively we could block until the initial blockchain sync is done, but then do mempool scanning asynchronously so that it doesn't interfere with processing of client requests.

[Kixunil]

I think that doing chain first and the asynchronously mempool is the best option. But still mempool sync needs to be optimized - otherwise the clients won't see the transactions for a long time. It's OK to have a bit of lag since the network behaves this way already anyway, just not crazy long times.

[conduition]

PR open to provide mempool scanning in a separate thread: https://github.com/romanz/electrs/pull/970

[romanz]

#979 should also help with this issue.