robrix icon indicating copy to clipboard operation
robrix copied to clipboard

Split background workers into Matrix-specific and general queues

Open ZhangHanDong opened this issue 2 months ago • 1 comments

Right now MatrixRequest mixes both Matrix timeline work and non-Matrix tasks such as SearchRoomMembers, all running on the sliding-sync Tokio runtime. Per the earlier review, we should move toward having two explicit worker loops: one dedicated to Matrix sync/timeline requests and another for non-Matrix/background tasks (similar to the existing TSP worker). Track the architectural refactor to introduce separate queues/runtimes and reroute SearchRoomMembers (and any other general jobs) onto the general worker in a dedicated PR.

ZhangHanDong avatar Oct 22 '25 08:10 ZhangHanDong

Thanks Alex, I'm generally in favor of this approach. However, I think instead of splitting it between "Matrix" vs. "non-Matrix" requests, we should split it into "CPU-bound" tasks (that run on native background threads) vs. "IO-bound" tasks (that run in an async background task). For example, a task like SearchRoomMembers is fully CPU-bound, so running it in an async context would be bad since it likely has no yield points.

I'm not an expert in managing tokio runtimes, but I don't think it's ideal have multiple different tokio runtimes, since they won't be able to coordinate across each other. I'd assume it's best to have a single tokio runtime that has purview over all async tasks in the whole system.

kevinaboos avatar Oct 22 '25 17:10 kevinaboos