atomic-server
atomic-server copied to clipboard
Consider parallelizing sorting in `Db::query`
When Db::query
is performed, most of the time we'll get an index hit. These are fast. However, if the index does not exist yet (i.e. because this is the first time that specific QueryFilter
is used), it has to be built during this query. That involves running a tpf
query and sorting it. The tpf
query is pretty fast, as it's indexed, but the sorting could take quite a while.
This is where it could help to have some degree of paralellization. Not sure how this would work, though, but let's use this issue to gather some ideas.
@AlexMikhalev may have some ideas already :)
If we decide we stick with tokio, tokio runtime already has work-stealing workers: https://docs.rs/tokio/latest/tokio/attr.main.html#multi-threaded-runtime so we can split work per CPU core. Also, we may consider merge sort for parallelized sorting.