Default ev ThreadPool size in conjunction with `ThreadPool::NextThread` is dangerous if not straight out broken

[itrofimow]

https://github.com/userver-framework/userver/blob/9c371d29fa688f79176eea5faf01721530f84ef3/core/src/engine/ev/thread_pool.cpp#L51-L55

So this function just takes next thread % 2 (where 2 is the default size), seems alright.

Now, engine::io::Socket creates 2 watchers on construction, one for read direction and one for write direction. If the socket is a server connection socket, it's likely to operate way more heavily on a read direction than on the write one: every new request is 2 syscalls in libev (remove from epoll on wakeup, add into epoll on WaitReadable), when writing response through the socket might very well don't even touch libev.

Since there is just a single LISTEN socket in a server and accept-ed socket are created sequentially, for every socket its read watcher goes into ev-thread 0, and write watcher goes into ev-thread 1 (well, until something interfered with the order in ThreadPool), which leads to this:

[Anton3]

We can fix that by using utils::RandRange. It also gets rid of a shared counter, which are notorious for creating contention.

[itrofimow]

That's one way to solve this. Maybe mt19937 is an overkill here and some lighter PRNG would be better

[dyumin]

Actually it seems not only default ev ThreadPool size is broken. In our observation only half of ev threads are working. The other half is 0.0%.

[itrofimow]

Could you try with an odd number of ev-threads? - I think right now some setups don't play nicely with an even number of ev-threads (which we should fix somehow)