Channel handling gets starved if an EventLoop's queue is filling up as fast as its being processed.

[mg-jump]

We have seen lambdas runEvery being hours delayed from when it was supposed to run. It seems if a queue of an EventLoop is being filled at the same rate that it is being ran. It is fairly regular pattern for a EventLoop to schedule future work befor the lamda runs. Basically if doRunInLoopFuncs() never returns channels will never be processed.

Some repos:

    trantor::EventLoopThreadPool pool(20);
    pool.start();
    std::atomic<int64_t> counter(0);
    for (int i = 0; i < 19; i++){
        auto loop = pool.getLoop(i);
        loop->queueInLoop([&]() {
            while (true) {
                pool.getLoop(19)->queueInLoop([&](){
                    counter++;
                });
            }
        });
    }
    auto* loop = pool.getLoop(19);
    loop->runEvery(std::chrono::microseconds(10), [&]() {
        std::cout << "Counter: " << counter.load() << std::endl;
    });
std::this_thread::sleep_for(std::chrono::minutes(2));

    trantor::EventLoopThread loop_thread;
    loop_thread.run();
    auto loop = loop_thread.getLoop();

    auto now = std::chrono::steady_clock::now();
    loop->runEvery(std::chrono::seconds(1), [&]() {
        auto temp = std::chrono::steady_clock::now();
        auto elapsed = std::chrono::duration_cast<std::chrono::milliseconds>(temp - now);
        now = temp;
        std::cout << "Elapsed time: " << elapsed.count() << " ms" << std::endl;
    });

    std::function<void(void)> lammy = [&, loop = loop]() {
        // std::cout << "Running lammy" << std::endl;
        loop->queueInLoop(lammy);
    };

    loop->runInLoop(lammy);
    std::this_thread::sleep_for(std::chrono::minutes(1));