Strange gaps in trace when a thread is scheduling tasks

[rasolca]

Using the default scheduler blank gaps appear in the trace when one of the threads is scheduling the tasks. The gaps disappear immediately after the scheduling task finished.

I tested the shared priority queue scheduler (@msimberg advice) and blanks disappeared

Note: the execution time is only slightly affected (2-3% faster with shared scheduler)

[msimberg]

Possibly related: #511. The shared_priority_queue_scheduler schedules new tasks by default on the current worker thread. While not necessarily always better, it does lead to different contention on the queues, and could explain part of the differences.

[msimberg]

This was my minimal change to test scheduling on the current thread in the default scheduler: https://github.com/pika-org/pika/compare/main...schedule-current-thread. It didn't seem to make a difference when I tested it but the algorithms in DLA-Future might have changed enough by then that it makes a difference now.