Use nanosleep on Linux in atomic_backoff when spinning for a long time (oneTBB)

[Lastique]

This is an updated version of https://github.com/oneapi-src/oneTBB/pull/106, which is rebased on top of oneTBB 2021.1.1 code base.

This allows to significantly reduce CPU system time while spinning, as nanosleep deschedules the thread and allows other threads to progress. Unlike sched_yield, nanosleep allows the thread to migrate to other CPUs, as well as other threads to migrate to the current CPU, which is what allows for more total progress.

See https://github.com/oneapi-src/oneTBB/issues/105. On the test presented there, this patch offers to reduce CPU system time by 25-30%. On a real world application that test is based on this patch offers ~10% improvement.

[pavelkumbrasev]

Now we are trying to use "mutex" and "rw_mutex" in cases where high contention might cause slowdowns. They are implemented using an adaptive approach, it guarantees that the thread that cannot acquire the lock spins before blocking(for example on cond var). You can check current wait implementation in waitable_atomic. If you still observe slowdowns on atomic_backoff waits please let as know.

[anton-potapov]

@Lastique, do you believe this patch still makes sense ?

[Lastique]

We're still using it in our production builds of TBB. Though we haven't tried the latest developments in the TBB code base as there haven't been a release yet.