oneTBB icon indicating copy to clipboard operation
oneTBB copied to clipboard

callback to be evoked on thread creation

Open bubnikv opened this issue 3 years ago • 3 comments

We would enjoy a callback to be called on thread creation. Namely, we would love to

  1. Set a thread name to be shown by the debugger.
  2. Change locales to "C".

We thought we had a workaround by pre-spawning the same number of threads as physical threads of the system and setting the thread names and locales on those pre-spawned threads https://github.com/prusa3d/PrusaSlicer/blob/dc3cf1f7b8db2a49888731fefe9129760ec7a85f/src/libslic3r/Thread.cpp#L192

however the workaround does not work reliably. Quite rarely, TBB decides to spawn additional threads. We have no idea why and we found out first after a major release of our software. Due to the locales not being set to "C", we had a major issue.

Thus we hope that TBB could be extended with a callback that would allow us to do our chores on each spawned thread.

For the reference, these are the chores that we do perform when pre-spawning our threads. Maybe others would also benefit of being able to configure TBB to name the threads for debugging and to set locales.

We set the thread name https://github.com/prusa3d/PrusaSlicer/blob/dc3cf1f7b8db2a49888731fefe9129760ec7a85f/src/libslic3r/Thread.cpp#L236 https://github.com/prusa3d/PrusaSlicer/blob/dc3cf1f7b8db2a49888731fefe9129760ec7a85f/src/libslic3r/Thread.cpp#L111 https://github.com/prusa3d/PrusaSlicer/blob/dc3cf1f7b8db2a49888731fefe9129760ec7a85f/src/libslic3r/Thread.cpp#L145 https://github.com/prusa3d/PrusaSlicer/blob/dc3cf1f7b8db2a49888731fefe9129760ec7a85f/src/libslic3r/Thread.cpp#L111

and we set locales in system dependent way: https://github.com/prusa3d/PrusaSlicer/blob/dc3cf1f7b8db2a49888731fefe9129760ec7a85f/src/libslic3r/Thread.cpp#L238

bubnikv avatar May 16 '22 13:05 bubnikv

I believe that task_scheduler_observer should cover this. Using it, you can observe and execute your code when either a thread joins TBB scheduling globally or enters a specific task arena.

vossmjp avatar May 23 '22 19:05 vossmjp

@vossmjp Thanks Mike for your hint. Frankly I am quite lost. According to the manual, observer's on_scheduler_entry() callback is called when

  1. for each thread that starts participating in oneTBB work
  2. enters an arena after the observation is enabled

However I don't quite understand what 1) means. We made some experiments, thus it seems to us that when starting a parallel_for loop, 1) happens once for every thread participating in the parallel_for loop. Thus your hint seems to be useful to us to solve the locales switching issue we have at one parallel pipeline.

However the observer is called at every parallel_for or parallel_pipeline, which seems quite expensive to me. We would love to have a callback which is called only once per newly created thread.

bubnikv avatar May 30 '22 07:05 bubnikv

The on_scheduler_entry callback is called each time a thread starts participating in executing tasks and on_scheduler_exit is called when it leaves. While TBB keeps threads in the arena for a short time looking for additional work, if there is a gap in work availability, the worker threads will leave the arena. So, if you are seeing the function called repeatedly at each parallel_for, the threads must be leaving the arena and reentering for each parallel_for. A possible work-around is to use a thread-local variable to track whether a particular thread has already executed your setup code and not re-execute the code if it has.

vossmjp avatar Jun 07 '22 17:06 vossmjp

@bubnikv is this issue still relevant? Could you please respond?

isaevil avatar Oct 05 '22 13:10 isaevil

@isaevil @vossmjp Thanks for your support. For now, we solved the issue with the callback you have suggested. Similarly, we used the same callback for https://github.com/oneapi-src/oneTBB/issues/896.

We still believe an "on create" callback would be valuable, however the on_scheduler_entry() you have suggested seems to fulfill our needs suffuciently.

bubnikv avatar Oct 05 '22 14:10 bubnikv

@bubnikv If these issues are resolved could you please close them? If you feel that "on create" callback is still useful feature - any patches are welcomed :)

isaevil avatar Oct 05 '22 14:10 isaevil