clcache icon indicating copy to clipboard operation
clcache copied to clipboard

ThreadPoolExecutor / concurrent.futures.as_completed prevent cProfile from providing actual info

Open Artalus opened this issue 5 years ago • 0 comments

Basically, what I get after combining pstats:

This doesn't help me if I want to profile processSingleSource() which is called inside scheduleJobs. What I actually want to see:

Notice how instead of scheduleJobs->as_completed->wait I get actual detalization of functions called inside processSingleSource.

I managed it by replacing ThreadPoolExecutor construct with simple processSingleSource() call, asserting that len(sourceFiles) == len(objectFiles) == 1. In my use case (CMake + Ninja) there is always 1 source file per 1 compiler call, so using thread pool becomes a waste of syscalls, since it will always create 1 thread to do the job and return immediately after this thread is done.

Artalus avatar Nov 15 '19 23:11 Artalus