Known issues when used with multiprocessing module?

[alxempirical]

In my hands, line_profiler is failing to record information when multiprocessing.Pool.map is used to call the functions I'm targetting. I can probably put together a minimal test case, but I thought first I would check whether this is a known issue.

[rkern]

Yeah, only the main process can be profiled.

[alxempirical]

Thanks. I didn't notice that mentioned in the README.rst. Is it documented anywhere?

[rawrgulmuffins]

An interesting feature would potentially be firing up line_profiler in each of the spawned interpreters and writing each interpreters results out to a file.

[rbnvrw]

If this is possible, this would be really useful.

[krish7919]

+1

[el3ment]

+1

[tburmeister]

Just ran into this myself, but was able to work around this by importing line_profiler in my script and creating a LineProfiler manually, which then gets copied to each child process. Finally, I had each process print out its stats on completion:

s = io.StringIO()
profile.print_stats(stream=s)
print(s.getvalue())

The use of StringIO is a hack to keep the output from the multiple processes from getting interleaved. Having each process output to a separate file should be pretty easy too.

[3droj7]

Just ran into this myself, but was able to work around this by importing line_profiler in my script and creating a LineProfiler manually, which then gets copied to each child process. Finally, I had each process print out its stats on completion:

s = io.StringIO()
profile.print_stats(stream=s)
print(s.getvalue())

The use of StringIO is a hack to keep the output from the multiple processes from getting interleaved. Having each process output to a separate file should be pretty easy too.

Tried it but still didn't work for me.. How did you do it?

[jmlarson1]

This would be a nice feature to have.