pyblish-base
pyblish-base copied to clipboard
pyblish
basic profiling support
Goal
It would be nice to have mode, that would generate profiling data for pyblish process and plugins. Producing lot of timing and other data, that can be later used for visualizing and debugging whole session.
Motivation
My idea is to capture whole context after each plugin is done as snapshots along with it's timing, memory consumtion, etc. to json. Then to have simple application processing this json file, drawing all data to canvas, so we'll be able to vizualize, what is going on with all data during publishing. This could be useful when debugging behaviour of pipelines where lot of plugins are doing many things.
This implements #348
This still have some caveats. Getting memory information on windows is somehow cumbersome without additional external dependencies (such as psutils. Problem with psutils is that it needs developer version of python on linux (gcc and python header), complicating installation somehow.
Also missing tool to process profiling info yet.
Thanks for this.
Re psutil; it's kind of the de-facto standard for doing these kinds of OS queries; how about we depend on it as an optional dependency? That is, if it's available, the profiling will be that more thorough, and without it you get the basics. That way, users on Windows can pip install psutil and those on Linux could e.g. apt-get install python-psutil or whatever means are available, if they are interested in that kind of information.
If there is a pure-Python equivalent, that would of course be best. But in my experience it just isn't possible. :(
I've changed it to use optional psutil module. Pure python isn't there - only resource module but that is available only on linux and darwin systems.
Hi @antirotor, revisiting this long-forgotten issue and would like to merge it. It's a good feature! There are just a few notes left before a first 1.0 of the feature, if you're up for it it would be good to see those though.
Hi @antirotor, revisiting this long-forgotten issue and would like to merge it. It's a good feature! There are just a few notes left before a first 1.0 of the feature, if you're up for it it would be good to see those though.
Sure thing, sorry for this zombie-state. I had more urgent things to do but it is still on my mind. I'll try to take care of it asap.