scalene
scalene copied to clipboard
plasma-umass
profile CLI tools
Is your feature request related to a problem? Please describe.
Right now the way to run scalene is to assume some input .py script to do scale my_prog.py and get profiles. But this doesn't quite work for a scenario where I'm using a CLI tool that happens to mostly be a Java codebase that spawns a python process to do an ML model inference in pytorch/serve
Describe the solution you'd like
scalene torchserve --start and get all the profiles I need
Describe alternatives you've considered A colleague has recommended I use https://github.com/benfred/py-spy but I find the experience in scalene to be far superior for my users.
Additional context Github issue describing some of my problems https://github.com/pytorch/serve/issues/1504#issuecomment-1081294549
For whatever reason @profile assumes an --on argument for which I couldn't find documentation for. `scalene.start_profile() doesn't play too well with the existing torchserve codebase and makes our existing metric collector not find its process ID. Logs and reasoning are all in the github issue
The reason I'm so interested in scalene in the first place is that a lot of my users struggle to prepare their models for inference and basically reinvent a profiling and benchmarking suite. I think scalene is great and would be happy to recommend it more in docs or even enable gated behind some environment variable by default.
Hi, I have a similar use-case. My python project is packaged into a CLI tool (say knock) using poetry. I run my program as:
$ knock [OPTIONS] COMMAND [ARGS]
I can profile the tool like:
$ scalene src/main.py --- [OPTIONS] COMMAND [ARGS]
Is it possible to profile via the command knock itself?
$ scalene knock --- [OPTIONS] COMMAND [ARGS]
or have something like --start and --stop sessions?
Just ended up with this issue too.
The workaround is to pass to scalene the full path to the console script.
You can get the full path by using the which command (for example, which knock).
Do you know how to use it with torchrun?
https://pytorch.org/docs/stable/elastic/run.html
Since I've opened this I've found a profiling workflow that works great for me with a combination of perf, ncu and the pytorch profiler so no longer experimenting with new profilers as I was in the past