scalene icon indicating copy to clipboard operation
scalene copied to clipboard

[BUG] Scalene can't generate the final JSON output

Open agemagician opened this issue 2 years ago • 1 comments

Describe the bug I am trying to debug a python script, but it crashes during the profile generation step.

To Reproduce Steps to reproduce the behavior: python -m scalene --cli --reduced-profile --profile-interval 180 --profile-all --outfile scalene-profile.txt --memory-leak-detector myapp.py & or python -m scalene --cli --reduced-profile --profile-interval 180 --profile-all --outfile scalene-profile.txt --memory-leak-detector myapp.py Unfortunately, the code is a company code and I can’t share it.

Expected behavior When I run the profiler, I can see the program started to produce and update the "scalene-profile.txt" every 180 seconds, but after it finishes and started to write the final JSON file it crashes.

Screenshots without using the final "&" at the command line profiler:

("'float' object is not callable",)
Traceback (most recent call last):
  File "/home/c3ebeix/.cache/pypoetry/virtualenvs/clause-comparison-MItsp9HR-py3.9/lib/python3.9/site-packages/scalene/scalene_profiler.py", line 1725, in run_profiler
    exit_status = profiler.profile_code(
  File "/home/c3ebeix/.cache/pypoetry/virtualenvs/clause-comparison-MItsp9HR-py3.9/lib/python3.9/site-packages/scalene/scalene_profiler.py", line 1525, in profile_code
    if not Scalene.output_profile():
  File "/home/c3ebeix/.cache/pypoetry/virtualenvs/clause-comparison-MItsp9HR-py3.9/lib/python3.9/site-packages/scalene/scalene_profiler.py", line 744, in output_profile
    did_output: bool = output.output_profiles(
  File "/home/c3ebeix/.cache/pypoetry/virtualenvs/clause-comparison-MItsp9HR-py3.9/lib/python3.9/site-packages/scalene/scalene_output.py", line 307, in output_profiles
    stats.merge_stats(python_alias_dir)
  File "/home/c3ebeix/.cache/pypoetry/virtualenvs/clause-comparison-MItsp9HR-py3.9/lib/python3.9/site-packages/scalene/scalene_statistics.py", line 357, in merge_stats
    value = unpickler.load()
TypeError: 'float' object is not callable

using the final "&" at the command line profiler:

  File "/home/c3ebeix/anaconda3/lib/python3.9/threading.py", line 973, in _bootstrap_inner
    self.run()
  File "/home/c3ebeix/anaconda3/lib/python3.9/threading.py", line 910, in run
    self._target(*self._args, **self._kwargs)
  File "/home/c3ebeix/.cache/pypoetry/virtualenvs/clause-comparison-MItsp9HR-py3.9/lib/python3.9/site-packages/scalene/scalene_sigqueue.py", line 41, in run
    self.process(*item)
  File "/home/c3ebeix/.cache/pypoetry/virtualenvs/clause-comparison-MItsp9HR-py3.9/lib/python3.9/site-packages/scalene/scalene_profiler.py", line 808, in cpu_sigqueue_processor
    Scalene.output_profile()
  File "/home/c3ebeix/.cache/pypoetry/virtualenvs/clause-comparison-MItsp9HR-py3.9/lib/python3.9/site-packages/scalene/scalene_profiler.py", line 744, in output_profile
    did_output: bool = output.output_profiles(
  File "/home/c3ebeix/.cache/pypoetry/virtualenvs/clause-comparison-MItsp9HR-py3.9/lib/python3.9/site-packages/scalene/scalene_output.py", line 307, in output_profiles
    stats.merge_stats(python_alias_dir)
  File "/home/c3ebeix/.cache/pypoetry/virtualenvs/clause-comparison-MItsp9HR-py3.9/lib/python3.9/site-packages/scalene/scalene_statistics.py", line 357, in merge_stats
    value = unpickler.load()
EOFError: Ran out of input

Desktop (please complete the following information):

  • OS: Ubuntu
  • Browser None
  • Version 20.04

Additional context I have checked the code and the problem seems to occur after the program finish and starts to write the final JSON file. I can also see that the profiler is working correctly because it is generating the intermediate output.

agemagician avatar Mar 02 '22 10:03 agemagician

I have used another machine and now it is working. I am not sure about the reason, but maybe it is related to sudo privilege, and the location that logs are stored.

agemagician avatar Mar 02 '22 17:03 agemagician