Patrick Gartung

I inadvertently removed the python file used to produce that report when I was cleaning up the repo. ``` /data/cmsbld/jenkins/workspace/ib-run-pr-profiling/profiling/Analyze_tool/compare_cpu_txt.py --old CMSSW_13_0_X_2023-01-12-2300_RES_CPU_step3.txt --new RES_CPU_step3.txt /data/cmsbld/jenkins/workspace/ib-run-pr-profiling/cms-bot/pr_testing/run-pr-profiling.sh: line 48: /data/cmsbld/jenkins/workspace/ib-run-pr-profiling/profiling/Analyze_tool/compare_cpu_txt.py: No such...

I reran the Jenkins job and the file was created. I am not sure where it was uploaded. You may need to rerun the pull request tests to get the...

It appears that the files for the IB could not be found or didn't get downloaded the second time I ran it. ``` + get_jenkins_artifacts igprof/CMSSW_13_0_X_2023-01-13-1100/el8_amd64_gcc11/profiling/11834.21/RES_CPU_step3.txt CMSSW_13_0_X_2023-01-13-1100_RES_CPU_step3.txt + local SRC=igprof/CMSSW_13_0_X_2023-01-13-1100/el8_amd64_gcc11/profiling/11834.21/RES_CPU_step3.txt...

The igprof cpu profile for the 01-13-1100 IB crashed so there was no 'CMSSW_13_0_X_2023-01-13-1100_RES_CPU_step3.txt' file prodiuced. https://cmssdt.cern.ch/SDT/jenkins-artifacts/igprof/CMSSW_13_0_X_2023-01-13-1100/el8_amd64_gcc11/profiling/11834.21/step3_igprof_cpu.log

It worked the second time https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-a3f7c8/30032/profiling/11834.21/RES_CPU_compare_11834.21.txt

@clacaputo It appears to be fixed.

The problem is that the cmsRun process itself gets a segfault while being profiled by Igprof. The same segfault might happen when being profiled with Vtune.

Usually the FastTimerService job completes and the average per module is contained in the raw json file if the resources piechart is not readable.

The IgprofService dumps the profile after the first, middle and next to last event. The first one might not have enough data to be meaningful.

@mandrenguyen Can you point me to a PR so I can look at the logs.