cmssw icon indicating copy to clipboard operation
cmssw copied to clipboard

Added ModuleAllocMonitor Service

Open Dr15Jones opened this issue 1 year ago • 9 comments

PR description:

This uses the AllocMonitor system to monitor the memory usage done by each module as well as the source and memory released at end of event.

PR validation:

Code compiles and unit tests pass.

Dr15Jones avatar May 23 '24 14:05 Dr15Jones

cms-bot internal usage

cmsbuild avatar May 23 '24 14:05 cmsbuild

please test

Dr15Jones avatar May 23 '24 14:05 Dr15Jones

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-45025/40317

  • This PR adds an extra 40KB to repository

cmsbuild avatar May 23 '24 14:05 cmsbuild

A new Pull Request was created by @Dr15Jones for master.

It involves the following packages:

  • PerfTools/AllocMonitor (core)

@makortel, @Dr15Jones, @smuzaffar can you please review it and eventually sign? Thanks. @makortel, @felicepantaleo this is something you requested to watch as well. @antoniovilela, @rappoccio, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

cmsbuild avatar May 23 '24 14:05 cmsbuild

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b1b04d/39489/summary.html COMMIT: e6148d552e6200a2f3d22019daf7597875b1491b CMSSW: CMSSW_14_1_X_2024-05-22-2300/el8_amd64_gcc12 User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/45025/39489/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • You potentially removed 1 lines from the logs
  • Reco comparison results: 13 differences found in the comparisons
  • DQMHistoTests: Total files compared: 48
  • DQMHistoTests: Total histograms compared: 3338862
  • DQMHistoTests: Total failures: 9
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3338833
  • DQMHistoTests: Total skipped: 20
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 47 files compared)
  • Checked 202 log files, 165 edm output root files, 48 DQM output files
  • TriggerResults: no differences found

cmsbuild avatar May 23 '24 17:05 cmsbuild

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-45025/40326

  • This PR adds an extra 40KB to repository

cmsbuild avatar May 23 '24 21:05 cmsbuild

Pull request #45025 was updated. @Dr15Jones, @smuzaffar, @cmsbuild, @makortel can you please check and sign again.

cmsbuild avatar May 23 '24 21:05 cmsbuild

please test

Dr15Jones avatar May 23 '24 21:05 Dr15Jones

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b1b04d/39498/summary.html COMMIT: 0002c7701a3436494f0aca0ecc6e7ecadcd923d3 CMSSW: CMSSW_14_1_X_2024-05-23-1100/el8_amd64_gcc12 User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/45025/39498/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 12 differences found in the comparisons
  • DQMHistoTests: Total files compared: 48
  • DQMHistoTests: Total histograms compared: 3338862
  • DQMHistoTests: Total failures: 6
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3338836
  • DQMHistoTests: Total skipped: 20
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 47 files compared)
  • Checked 202 log files, 165 edm output root files, 48 DQM output files
  • TriggerResults: no differences found

cmsbuild avatar May 24 '24 00:05 cmsbuild

please test

let's see if this is affected by the OS change as well.

Dr15Jones avatar Jun 07 '24 16:06 Dr15Jones

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b1b04d/39789/summary.html COMMIT: 0002c7701a3436494f0aca0ecc6e7ecadcd923d3 CMSSW: CMSSW_14_1_X_2024-06-07-1100/el8_amd64_gcc12 User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/45025/39789/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • You potentially added 1 lines to the logs
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 48
  • DQMHistoTests: Total histograms compared: 3338974
  • DQMHistoTests: Total failures: 0
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3338954
  • DQMHistoTests: Total skipped: 20
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 47 files compared)
  • Checked 202 log files, 165 edm output root files, 48 DQM output files
  • TriggerResults: no differences found

cmsbuild avatar Jun 07 '24 18:06 cmsbuild

+1

Dr15Jones avatar Jun 11 '24 18:06 Dr15Jones

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @rappoccio, @antoniovilela, @sextonkennedy (and backports should be raised in the release meeting by the corresponding L2)

cmsbuild avatar Jun 11 '24 18:06 cmsbuild

+1

rappoccio avatar Jun 12 '24 01:06 rappoccio