cmssw icon indicating copy to clipboard operation
cmssw copied to clipboard

More plots from GE21 for the GEM DQMs, a backport to 12_3_X

Open quark2 opened this issue 3 years ago • 42 comments

PR description:

We have several updates on GEM DQMs.

  • Migrations of some plots to other places (online -> offline or vice versa)
  • More plots for GE21 is now available

This PR is the backport of #38398 (+#38526) to CMSSW_12_3_X.

PR validation:

Test are done and one can check again by runTheMatrix workflows

@jshlee @watson-ij @seungjin-yang

quark2 avatar Jun 24 '22 15:06 quark2

A new Pull Request was created by @quark2 for CMSSW_12_3_X.

It involves the following packages:

  • DQM/GEM (dqm)
  • EventFilter/GEMRawToDigi (reconstruction)

@emanueleusai, @ahmad3213, @cmsbuild, @jfernan2, @clacaputo, @jpata, @pmandrik, @micsucmed, @rvenditti can you please review it and eventually sign? Thanks. @jshlee, @missirol, @watson-ij, @Martin-Grunewald this is something you requested to watch as well. @perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

  • Backported from #38526

cmsbuild avatar Jun 24 '22 15:06 cmsbuild

type gem

emanueleusai avatar Jun 25 '22 07:06 emanueleusai

type feature

emanueleusai avatar Jun 25 '22 07:06 emanueleusai

backport of #38398

emanueleusai avatar Jun 25 '22 07:06 emanueleusai

please test

emanueleusai avatar Jun 25 '22 07:06 emanueleusai

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-bba81e/25779/summary.html COMMIT: c01f8df7af5aad899e198d80d2a672c940c0b85c CMSSW: CMSSW_12_3_X_2022-06-24-2300/slc7_amd64_gcc10 User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/38499/25779/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 66 differences found in the comparisons
  • DQMHistoTests: Total files compared: 49
  • DQMHistoTests: Total histograms compared: 3626036
  • DQMHistoTests: Total failures: 40
  • DQMHistoTests: Total nulls: 63
  • DQMHistoTests: Total successes: 3625911
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 165746.875 KiB( 48 files compared)
  • DQMHistoSizes: changed ( 10024.0,... ): 200.027 KiB GEM/RecHits
  • DQMHistoSizes: changed ( 10024.0,... ): 7.764 KiB GEM/Digis
  • DQMHistoSizes: changed ( 11634.0,... ): 2034.563 KiB GEM/EventInfo
  • DQMHistoSizes: changed ( 11634.0,... ): 456.305 KiB GEM/RecHits
  • DQMHistoSizes: changed ( 11634.0,... ): 202.308 KiB GEM/DAQStatus
  • DQMHistoSizes: changed ( 11634.0,... ): 77.393 KiB GEM/Digis
  • DQMHistoSizes: changed ( 138.4,... ): 1798.537 KiB GEM/EventInfo
  • DQMHistoSizes: changed ( 138.4,... ): 353.492 KiB GEM/RecHits
  • DQMHistoSizes: changed ( 138.4,... ): 194.687 KiB GEM/DAQStatus
  • DQMHistoSizes: changed ( 138.4,... ): 15.527 KiB GEM/Digis
  • DQMHistoSizes: changed ( 23234.0 ): ...
  • Checked 205 log files, 45 edm output root files, 49 DQM output files
  • TriggerResults: no differences found

cmsbuild avatar Jun 25 '22 11:06 cmsbuild

testing at P5

emanueleusai avatar Jun 25 '22 16:06 emanueleusai

@quark2 do you need this deployed online?

emanueleusai avatar Jun 25 '22 16:06 emanueleusai

Hi @emanueleusai,

Yep, we need this update to be deployed in P5.

quark2 avatar Jun 25 '22 16:06 quark2

@quark2 @emanueleusai does this need to go into 12_3_6 ?

tvami avatar Jun 27 '22 12:06 tvami

+reconstruction

  • on the reco side, it's transparent / minimal (no observed changes)
  • the reco part is a direct backport of https://github.com/cms-sw/cmssw/pull/38398

jpata avatar Jun 27 '22 13:06 jpata

Pull request #38499 was updated. @emanueleusai, @ahmad3213, @cmsbuild, @jfernan2, @clacaputo, @jpata, @pmandrik, @micsucmed, @rvenditti can you please check and sign again.

cmsbuild avatar Jun 27 '22 16:06 cmsbuild

I've found that an update on the configurations on some plots is needed to avoid unexpected issues

quark2 avatar Jun 27 '22 16:06 quark2

please test

emanueleusai avatar Jun 27 '22 17:06 emanueleusai

I've found that an update on the configurations on some plots is needed to avoid unexpected issues

Have you done that change in master too? What's the PR?

tvami avatar Jun 27 '22 17:06 tvami

Hi @tvami,

Not yet, I'm now making the PR to the master. I'll notice when it is ready.

Best regards, Byeonghak Ko

quark2 avatar Jun 27 '22 17:06 quark2

OK, with this it seems this will not enter 12_3_6, given that the PR touches the reco, I'm not sure if it can be applied on the top of 12_3_6 in DQM. @qliphy @perrotta ?

tvami avatar Jun 27 '22 17:06 tvami

The PR to the master branch has been made; #38526.

quark2 avatar Jun 27 '22 17:06 quark2

backport of https://github.com/cms-sw/cmssw/pull/38526

tvami avatar Jun 27 '22 18:06 tvami

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-bba81e/25842/summary.html COMMIT: f9397feae517a8e9f97f1e952970ea1ac0853994 CMSSW: CMSSW_12_3_X_2022-06-27-1100/slc7_amd64_gcc10 User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/38499/25842/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 30 differences found in the comparisons
  • DQMHistoTests: Total files compared: 49
  • DQMHistoTests: Total histograms compared: 3626036
  • DQMHistoTests: Total failures: 23
  • DQMHistoTests: Total nulls: 62
  • DQMHistoTests: Total successes: 3625929
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 96943.39800000002 KiB( 48 files compared)
  • DQMHistoSizes: changed ( 10024.0,... ): 200.027 KiB GEM/RecHits
  • DQMHistoSizes: changed ( 10024.0,... ): 7.764 KiB GEM/Digis
  • DQMHistoSizes: changed ( 11634.0,... ): 456.305 KiB GEM/RecHits
  • DQMHistoSizes: changed ( 11634.0,... ): 77.393 KiB GEM/Digis
  • DQMHistoSizes: changed ( 11634.0,... ): 3.201 KiB GEM/DAQStatus
  • DQMHistoSizes: changed ( 11634.0,... ): 1.261 KiB GEM/EventInfo
  • DQMHistoSizes: changed ( 138.4,... ): 353.492 KiB GEM/RecHits
  • DQMHistoSizes: changed ( 138.4,... ): 15.527 KiB GEM/Digis
  • DQMHistoSizes: changed ( 138.4,... ): 2.756 KiB GEM/DAQStatus
  • DQMHistoSizes: changed ( 138.4,... ): 0.920 KiB GEM/EventInfo
  • DQMHistoSizes: changed ( 23234.0 ): ...
  • Checked 205 log files, 45 edm output root files, 49 DQM output files
  • TriggerResults: no differences found

cmsbuild avatar Jun 27 '22 22:06 cmsbuild

+reconstruction

  • resign with no reco-related changes
  • on the reco side, it's transparent / minimal (no observed changes)
  • the reco part is a direct backport of https://github.com/cms-sw/cmssw/pull/38398

jpata avatar Jun 28 '22 12:06 jpata

tests at P5 completed successfully

emanueleusai avatar Jun 29 '22 06:06 emanueleusai

+1

emanueleusai avatar Jun 29 '22 06:06 emanueleusai

This pull request is fully signed and it will be integrated in one of the next CMSSW_12_3_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_12_5_X is complete. This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

cmsbuild avatar Jun 29 '22 06:06 cmsbuild

The modification on EventFilter/GEMRawToDigi/plugins/GEMRawToDigiModule.cc touches reco. But they are said to just add small items that keep simple bitwise statuses, and these are not used anywhere except onlineDQM and offlineDQM, see https://github.com/cms-sw/cmssw/pull/38398#issuecomment-1162088489

However, to stay on the safe side, I would suggest that this PR is only included in the online DQM release for 12_3_X, but not included in the central now closed release

perrotta avatar Jun 29 '22 07:06 perrotta

Hi @perrotta,

I think it's surely okay. Btw for the onlineDQM release, we also need to merge #38546 to the release. Is it okay?

quark2 avatar Jun 29 '22 07:06 quark2

wait does this mean you need #38546 deployed online as well? as it stands right now we just deployed #38499

emanueleusai avatar Jun 29 '22 07:06 emanueleusai

However, to stay on the safe side, I would suggest that this PR is only included in the online DQM release for 12_3_X, but not included in the central now closed release

@perrotta Can I ask where I can check the online/offline DQM release schedule?

quark2 avatar Jun 29 '22 07:06 quark2

@emanueleusai Exactly... Even it's more important than this PR.

quark2 avatar Jun 29 '22 07:06 quark2

@quark2 why didnt you add the commit https://github.com/cms-sw/cmssw/pull/38546/commits/a99221e5944edd56bb39cd060bf4efba6e0a9ccd to this PR? (Is it just logically a very different change from what you change in this PR? )

tvami avatar Jun 29 '22 07:06 tvami