cmssw icon indicating copy to clipboard operation
cmssw copied to clipboard
verified publisher icon cms-sw

Fix trigger matching issues in L1Trigger DQMOffline

Open abrinke1 opened this issue 3 years ago • 17 comments

PR description:

Fixed issue in L1T Offline DQM preventing HLT objects from being found, which leads to empty Muon and EGamma efficiency plots. Tested in l1t-integration here: https://github.com/cms-l1t-offline/cmssw/pull/1051

PR validation:

cmsDriver.py step2 --conditions 124X_dataRun3_Prompt_v4 --data --datatier RECO,DQMIO --era Run3 --eventcontent RECO,DQM --filein file:/eos/cms/store/data/Run2022C/Muon/RAW/v1/000/356/570/00000/a76e1b83-d169-4db6-8009-96d7b76ee22c.root --fileout file:step2.root --nStreams 2 --nThreads 8 --no_exec --number 10 --process RECO --python_filename step_2L1T_cfg.py --scenario pp --step RAW2DIGI,L1Reco,RECO,DQM:@L1TMuon

cmsDriver.py step3 --conditions 124X_dataRun3_Prompt_v4 --data --era Run3 --filein file:step2_inDQM.root --fileout file:step3.root --filetype DQM --nStreams 2 --no_exec --number 10 --python_filename step_3_cfg.py --scenario pp --step HARVESTING:DQMHarvestL1TMuon

Also can run the above replacing "L1TMuon" with "L1TEgamma", and with the following input file: /eos/cms/store/data/Run2022C/EGamma/RAW/v1/000/356/570/00000/aa8267bc-e993-4ed2-920e-6349cc460523.root

abrinke1 avatar Oct 10 '22 18:10 abrinke1

Please take a look @epalencia , @cecilecaillol , and @vukasinmilosevic .

abrinke1 avatar Oct 10 '22 18:10 abrinke1

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-39691/32517

  • This PR adds an extra 24KB to repository

Code check has found code style and quality issues which could be resolved by applying following patch(s)

  • code-format: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-39691/32517/code-format.patch e.g. curl -k https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-39691/32517/code-format.patch | patch -p1 You can also run scram build code-format to apply code format directly

cmsbuild avatar Oct 10 '22 18:10 cmsbuild

@abrinke1 Thanks! Could you run "scram build code-checks" and "scram build code-format"? Could you please also replace all the couts with edmLogError/Warning/... ?

cecilecaillol avatar Oct 12 '22 08:10 cecilecaillol

Hi @cecilecaillol , thanks for the instructions: all code-checks should be implemented. All of the new "cout" statements are connected with the "m_verbose" flag, which is only used for developer testing and debugging, and is set here: https://github.com/cms-sw/cmssw/blob/master/DQMOffline/L1Trigger/python/L1TMuonDQMOffline_cfi.py#L76

These "cout" statements are not really warnings at all, and this follows the existing code approach: https://github.com/cms-sw/cmssw/blob/master/DQMOffline/L1Trigger/src/L1TMuonDQMOffline.cc#L178

abrinke1 avatar Oct 12 '22 20:10 abrinke1

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-39691/32552

  • This PR adds an extra 20KB to repository

cmsbuild avatar Oct 12 '22 20:10 cmsbuild

A new Pull Request was created by @abrinke1 for master.

It involves the following packages:

  • DQMOffline/L1Trigger (dqm, l1)

@epalencia, @emanueleusai, @ahmad3213, @cmsbuild, @rekovic, @jfernan2, @syuvivida, @pmandrik, @micsucmed, @cecilecaillol, @rvenditti can you please review it and eventually sign? Thanks. @missirol, @rociovilar this is something you requested to watch as well. @perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

cmsbuild avatar Oct 12 '22 20:10 cmsbuild

Please test

epalencia avatar Oct 12 '22 21:10 epalencia

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-234fc6/28213/summary.html COMMIT: 795600aaec6d7cd2d389eef8514e662e0b97fbde CMSSW: CMSSW_12_6_X_2022-10-12-1100/el8_amd64_gcc10 User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/39691/28213/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

@slava77 comparisons for the following workflows were not done due to missing matrix map:

  • /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-234fc6/20834.0_TTbar_14TeV+2026D88+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14+DigiTrigger+RecoGlobal+HARVESTGlobal
  • /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-234fc6/20834.75_TTbar_14TeV+2026D88_HLT75e33+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14+DigiTrigger+RecoGlobal+HLT75e33+HARVESTGlobal
  • /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-234fc6/20896.0_CloseByPGun_CE_E_Front_120um+2026D88+CE_E_Front_120um_GenSimHLBeamSpotHGCALCloseBy+DigiTrigger+RecoGlobal+HARVESTGlobal
  • /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-234fc6/20900.0_CloseByPGun_CE_H_Coarse_Scint+2026D88+CE_H_Coarse_Scint_GenSimHLBeamSpotHGCALCloseBy+DigiTrigger+RecoGlobal+HARVESTGlobal
  • /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-234fc6/21034.999_TTbar_14TeV+2026D88PU_PMXS1S2PR+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14+PREMIX_PremixHLBeamSpot14PU+DigiTriggerPU+RecoGlobalPU+HARVESTGlobalPU
  • /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-234fc6/23234.0_TTbar_14TeV+2026D94+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14+DigiTrigger+RecoGlobal+HARVESTGlobal

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 16 differences found in the comparisons
  • DQMHistoTests: Total files compared: 49
  • DQMHistoTests: Total histograms compared: 3392309
  • DQMHistoTests: Total failures: 16
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3392271
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
  • Checked 204 log files, 49 edm output root files, 49 DQM output files
  • TriggerResults: no differences found

cmsbuild avatar Oct 13 '22 03:10 cmsbuild

@abrinke1 I understand these couts are related to the verbose mode and are not errors or warnings, but couts are not allowed in CMSSW code anymore. You could use edmLogInfo (all possibilities detailed in https://twiki.cern.ch/twiki/bin/view/CMSPublic/SWGuideMessageLogger)

cecilecaillol avatar Oct 13 '22 10:10 cecilecaillol

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-39691/32562

  • This PR adds an extra 16KB to repository

Code check has found code style and quality issues which could be resolved by applying following patch(s)

  • code-format: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-39691/32562/code-format.patch e.g. curl -k https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-39691/32562/code-format.patch | patch -p1 You can also run scram build code-format to apply code format directly

cmsbuild avatar Oct 13 '22 15:10 cmsbuild

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-39691/32565

  • This PR adds an extra 16KB to repository

cmsbuild avatar Oct 13 '22 16:10 cmsbuild

Pull request #39691 was updated. @epalencia, @emanueleusai, @ahmad3213, @cmsbuild, @rekovic, @jfernan2, @syuvivida, @pmandrik, @micsucmed, @cecilecaillol, @rvenditti can you please check and sign again.

cmsbuild avatar Oct 13 '22 16:10 cmsbuild

Hi @cecilecaillol , should be all set - just need the jenkins test.

abrinke1 avatar Oct 13 '22 19:10 abrinke1

Please test

epalencia avatar Oct 13 '22 19:10 epalencia

-1

Failed Tests: RelVals-INPUT Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-234fc6/28244/summary.html COMMIT: e2cc2902a7bfa43bc099262e027b78e9902c12c0 CMSSW: CMSSW_12_6_X_2022-10-13-1100/el8_amd64_gcc10 User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/39691/28244/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals-INPUT

The relvals timed out after 4 hours.

Comparison Summary

@slava77 comparisons for the following workflows were not done due to missing matrix map:

  • /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-234fc6/20834.0_TTbar_14TeV+2026D88+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14+DigiTrigger+RecoGlobal+HARVESTGlobal
  • /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-234fc6/20834.75_TTbar_14TeV+2026D88_HLT75e33+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14+DigiTrigger+RecoGlobal+HLT75e33+HARVESTGlobal
  • /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-234fc6/20896.0_CloseByPGun_CE_E_Front_120um+2026D88+CE_E_Front_120um_GenSimHLBeamSpotHGCALCloseBy+DigiTrigger+RecoGlobal+HARVESTGlobal
  • /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-234fc6/20900.0_CloseByPGun_CE_H_Coarse_Scint+2026D88+CE_H_Coarse_Scint_GenSimHLBeamSpotHGCALCloseBy+DigiTrigger+RecoGlobal+HARVESTGlobal
  • /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-234fc6/21034.999_TTbar_14TeV+2026D88PU_PMXS1S2PR+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14+PREMIX_PremixHLBeamSpot14PU+DigiTriggerPU+RecoGlobalPU+HARVESTGlobalPU
  • /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-234fc6/23234.0_TTbar_14TeV+2026D94+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14+DigiTrigger+RecoGlobal+HARVESTGlobal

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 3 differences found in the comparisons
  • DQMHistoTests: Total files compared: 49
  • DQMHistoTests: Total histograms compared: 3392309
  • DQMHistoTests: Total failures: 7
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3392280
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
  • Checked 204 log files, 49 edm output root files, 49 DQM output files
  • TriggerResults: no differences found

cmsbuild avatar Oct 14 '22 00:10 cmsbuild

please test

cecilecaillol avatar Oct 14 '22 07:10 cecilecaillol

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-234fc6/28252/summary.html COMMIT: e2cc2902a7bfa43bc099262e027b78e9902c12c0 CMSSW: CMSSW_12_6_X_2022-10-13-2300/el8_amd64_gcc10 User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/39691/28252/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 5 differences found in the comparisons
  • Reco comparison had 6 failed jobs
  • DQMHistoTests: Total files compared: 49
  • DQMHistoTests: Total histograms compared: 3392309
  • DQMHistoTests: Total failures: 10
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3392277
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
  • Checked 204 log files, 49 edm output root files, 49 DQM output files
  • TriggerResults: no differences found

cmsbuild avatar Oct 14 '22 11:10 cmsbuild

+l1

cecilecaillol avatar Oct 17 '22 07:10 cecilecaillol

+1

emanueleusai avatar Oct 17 '22 10:10 emanueleusai

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

cmsbuild avatar Oct 17 '22 10:10 cmsbuild

+1

perrotta avatar Oct 17 '22 10:10 perrotta