cmssw icon indicating copy to clipboard operation
cmssw copied to clipboard

[L1T] Phase-2, remove old simulation and adapt DQM

Open cecilecaillol opened this issue 3 years ago • 57 comments

PR description:

This PR switches to the new Phase-2 emulators and removes the old simulations. It also fixes the PF DQM problem and adds DQM for muons. HLT needs to switch to new objects for this PR to pass the validation tests (@trtomei )

cecilecaillol avatar Jun 20 '22 13:06 cecilecaillol

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38442/30640

  • This PR adds an extra 216KB to repository

  • There are other open Pull requests which might conflict with changes you have proposed:

    • File DQMOffline/L1Trigger/interface/L1TPhase2MuonOffline.h modified in PR(s): #38375
    • File DQMOffline/L1Trigger/python/L1TPhase2MuonDQMEfficiency_cfi.py modified in PR(s): #38375
    • File DQMOffline/L1Trigger/python/L1TPhase2MuonOffline_cfi.py modified in PR(s): #38375
    • File DQMOffline/L1Trigger/python/L1TPhase2Offline_cfi.py modified in PR(s): #38375, #38314, #38342
    • File DQMOffline/L1Trigger/python/L1TriggerDqmOffline_SecondStep_cff.py modified in PR(s): #38375
    • File DQMOffline/L1Trigger/src/L1TPhase2MuonOffline.cc modified in PR(s): #38375
    • File DataFormats/L1TrackTrigger/src/classes_def.xml modified in PR(s): #38334, #38341
    • File L1Trigger/Configuration/python/L1Trigger_EventContent_cff.py modified in PR(s): #38375, #38313, #38342
    • File L1Trigger/Configuration/python/SimL1Emulator_cff.py modified in PR(s): #38375, #38316, #38313, #38327, #38340, #38342
    • File L1Trigger/L1TTrackMatch/interface/L1TkMuCorrDynamicWindows.h modified in PR(s): #38316
    • File L1Trigger/L1TTrackMatch/interface/L1TkMuMantra.h modified in PR(s): #38316
    • File L1Trigger/L1TTrackMatch/interface/MuMatchWindow.h modified in PR(s): #38316
    • File L1Trigger/L1TTrackMatch/plugins/L1TkGlbMuonProducer.cc modified in PR(s): #38316
    • File L1Trigger/L1TTrackMatch/plugins/L1TkMuonProducer.cc modified in PR(s): #38316
    • File L1Trigger/L1TTrackMatch/plugins/L1TrackSelectionProducer.cc modified in PR(s): #38334
    • File L1Trigger/L1TTrackMatch/plugins/L1TrackerEtMissEmulatorProducer.cc modified in PR(s): #38334
    • File L1Trigger/L1TTrackMatch/plugins/L1TrackerEtMissProducer.cc modified in PR(s): #38334
    • File L1Trigger/L1TTrackMatch/python/L1TkGlbMuonProducer_cfi.py modified in PR(s): #38316
    • File L1Trigger/L1TTrackMatch/python/L1TkMuonProducer_cfi.py modified in PR(s): #38316
    • File L1Trigger/L1TTrackMatch/python/L1TrackSelectionProducer_cfi.py modified in PR(s): #38334
    • File L1Trigger/L1TTrackMatch/python/L1TrackerEtMissEmulatorProducer_cfi.py modified in PR(s): #38334
    • File L1Trigger/L1TTrackMatch/python/L1TrackerEtMissProducer_cfi.py modified in PR(s): #38334
    • File L1Trigger/L1TTrackMatch/src/L1TkMuCorrDynamicWindows.cc modified in PR(s): #38316
    • File L1Trigger/L1TTrackMatch/src/L1TkMuMantra.cc modified in PR(s): #38316
    • File L1Trigger/L1TTrackMatch/src/MuMatchWindow.cc modified in PR(s): #38316
    • File L1Trigger/L1TTrackMatch/test/L1TrackObjectNtupleMaker.cc modified in PR(s): #38334
    • File L1Trigger/L1TTrackMatch/test/L1TrackObjectNtupleMaker_cfg.py modified in PR(s): #38334
    • File L1Trigger/Phase2L1ParticleFlow/interface/LinearizedPuppiAlgo.h modified in PR(s): #38316
    • File L1Trigger/Phase2L1ParticleFlow/interface/PUAlgoBase.h modified in PR(s): #38316
    • File L1Trigger/Phase2L1ParticleFlow/interface/PuppiAlgo.h modified in PR(s): #38316
    • File L1Trigger/Phase2L1ParticleFlow/plugins/L1TPFProducer.cc modified in PR(s): #38316
    • File L1Trigger/Phase2L1ParticleFlow/python/L1NNTauProducer_cff.py modified in PR(s): #38316
    • File L1Trigger/Phase2L1ParticleFlow/python/l1ParticleFlow_cff.py modified in PR(s): #38316
    • File L1Trigger/Phase2L1ParticleFlow/python/l1pfJetMet_cff.py modified in PR(s): #38375, #38316, #38342
    • File L1Trigger/Phase2L1ParticleFlow/python/l1pfProducer_cfi.py modified in PR(s): #38316
    • File L1Trigger/Phase2L1ParticleFlow/src/LinearizedPuppiAlgo.cc modified in PR(s): #38316
    • File L1Trigger/Phase2L1ParticleFlow/src/PUAlgoBase.cc modified in PR(s): #38316
    • File L1Trigger/Phase2L1ParticleFlow/src/PuppiAlgo.cc modified in PR(s): #38316

Code check has found code style and quality issues which could be resolved by applying following patch(s)

  • code-checks: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38442/30640/code-checks.patch e.g. curl -k https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38442/30640/code-checks.patch | patch -p1 You can also run scram build code-checks to apply code checks directly

  • code-format: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38442/30640/code-format.patch e.g. curl -k https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38442/30640/code-format.patch | patch -p1 You can also run scram build code-format to apply code format directly

cmsbuild avatar Jun 20 '22 13:06 cmsbuild

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38442/30641

  • This PR adds an extra 192KB to repository

  • There are other open Pull requests which might conflict with changes you have proposed:

    • File DQMOffline/L1Trigger/interface/L1TPhase2MuonOffline.h modified in PR(s): #38375
    • File DQMOffline/L1Trigger/python/L1TPhase2MuonDQMEfficiency_cfi.py modified in PR(s): #38375
    • File DQMOffline/L1Trigger/python/L1TPhase2MuonOffline_cfi.py modified in PR(s): #38375
    • File DQMOffline/L1Trigger/python/L1TPhase2Offline_cfi.py modified in PR(s): #38375, #38314, #38342
    • File DQMOffline/L1Trigger/python/L1TriggerDqmOffline_SecondStep_cff.py modified in PR(s): #38375
    • File DQMOffline/L1Trigger/src/L1TPhase2MuonOffline.cc modified in PR(s): #38375
    • File DataFormats/L1TrackTrigger/src/classes_def.xml modified in PR(s): #38334, #38341
    • File L1Trigger/Configuration/python/L1Trigger_EventContent_cff.py modified in PR(s): #38375, #38313, #38342
    • File L1Trigger/Configuration/python/SimL1Emulator_cff.py modified in PR(s): #38375, #38316, #38313, #38327, #38340, #38342
    • File L1Trigger/L1TTrackMatch/interface/L1TkMuCorrDynamicWindows.h modified in PR(s): #38316
    • File L1Trigger/L1TTrackMatch/interface/L1TkMuMantra.h modified in PR(s): #38316
    • File L1Trigger/L1TTrackMatch/interface/MuMatchWindow.h modified in PR(s): #38316
    • File L1Trigger/L1TTrackMatch/plugins/L1TkGlbMuonProducer.cc modified in PR(s): #38316
    • File L1Trigger/L1TTrackMatch/plugins/L1TkMuonProducer.cc modified in PR(s): #38316
    • File L1Trigger/L1TTrackMatch/plugins/L1TrackSelectionProducer.cc modified in PR(s): #38334
    • File L1Trigger/L1TTrackMatch/plugins/L1TrackerEtMissEmulatorProducer.cc modified in PR(s): #38334
    • File L1Trigger/L1TTrackMatch/plugins/L1TrackerEtMissProducer.cc modified in PR(s): #38334
    • File L1Trigger/L1TTrackMatch/python/L1TkGlbMuonProducer_cfi.py modified in PR(s): #38316
    • File L1Trigger/L1TTrackMatch/python/L1TkMuonProducer_cfi.py modified in PR(s): #38316
    • File L1Trigger/L1TTrackMatch/python/L1TrackSelectionProducer_cfi.py modified in PR(s): #38334
    • File L1Trigger/L1TTrackMatch/python/L1TrackerEtMissEmulatorProducer_cfi.py modified in PR(s): #38334
    • File L1Trigger/L1TTrackMatch/python/L1TrackerEtMissProducer_cfi.py modified in PR(s): #38334
    • File L1Trigger/L1TTrackMatch/src/L1TkMuCorrDynamicWindows.cc modified in PR(s): #38316
    • File L1Trigger/L1TTrackMatch/src/L1TkMuMantra.cc modified in PR(s): #38316
    • File L1Trigger/L1TTrackMatch/src/MuMatchWindow.cc modified in PR(s): #38316
    • File L1Trigger/L1TTrackMatch/test/L1TrackObjectNtupleMaker.cc modified in PR(s): #38334
    • File L1Trigger/L1TTrackMatch/test/L1TrackObjectNtupleMaker_cfg.py modified in PR(s): #38334
    • File L1Trigger/Phase2L1ParticleFlow/interface/LinearizedPuppiAlgo.h modified in PR(s): #38316
    • File L1Trigger/Phase2L1ParticleFlow/interface/PUAlgoBase.h modified in PR(s): #38316
    • File L1Trigger/Phase2L1ParticleFlow/interface/PuppiAlgo.h modified in PR(s): #38316
    • File L1Trigger/Phase2L1ParticleFlow/plugins/L1TPFProducer.cc modified in PR(s): #38316
    • File L1Trigger/Phase2L1ParticleFlow/python/L1NNTauProducer_cff.py modified in PR(s): #38316
    • File L1Trigger/Phase2L1ParticleFlow/python/l1ParticleFlow_cff.py modified in PR(s): #38316
    • File L1Trigger/Phase2L1ParticleFlow/python/l1pfJetMet_cff.py modified in PR(s): #38375, #38316, #38342
    • File L1Trigger/Phase2L1ParticleFlow/python/l1pfProducer_cfi.py modified in PR(s): #38316
    • File L1Trigger/Phase2L1ParticleFlow/src/LinearizedPuppiAlgo.cc modified in PR(s): #38316
    • File L1Trigger/Phase2L1ParticleFlow/src/PUAlgoBase.cc modified in PR(s): #38316
    • File L1Trigger/Phase2L1ParticleFlow/src/PuppiAlgo.cc modified in PR(s): #38316

cmsbuild avatar Jun 20 '22 13:06 cmsbuild

A new Pull Request was created by @cecilecaillol for master.

It involves the following packages:

  • DQMOffline/L1Trigger (dqm, l1)
  • DataFormats/L1TCorrelator (upgrade, l1)
  • DataFormats/L1TParticleFlow (upgrade, l1)
  • DataFormats/L1TrackTrigger (upgrade, l1)
  • DataFormats/L1Trigger (l1)
  • L1Trigger/Configuration (l1)
  • L1Trigger/L1CaloTrigger (upgrade, l1)
  • L1Trigger/L1TTrackMatch (upgrade, l1)
  • L1Trigger/Phase2L1GMT (l1)
  • L1Trigger/Phase2L1ParticleFlow (upgrade, l1)
  • L1Trigger/Phase2L1Taus (l1)
  • L1Trigger/VertexFinder (l1)

@rekovic, @epalencia, @emanueleusai, @ahmad3213, @cmsbuild, @AdrianoDee, @srimanob, @jfernan2, @pmandrik, @micsucmed, @cecilecaillol, @rvenditti can you please review it and eventually sign? Thanks. @kreczko, @sviret, @eyigitba, @Martin-Grunewald, @missirol, @dinyar, @thomreis, @trtomei, @rovere, @beaucero, @rociovilar this is something you requested to watch as well. @perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

cmsbuild avatar Jun 20 '22 13:06 cmsbuild

please test with https://github.com/cms-data/L1Trigger-Phase2L1ParticleFlow/pull/4

cecilecaillol avatar Jun 20 '22 13:06 cecilecaillol

-1

Failed Tests: UnitTests RelVals RelVals-INPUT AddOn Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-6d4179/25643/summary.html COMMIT: 64cbf05b56147cf232c67275644da96c3c7eb461 CMSSW: CMSSW_12_5_X_2022-06-20-1100/el8_amd64_gcc10 User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/38442/25643/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

  • @valsdav cms-sw/cmssw#37115
  • @mmusich cms-sw/cmssw#38389

You can see more details here: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-6d4179/25643/git-recent-commits.json https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-6d4179/25643/git-merge-result

Unit Tests

I found errors in the following unit tests:

---> test testRecoMETMETProducers had ERRORS

RelVals

  • 136.731136.731_RunSinglePh2016B+RunSinglePh2016B+HLTDR2_2016+RECODR2_2016reHLT_skimSinglePh_HIPM+HARVESTDR2_skimSinglePh/step2_RunSinglePh2016B+RunSinglePh2016B+HLTDR2_2016+RECODR2_2016reHLT_skimSinglePh_HIPM+HARVESTDR2_skimSinglePh.log
  • 5.15.1_TTbar+TTbarFS+HARVESTFS/step1_TTbar+TTbarFS+HARVESTFS.log
  • 135.4135.4_ZEE_13+ZEEFS_13+HARVESTUP15FS+MINIAODMCUP15FS/step1_ZEE_13+ZEEFS_13+HARVESTUP15FS+MINIAODMCUP15FS.log
Expand to see more relval errors ...

RelVals-INPUT

  • 4.224.22_RunCosmics2011A+RunCosmics2011A+RECOCOSD+ALCACOSD+SKIMCOSD+HARVESTDC/step2_RunCosmics2011A+RunCosmics2011A+RECOCOSD+ALCACOSD+SKIMCOSD+HARVESTDC.log
  • 4.64.6_MinimumBias2010A+MinimumBias2010A+RECOSKIMALCA+HARVESTDR1/step2_MinimumBias2010A+MinimumBias2010A+RECOSKIMALCA+HARVESTDR1.log
  • 6.06.0_SingleMuPt1+SingleMuPt1INPUT+DIGI+RECO+HARVEST/step2_SingleMuPt1+SingleMuPt1INPUT+DIGI+RECO+HARVEST.log
Expand to see more relval errors ...

AddOn Tests

  • fastsimcmsDriver.py TTbar_8TeV_TuneCUETP8M1_cfi --conditions auto:run1_mc --fast -n 100 --eventcontent AODSIM,DQM --relval 100000,1000 -s GEN,SIM,RECOBEFMIX,DIGI:pdigi_valid,L1,DIGI2RAW,L1Reco,RECO,VALIDATION --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --datatier GEN-SIM-DIGI-RECO,DQMIO --beamspot Realistic8TeVCollision : FAILED - time: date Mon Jun 20 18:51:47 2022-date Mon Jun 20 18:51:35 2022 s - exit: 256
  • fastsim1cmsDriver.py TTbar_13TeV_TuneCUETP8M1_cfi --conditions auto:run2_mc_l1stage1 --fast -n 100 --eventcontent AODSIM,DQM --relval 100000,1000 -s GEN,SIM,RECOBEFMIX,DIGI:pdigi_valid,L1,DIGI2RAW,L1Reco,RECO,VALIDATION --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --datatier GEN-SIM-DIGI-RECO,DQMIO --beamspot NominalCollision2015 --era Run2_25ns : FAILED - time: date Mon Jun 20 18:51:47 2022-date Mon Jun 20 18:51:40 2022 s - exit: 256
  • fastsim2cmsDriver.py TTbar_13TeV_TuneCUETP8M1_cfi --conditions auto:run2_mc --fast -n 100 --eventcontent AODSIM,DQM --relval 100000,1000 -s GEN,SIM,RECOBEFMIX,DIGI:pdigi_valid,L1,DIGI2RAW,L1Reco,RECO,VALIDATION --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --datatier GEN-SIM-DIGI-RECO,DQMIO --beamspot NominalCollision2015 --era Run2_2016 : FAILED - time: date Mon Jun 20 18:51:48 2022-date Mon Jun 20 18:51:43 2022 s - exit: 256
Expand to see more addon errors ...

cmsbuild avatar Jun 20 '22 17:06 cmsbuild

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38442/30653

  • This PR adds an extra 216KB to repository

  • There are other open Pull requests which might conflict with changes you have proposed:

    • File DQMOffline/L1Trigger/interface/L1TPhase2MuonOffline.h modified in PR(s): #38375
    • File DQMOffline/L1Trigger/python/L1TPhase2MuonDQMEfficiency_cfi.py modified in PR(s): #38375
    • File DQMOffline/L1Trigger/python/L1TPhase2MuonOffline_cfi.py modified in PR(s): #38375
    • File DQMOffline/L1Trigger/python/L1TPhase2Offline_cfi.py modified in PR(s): #38375, #38314, #38342
    • File DQMOffline/L1Trigger/python/L1TriggerDqmOffline_SecondStep_cff.py modified in PR(s): #38375
    • File DQMOffline/L1Trigger/src/L1TPhase2MuonOffline.cc modified in PR(s): #38375
    • File DataFormats/L1TrackTrigger/src/classes_def.xml modified in PR(s): #38334, #38341
    • File L1Trigger/Configuration/python/L1Trigger_EventContent_cff.py modified in PR(s): #38375, #38313, #38342
    • File L1Trigger/Configuration/python/SimL1Emulator_cff.py modified in PR(s): #38375, #38316, #38313, #38327, #38340, #38342
    • File L1Trigger/L1CaloTrigger/python/Phase1L1TJetProducer_cfi.py modified in PR(s): #38375, #38342
    • File L1Trigger/L1TTrackMatch/interface/L1TkMuCorrDynamicWindows.h modified in PR(s): #38316
    • File L1Trigger/L1TTrackMatch/interface/L1TkMuMantra.h modified in PR(s): #38316
    • File L1Trigger/L1TTrackMatch/interface/MuMatchWindow.h modified in PR(s): #38316
    • File L1Trigger/L1TTrackMatch/plugins/L1TkGlbMuonProducer.cc modified in PR(s): #38316
    • File L1Trigger/L1TTrackMatch/plugins/L1TkMuonProducer.cc modified in PR(s): #38316
    • File L1Trigger/L1TTrackMatch/plugins/L1TrackSelectionProducer.cc modified in PR(s): #38334
    • File L1Trigger/L1TTrackMatch/plugins/L1TrackerEtMissEmulatorProducer.cc modified in PR(s): #38334
    • File L1Trigger/L1TTrackMatch/plugins/L1TrackerEtMissProducer.cc modified in PR(s): #38334
    • File L1Trigger/L1TTrackMatch/python/L1TkGlbMuonProducer_cfi.py modified in PR(s): #38316
    • File L1Trigger/L1TTrackMatch/python/L1TkMuonProducer_cfi.py modified in PR(s): #38316
    • File L1Trigger/L1TTrackMatch/python/L1TrackSelectionProducer_cfi.py modified in PR(s): #38334
    • File L1Trigger/L1TTrackMatch/python/L1TrackerEtMissEmulatorProducer_cfi.py modified in PR(s): #38334
    • File L1Trigger/L1TTrackMatch/python/L1TrackerEtMissProducer_cfi.py modified in PR(s): #38334
    • File L1Trigger/L1TTrackMatch/src/L1TkMuCorrDynamicWindows.cc modified in PR(s): #38316
    • File L1Trigger/L1TTrackMatch/src/L1TkMuMantra.cc modified in PR(s): #38316
    • File L1Trigger/L1TTrackMatch/src/MuMatchWindow.cc modified in PR(s): #38316
    • File L1Trigger/L1TTrackMatch/test/L1TrackObjectNtupleMaker.cc modified in PR(s): #38334
    • File L1Trigger/L1TTrackMatch/test/L1TrackObjectNtupleMaker_cfg.py modified in PR(s): #38334
    • File L1Trigger/Phase2L1ParticleFlow/interface/LinearizedPuppiAlgo.h modified in PR(s): #38316
    • File L1Trigger/Phase2L1ParticleFlow/interface/PUAlgoBase.h modified in PR(s): #38316
    • File L1Trigger/Phase2L1ParticleFlow/interface/PuppiAlgo.h modified in PR(s): #38316
    • File L1Trigger/Phase2L1ParticleFlow/plugins/L1TPFProducer.cc modified in PR(s): #38316
    • File L1Trigger/Phase2L1ParticleFlow/python/L1NNTauProducer_cff.py modified in PR(s): #38316
    • File L1Trigger/Phase2L1ParticleFlow/python/l1ParticleFlow_cff.py modified in PR(s): #38316
    • File L1Trigger/Phase2L1ParticleFlow/python/l1pfJetMet_cff.py modified in PR(s): #38375, #38316, #38342
    • File L1Trigger/Phase2L1ParticleFlow/python/l1pfProducer_cfi.py modified in PR(s): #38316
    • File L1Trigger/Phase2L1ParticleFlow/src/LinearizedPuppiAlgo.cc modified in PR(s): #38316
    • File L1Trigger/Phase2L1ParticleFlow/src/PUAlgoBase.cc modified in PR(s): #38316
    • File L1Trigger/Phase2L1ParticleFlow/src/PuppiAlgo.cc modified in PR(s): #38316

cmsbuild avatar Jun 21 '22 08:06 cmsbuild

Pull request #38442 was updated. @rekovic, @epalencia, @emanueleusai, @ahmad3213, @cmsbuild, @AdrianoDee, @srimanob, @jfernan2, @pmandrik, @micsucmed, @cecilecaillol, @rvenditti can you please check and sign again.

cmsbuild avatar Jun 21 '22 08:06 cmsbuild

please test

cecilecaillol avatar Jun 21 '22 08:06 cecilecaillol

please test with https://github.com/cms-data/L1Trigger-Phase2L1ParticleFlow/pull/4

cecilecaillol avatar Jun 21 '22 08:06 cecilecaillol

From https://github.com/cms-sw/cmssw/pull/38442#issuecomment-1161402656 I understand that there are other PRs wich overlap with this one, or maybe they are just replaced by it. Since there are quite several L1T Phase-2 PR queued, could you please verify those overlaps, and possibly close the ones which are not intended to be merged?

perrotta avatar Jun 21 '22 08:06 perrotta

-1

Failed Tests: UnitTests RelVals Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-6d4179/25658/summary.html COMMIT: 07d563f6e6d024aed53907340d041a54b3a773d9 CMSSW: CMSSW_12_5_X_2022-06-20-2300/el8_amd64_gcc10 User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/38442/25658/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

  • @civanch cms-sw/cmssw#38194
  • @francescobrivio cms-sw/cmssw#38438
  • @cms-sw cms-sw/cmsdist#7950

You can see more details here: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-6d4179/25658/git-recent-commits.json https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-6d4179/25658/git-merge-result

Unit Tests

I found errors in the following unit tests:

---> test TestDQMOnlineClient-hlt_dqm_clientPB had ERRORS
---> test TestDQMOnlineClient-hcalreco_dqm_sourceclient had ERRORS
---> test TestDQMOnlineClient-es_dqm_sourceclient had ERRORS
---> test TestDQMOnlineClient-hcal_dqm_sourceclient had ERRORS
and more ...

RelVals

----- Begin Fatal Exception 21-Jun-2022 12:22:08 CEST-----------------------
An exception of category 'PluginLibraryLoadError' occurred while
   [0] Constructing the EventProcessor
   [1] While attempting to load plugin L1TkElectronTrackProducer
Exception Message:
unable to load /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/38442/25658/CMSSW_12_5_X_2022-06-20-2300/lib/el8_amd64_gcc10/poisoned/plugin-poisoned-L1TrackTriggerPlugins.so because /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/38442/25658/CMSSW_12_5_X_2022-06-20-2300/lib/el8_amd64_gcc10/poisoned/plugin-poisoned-L1TrackTriggerPlugins.so: cannot open shared object file: No such file or directory
----- End Fatal Exception -------------------------------------------------

cmsbuild avatar Jun 21 '22 10:06 cmsbuild

From #38442 (comment) I understand that there are other PRs wich overlap with this one, or maybe they are just replaced by it. Since there are quite several L1T Phase-2 PR queued, could you please verify those overlaps, and possibly close the ones which are not intended to be merged?

@perrotta Indeed there is some overlap and I have closed a few PRs. If this one succeeds with the HLT changes, it is the only one to be merged as it includes all the other ones

cecilecaillol avatar Jun 21 '22 10:06 cecilecaillol

@trtomei Can you please make a PR to this branch with the changes to the HLT code to run over the new L1 objects? I believe this is the only missing item to get the tests to succeed. What timescale do you expect for your changes? Thanks!

cecilecaillol avatar Jun 21 '22 10:06 cecilecaillol

@perrotta This PR addresses both comments raised in https://github.com/cms-sw/cmssw/pull/38230#issuecomment-1151531379

cecilecaillol avatar Jun 21 '22 16:06 cecilecaillol

Hi @cecilecaillol Maybe you can guide, what should be a replace object of L1TkElectronTrackProducer? It is used in https://github.com/cms-sw/cmssw/blob/master/HLTrigger/Configuration/python/HLT_75e33/modules/hltL1TkElectronsEllipticMatchCrystal_cfi.py https://github.com/cms-sw/cmssw/blob/master/HLTrigger/Configuration/python/HLT_75e33/modules/hltL1TkElectronsEllipticMatchHGC_cfi.py

srimanob avatar Jun 21 '22 16:06 srimanob

Hi @cecilecaillol Maybe you can guide, what should be a replace object of L1TkElectronTrackProducer? It is used in https://github.com/cms-sw/cmssw/blob/master/HLTrigger/Configuration/python/HLT_75e33/modules/hltL1TkElectronsEllipticMatchCrystal_cfi.py https://github.com/cms-sw/cmssw/blob/master/HLTrigger/Configuration/python/HLT_75e33/modules/hltL1TkElectronsEllipticMatchHGC_cfi.py

Hi @srimanob , You can find the list of updated objects in https://github.com/jheikkil/cmssw/blob/workNtuples_1230/L1Trigger/L1TNtuples/python/l1PhaseIITreeStep1Producer_cfi.py, and the cfi we run now to make our objects in https://github.com/cms-sw/cmssw/blob/07d563f6e6d024aed53907340d041a54b3a773d9/L1Trigger/Configuration/python/SimL1Emulator_cff.py. @cerminar can probably help you more if the two files above are not enough to make the changes (but there are many other changes to make on the HLT side since many L1 objects have been changed)

cecilecaillol avatar Jun 21 '22 16:06 cecilecaillol

Hello @srimanob , The TkElectron and TkEM objects are now produced by the Layer-1 Correlator emulator. In particular, the module: https://github.com/cms-sw/cmssw/blob/master/L1Trigger/Phase2L1ParticleFlow/python/l1ctLayer1_cff.py#L470 merges the output of the different boards for downstream consumption.

The relevant input tags are:

  • Barrel Layer-1 Correlator objects:
    • TkElectron collection: cms.InputTag("l1ctLayer1EG", "L1TkEleEB")
    • TkEm collection: cms.InputTag("l1ctLayer1EG", "L1TkEmEB")
  • Endcap Layer-1 Correlator objects:
    • TkElectron collection: cms.InputTag("l1ctLayer1EG", "L1TkEleEE")
    • TkEm collection: cms.InputTag("l1ctLayer1EG", "L1TkEmEE")

As soon as the Layer-2 emulator will be fully commissioned these collections will be merged in a single one. But this is for the next round. I hope this clarifies.

cerminar avatar Jun 22 '22 11:06 cerminar

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38442/30745

  • This PR adds an extra 264KB to repository

  • There are other open Pull requests which might conflict with changes you have proposed:

    • File DataFormats/L1TrackTrigger/src/classes_def.xml modified in PR(s): #38334, #38341
    • File L1Trigger/Configuration/python/SimL1Emulator_cff.py modified in PR(s): #38327
    • File L1Trigger/L1TTrackMatch/plugins/L1TrackSelectionProducer.cc modified in PR(s): #38334
    • File L1Trigger/L1TTrackMatch/plugins/L1TrackerEtMissEmulatorProducer.cc modified in PR(s): #38334
    • File L1Trigger/L1TTrackMatch/plugins/L1TrackerEtMissProducer.cc modified in PR(s): #38334
    • File L1Trigger/L1TTrackMatch/python/L1TrackSelectionProducer_cfi.py modified in PR(s): #38334
    • File L1Trigger/L1TTrackMatch/python/L1TrackerEtMissEmulatorProducer_cfi.py modified in PR(s): #38334
    • File L1Trigger/L1TTrackMatch/python/L1TrackerEtMissProducer_cfi.py modified in PR(s): #38334
    • File L1Trigger/L1TTrackMatch/test/L1TrackObjectNtupleMaker.cc modified in PR(s): #38334
    • File L1Trigger/L1TTrackMatch/test/L1TrackObjectNtupleMaker_cfg.py modified in PR(s): #38334

cmsbuild avatar Jun 27 '22 07:06 cmsbuild

Pull request #38442 was updated. @Martin-Grunewald, @missirol, @epalencia, @emanueleusai, @ahmad3213, @cecilecaillol, @cmsbuild, @AdrianoDee, @srimanob, @jfernan2, @pmandrik, @micsucmed, @rekovic, @rvenditti can you please check and sign again.

cmsbuild avatar Jun 27 '22 07:06 cmsbuild

please test with https://github.com/cms-data/L1Trigger-Phase2L1ParticleFlow/pull/4

cecilecaillol avatar Jun 27 '22 07:06 cecilecaillol

-1

Failed Tests: RelVals Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-6d4179/25824/summary.html COMMIT: f86cd2b63e21d38b3ce7e0d1ff9c4ca4fce6ceec CMSSW: CMSSW_12_5_X_2022-06-26-2300/el8_amd64_gcc10 User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/38442/25824/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

  • @missirol cms-sw/cmssw#38509
  • @tvami cms-sw/cmssw#38510
  • @cms-sw cms-sw/cmsdist#7960

You can see more details here: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-6d4179/25824/git-recent-commits.json https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-6d4179/25824/git-merge-result

RelVals

  • 39434.7539434.75_TTbar_14TeV+2026D88_HLT75e33+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14+DigiTrigger+RecoGlobal+HLT75e33+HARVESTGlobal/step4_TTbar_14TeV+2026D88_HLT75e33+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14+DigiTrigger+RecoGlobal+HLT75e33+HARVESTGlobal.log

cmsbuild avatar Jun 27 '22 13:06 cmsbuild

Failed Tests 39434.75

Looks like this line needs to be removed. @trtomei https://github.com/cms-sw/cmssw/blob/f86cd2b63e21d38b3ce7e0d1ff9c4ca4fce6ceec/HLTrigger/Configuration/python/HLT_75e33_cff.py#L326

missirol avatar Jun 28 '22 14:06 missirol

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38442/30926

  • This PR adds an extra 336KB to repository

  • There are other open Pull requests which might conflict with changes you have proposed:

    • File DataFormats/L1TrackTrigger/src/classes_def.xml modified in PR(s): #38334, #38341
    • File L1Trigger/L1TTrackMatch/plugins/L1TrackSelectionProducer.cc modified in PR(s): #38334
    • File L1Trigger/L1TTrackMatch/plugins/L1TrackerEtMissEmulatorProducer.cc modified in PR(s): #38334
    • File L1Trigger/L1TTrackMatch/plugins/L1TrackerEtMissProducer.cc modified in PR(s): #38334
    • File L1Trigger/L1TTrackMatch/python/L1TrackSelectionProducer_cfi.py modified in PR(s): #38334
    • File L1Trigger/L1TTrackMatch/python/L1TrackerEtMissEmulatorProducer_cfi.py modified in PR(s): #38334
    • File L1Trigger/L1TTrackMatch/python/L1TrackerEtMissProducer_cfi.py modified in PR(s): #38334
    • File L1Trigger/L1TTrackMatch/test/L1TrackObjectNtupleMaker.cc modified in PR(s): #38334
    • File L1Trigger/L1TTrackMatch/test/L1TrackObjectNtupleMaker_cfg.py modified in PR(s): #38334

cmsbuild avatar Jul 07 '22 16:07 cmsbuild

Pull request #38442 was updated. @Martin-Grunewald, @missirol, @epalencia, @emanueleusai, @ahmad3213, @cecilecaillol, @cmsbuild, @AdrianoDee, @srimanob, @jfernan2, @pmandrik, @micsucmed, @rekovic, @rvenditti can you please check and sign again.

cmsbuild avatar Jul 07 '22 16:07 cmsbuild

please test

cecilecaillol avatar Jul 07 '22 16:07 cecilecaillol

-1

Failed Tests: RelVals RelVals-INPUT Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-6d4179/26064/summary.html COMMIT: d7baccd2678a00ea8eafb1fac64bb621c3e0d0aa CMSSW: CMSSW_12_5_X_2022-07-07-1100/el8_amd64_gcc10 User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/38442/26064/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals

----- Begin Fatal Exception 07-Jul-2022 16:58:51 UTC-----------------------
An exception of category 'Configuration' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing module: class=L1TCorrectedPFJetProducer label='scPFL1PuppiCorrectedEmulator'
Exception Message:
cannot find directory 'L1PuppiSC4EmuDeregJets' in file L1Trigger/Phase2L1ParticleFlow/data/jecs/jecs.PU200_110X.root 
----- End Fatal Exception -------------------------------------------------
----- Begin Fatal Exception 07-Jul-2022 16:59:16 UTC-----------------------
An exception of category 'Configuration' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing module: class=L1TCorrectedPFJetProducer label='scPFL1PuppiCorrectedEmulator'
Exception Message:
cannot find directory 'L1PuppiSC4EmuDeregJets' in file L1Trigger/Phase2L1ParticleFlow/data/jecs/jecs.PU200_110X.root 
----- End Fatal Exception -------------------------------------------------
----- Begin Fatal Exception 07-Jul-2022 17:05:27 UTC-----------------------
An exception of category 'Configuration' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing module: class=L1TCorrectedPFJetProducer label='scPFL1PuppiCorrectedEmulator'
Exception Message:
cannot find directory 'L1PuppiSC4EmuDeregJets' in file L1Trigger/Phase2L1ParticleFlow/data/jecs/jecs.PU200_110X.root 
----- End Fatal Exception -------------------------------------------------
Expand to see more relval errors ...

RelVals-INPUT

  • 23234.023234.0_TTbar_14TeV+2026D49+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14INPUT+DigiTrigger+RecoGlobal+HARVESTGlobal/step2_TTbar_14TeV+2026D49+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14INPUT+DigiTrigger+RecoGlobal+HARVESTGlobal.log
  • 23234.2123234.21_TTbar_14TeV+2026D49_ProdLike+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14INPUT+DigiTrigger+RecoGlobal+MiniAOD/step2_TTbar_14TeV+2026D49_ProdLike+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14INPUT+DigiTrigger+RecoGlobal+MiniAOD.log
  • 23434.2123434.21_TTbar_14TeV+2026D49PU_ProdLike+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14INPUT+DigiTriggerPU+RecoGlobalPU+MiniAODPU/step2_TTbar_14TeV+2026D49PU_ProdLike+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14INPUT+DigiTriggerPU+RecoGlobalPU+MiniAODPU.log
Expand to see more relval errors ...

cmsbuild avatar Jul 07 '22 17:07 cmsbuild

please test with https://github.com/cms-data/L1Trigger-Phase2L1ParticleFlow/pull/4

cecilecaillol avatar Jul 07 '22 17:07 cecilecaillol

-1

Failed Tests: RelVals Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-6d4179/26065/summary.html COMMIT: d7baccd2678a00ea8eafb1fac64bb621c3e0d0aa CMSSW: CMSSW_12_5_X_2022-07-07-1100/el8_amd64_gcc10 User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/38442/26065/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

  • @ronchese cms-sw/cmssw#38524
  • @alejands cms-sw/cmssw#38579
  • @mmusich cms-sw/cmssw#38598
  • @bsunanda cms-sw/cmssw#38608
  • @smuzaffar cms-sw/cmssw#38609
  • @francescobrivio cms-sw/cmssw#38632
  • @watson-ij cms-sw/cmssw#38622
  • @francescobrivio cms-sw/cmssw#38611
  • @elusian cms-sw/cmssw#38540
  • @fabferro cms-sw/cmssw#38606
  • @cms-sw cms-sw/cmsdist#7980
  • @cms-sw cms-sw/cmsdist#7982
  • @vkuznet cms-sw/cmsdist#7978

You can see more details here: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-6d4179/26065/git-recent-commits.json https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-6d4179/26065/git-merge-result

RelVals

  • 39434.7539434.75_TTbar_14TeV+2026D88_HLT75e33+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14+DigiTrigger+RecoGlobal+HLT75e33+HARVESTGlobal/step4_TTbar_14TeV+2026D88_HLT75e33+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14+DigiTrigger+RecoGlobal+HLT75e33+HARVESTGlobal.log

cmsbuild avatar Jul 07 '22 21:07 cmsbuild

@trtomei This is failing because of calls to L1TkPrimaryVertex, which has been deprecated. Can you please replace these occurrences in the HLT code?

cecilecaillol avatar Jul 08 '22 08:07 cecilecaillol

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38442/31125

  • This PR adds an extra 476KB to repository

  • There are other open Pull requests which might conflict with changes you have proposed:

    • File DataFormats/L1TrackTrigger/src/classes_def.xml modified in PR(s): #38334, #38341
    • File L1Trigger/L1TTrackMatch/plugins/L1TrackSelectionProducer.cc modified in PR(s): #38334
    • File L1Trigger/L1TTrackMatch/plugins/L1TrackerEtMissEmulatorProducer.cc modified in PR(s): #38334
    • File L1Trigger/L1TTrackMatch/plugins/L1TrackerEtMissProducer.cc modified in PR(s): #38334
    • File L1Trigger/L1TTrackMatch/python/L1TrackSelectionProducer_cfi.py modified in PR(s): #38334
    • File L1Trigger/L1TTrackMatch/python/L1TrackerEtMissEmulatorProducer_cfi.py modified in PR(s): #38334
    • File L1Trigger/L1TTrackMatch/python/L1TrackerEtMissProducer_cfi.py modified in PR(s): #38334
    • File L1Trigger/L1TTrackMatch/test/L1TrackObjectNtupleMaker.cc modified in PR(s): #38334
    • File L1Trigger/L1TTrackMatch/test/L1TrackObjectNtupleMaker_cfg.py modified in PR(s): #38334

cmsbuild avatar Jul 19 '22 08:07 cmsbuild