cmssw Update of Phase-2 GMT Emulator

PR description:

Major update of the Phase-2 GMT Emulator:

Add the Phase-2 KMTF alogirthm for barrel muon
Get the Phase-2 OMTF and EMTF to the Phase-2 Standalone Muon ghost busting algorithm
Refined the Tracker Muon matching and quality bits for low pt muon
Restructure the TPS code into header and source files.
Update the Tracker Muon output to GT, sending quality score and isolation sum

PR validation:

Validated by L1 menu team

Mar 21 '24 00:03 zhenbinwu

cms-bot internal usage

Mar 21 '24 00:03 cmsbuild

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-44498/39609

This PR adds an extra 376KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File L1Trigger/Phase2L1GT/plugins/L1GTProducer.cc modified in PR(s): #44346

Code check has found code style and quality issues which could be resolved by applying following patch(s)

code-format: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-44498/39609/code-format.patch e.g. curl -k https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-44498/39609/code-format.patch | patch -p1 You can also run scram build code-format to apply code format directly

Mar 21 '24 00:03 cmsbuild

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-44498/39610

This PR adds an extra 852KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File L1Trigger/Phase2L1GT/plugins/L1GTProducer.cc modified in PR(s): #44346

Mar 21 '24 01:03 cmsbuild

A new Pull Request was created by @zhenbinwu for master.

It involves the following packages:

DataFormats/L1TMuonPhase2 (l1)
L1Trigger/Configuration (l1)
L1Trigger/Phase2L1GMT (l1)
L1Trigger/Phase2L1GT (upgrade, l1)

@epalencia, @subirsarkar, @aloeliger, @srimanob, @cmsbuild can you please review it and eventually sign? Thanks. @rovere, @eyigitba, @Martin-Grunewald, @dinyar, @JanFSchulte, @missirol, @thomreis this is something you requested to watch as well. @rappoccio, @antoniovilela, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

Mar 21 '24 01:03 cmsbuild

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-44498/39622

This PR adds an extra 28KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File L1Trigger/Phase2L1GT/plugins/L1GTProducer.cc modified in PR(s): #44346

Mar 22 '24 00:03 cmsbuild

Pull request #44498 was updated. @subirsarkar, @cmsbuild, @epalencia, @srimanob, @aloeliger can you please check and sign again.

Mar 22 '24 00:03 cmsbuild

Please test

Mar 22 '24 06:03 epalencia

-1

Failed Tests: RelVals RelVals-INPUT AddOn Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-f13131/38338/summary.html COMMIT: 5924afffab0edc776e43181159b2493a46b2073b CMSSW: CMSSW_14_1_X_2024-03-21-2300/el8_amd64_gcc12 User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/44498/38338/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals

5.15.1_TTbarFS/step1_TTbarFS.log
135.4135.4_ZEEFS_13/step1_ZEEFS_13.log
4.224.22_RunCosmics2011A/step2_RunCosmics2011A.log

Expand to see more relval errors ...

RelVals-INPUT

4.224.22_RunCosmics2011A/step2_RunCosmics2011A.log
4.64.6_MinimumBias2010A/step2_MinimumBias2010A.log
6.06.0_SingleMuPt1/step2_SingleMuPt1.log

Expand to see more relval errors ...

AddOn Tests

fastsim

[fastsim:1] cmsDriver.py TTbar_8TeV_TuneCUETP8M1_cfi  --conditions auto:run1_mc --fast  -n 100 --eventcontent AODSIM,DQM --relval 100000,1000 -s GEN,SIM,RECOBEFMIX,DIGI:pdigi_valid,L1,DIGI2RAW,L1Reco,RECO,VALIDATION  --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --datatier GEN-SIM-DIGI-RECO,DQMIO --beamspot Realistic8TeVCollision : FAILED - elapsed time: 6 sec (ended on Fri Mar 22 08:25:19 2024) - exit: 256

fastsim1

[fastsim1:1] cmsDriver.py TTbar_13TeV_TuneCUETP8M1_cfi --conditions auto:run2_mc_l1stage1 --fast  -n 100 --eventcontent AODSIM,DQM --relval 100000,1000 -s GEN,SIM,RECOBEFMIX,DIGI:pdigi_valid,L1,DIGI2RAW,L1Reco,RECO,VALIDATION  --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --datatier GEN-SIM-DIGI-RECO,DQMIO --beamspot NominalCollision2015 --era Run2_25ns : FAILED - elapsed time: 4 sec (ended on Fri Mar 22 08:25:19 2024) - exit: 256

fastsim2

[fastsim2:1] cmsDriver.py TTbar_13TeV_TuneCUETP8M1_cfi --conditions auto:run2_mc --fast  -n 100 --eventcontent AODSIM,DQM --relval 100000,1000 -s GEN,SIM,RECOBEFMIX,DIGI:pdigi_valid,L1,DIGI2RAW,L1Reco,RECO,VALIDATION  --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --datatier GEN-SIM-DIGI-RECO,DQMIO --beamspot NominalCollision2015 --era Run2_2016 : FAILED - elapsed time: 3 sec (ended on Fri Mar 22 08:25:20 2024) - exit: 256

Expand to see more addon errors ...

Mar 22 '24 07:03 cmsbuild

@zhenbinwu You seem to be missing a commit or file somewhere:

ModuleNotFoundError: No module named 'L1Trigger.L1TMuonOverlapPhase2'

Mar 22 '24 11:03 aloeliger

@zhenbinwu You seem to be missing a commit or file somewhere:
ModuleNotFoundError: No module named 'L1Trigger.L1TMuonOverlapPhase2'

Oh no! It seems the Phase-2 OMTF is not in the #43687, but is merged in https://github.com/cms-l1t-offline/cmssw/pull/1196.

Mar 22 '24 14:03 zhenbinwu

@mbluj It looks like in constructing the PR to CMSSW, some items were forgotten from https://github.com/cms-sw/cmssw/pull/43687, in particular, a python configuration L1Trigger/L1TMuonOverlapPhase2.

Could you please take a look at the cherry picked or rebased commits that made it into CMSSW and figure out if any files are missing?

Mar 25 '24 13:03 aloeliger

@mbluj It looks like in constructing the PR to CMSSW, some items were forgotten from #43687, in particular, a python configuration L1Trigger/L1TMuonOverlapPhase2.

Could you please take a look at the cherry picked or rebased commits that made it into CMSSW and figure out if any files are missing?

FYI, @kbunkow Before #43687 OMTF phase-1 has been used in phase-2 L1T configurations as a proxy. #43687 introduced changes in code and configuration to adapt OMTF for differences in phases 1&2, but all phase-2 stuff did not entered #43687. Instead it was PR'ed to l1t-integration via https://github.com/cms-l1t-offline/cmssw/pull/1196 which we supposed to be quickly integrated with the official CMSSW. Did it happen? If not we should introduce some patching to still use OMTF phase-1 as the proxy of its phase-2 version until the actual phase-2 version is integrated.

Mar 25 '24 13:03 mbluj

I have checked and actually it is this PR which introduces a call to L1Trigger.L1TMuonOverlapPhase2 package (not yet existing in the official CMSSW) here: https://github.com/cms-sw/cmssw/pull/44498/files#diff-c0b64ac0c0be97495049537672bca975f5be0de08e90cb72bd6000e8496bbf4cR133 I guess that before the current PR OMTF phase-1 was used for phase2 as before integration of #43687.

Mar 25 '24 14:03 mbluj

@mbluj Will your team make a PR of the Phase-2 OMTF (as in https://github.com/cms-l1t-offline/cmssw/pull/1196) to the central CMSSW? I assumed it was ready since it has been merged in the l1t-offline. The GMT has updated the interface to the Phase-2 OMTF for the SAMuon.

Mar 25 '24 14:03 zhenbinwu

@mbluj Will your team make a PR of the Phase-2 OMTF (as in cms-l1t-offline#1196) to the central CMSSW? I assumed it was ready since it has been merged in the l1t-offline. The GMT has updated the interface to the Phase-2 OMTF for the SAMuon.

My understanding is that it should be done centrally by L1T software coordinators. But, we can prepare a PR ourselves if this is a correct way to proceed. @aloeliger @epalencia could you please comment on this?

Mar 25 '24 14:03 mbluj

@mbluj We prefer the teams who made the PR to open these up to central CMSSW because other reviewers may have additional comments, and we may not be in the best position to answer any comments that get made in review.

Mar 26 '24 13:03 aloeliger

@mbluj We prefer the teams who made the PR to open these up to central CMSSW because other reviewers may have additional comments, and we may not be in the best position to answer any comments that get made in review.

OK, with me. I just thought that procedure is different. So I will open PR with OMTF phase2 to official CMSSW.

Mar 26 '24 17:03 mbluj

@mbluj Thanks. We can help you with anything you may need making the PR here. Let us know if there are any issues.

Mar 26 '24 17:03 aloeliger

FYI, PR with phase-2 OMTF has just been created: https://github.com/cms-sw/cmssw/pull/44557

Mar 27 '24 15:03 mbluj

please test with https://github.com/cms-sw/cmssw/pull/44557

Apr 02 '24 10:04 aloeliger

-1

Failed Tests: RelVals RelVals-INPUT Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-f13131/38540/summary.html COMMIT: 5924afffab0edc776e43181159b2493a46b2073b CMSSW: CMSSW_14_1_X_2024-04-02-1100/el8_amd64_gcc12 User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/44498/38540/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals

24896.0

----- Begin Fatal Exception 02-Apr-2024 14:17:38 CEST-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 1 stream: 0
   [1] Running path 'HLTriggerFinalPath'
   [2] Prefetching for module TriggerSummaryProducerAOD/'hltTriggerSummaryAOD'
   [3] Calling method for module L1GTProducer/'l1tGTProducer'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for type: std::vector<l1t::SAMuon>
Looking for module label: l1tSAMuonsGmt
Looking for productInstanceName: promptSAMuons

   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "TryToContinue = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.

----- End Fatal Exception -------------------------------------------------

24900.0

----- Begin Fatal Exception 02-Apr-2024 14:17:53 CEST-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 1 stream: 0
   [1] Running path 'HLTriggerFinalPath'
   [2] Prefetching for module TriggerSummaryProducerAOD/'hltTriggerSummaryAOD'
   [3] Calling method for module L1GTProducer/'l1tGTProducer'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for type: std::vector<l1t::SAMuon>
Looking for module label: l1tSAMuonsGmt
Looking for productInstanceName: promptSAMuons

   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "TryToContinue = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.

----- End Fatal Exception -------------------------------------------------

24834.0

----- Begin Fatal Exception 02-Apr-2024 14:20:32 CEST-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 1 stream: 0
   [1] Running path 'HLTriggerFinalPath'
   [2] Prefetching for module TriggerSummaryProducerAOD/'hltTriggerSummaryAOD'
   [3] Calling method for module L1GTProducer/'l1tGTProducer'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for type: std::vector<l1t::SAMuon>
Looking for module label: l1tSAMuonsGmt
Looking for productInstanceName: promptSAMuons

   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "TryToContinue = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.

----- End Fatal Exception -------------------------------------------------

Expand to see more relval errors ...

RelVals-INPUT

20834.020834.0_TTbar_14TeV+2026D88/step2_TTbar_14TeV+2026D88.log
23634.023634.0_TTbar_14TeV+2026D95/step2_TTbar_14TeV+2026D95.log
24834.024834.0_TTbar_14TeV+2026D98/step2_TTbar_14TeV+2026D98.log

Expand to see more relval errors ...

Apr 02 '24 12:04 cmsbuild

Ahh, okay, I ran into this one earlier. @zhenbinwu

In the change to the new GMT muons here:

https://github.com/cms-sw/cmssw/blob/5924afffab0edc776e43181159b2493a46b2073b/L1Trigger/Configuration/python/SimL1Emulator_cff.py#L172-L177

The producer/products ultimately come from here: https://github.com/cms-sw/cmssw/blob/5924afffab0edc776e43181159b2493a46b2073b/L1Trigger/Phase2L1GMT/plugins/Phase2L1TGMTSAMuonGhostCleaner.cc#L59-L60

Which changes the name from promptSAMuons to just prompt

You need to either change every consumer of GMT products to find this change, or change the name back to the previously agreed upon product.

Apr 02 '24 14:04 aloeliger

I have some fixes (changing inputs to prompt) that I will be inserting into the prototype.

Apr 02 '24 14:04 aloeliger

@zhenbinwu This touches the correlator too, so I think it's best to change everything everywhere at once, and change it over to the new name. I will have a PR to your branch later, and will merge this fix into the prototype myself.

Apr 02 '24 15:04 aloeliger

@zhenbinwu please merge https://github.com/zhenbinwu/cmssw/pull/3

Apr 02 '24 19:04 aloeliger

Thanks @aloeliger . We changed to production name as a new standard. I merged your changes.

Apr 02 '24 19:04 zhenbinwu

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-44498/39771

This PR adds an extra 440KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File L1Trigger/Configuration/python/SimL1Emulator_cff.py modified in PR(s): #44571

Apr 02 '24 19:04 cmsbuild

Pull request #44498 was updated. @epalencia, @subirsarkar, @aloeliger, @cmsbuild, @srimanob can you please check and sign again.

Apr 02 '24 19:04 cmsbuild

please test with https://github.com/cms-sw/cmssw/pull/44557

Apr 02 '24 19:04 aloeliger

-1

Failed Tests: RelVals Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-f13131/38558/summary.html COMMIT: d1b67eebba7c755d99b2a9fa5b4dd2bb94f1e572 CMSSW: CMSSW_14_1_X_2024-04-02-1100/el8_amd64_gcc12 User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/44498/38558/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals

24896.0

----- Begin Fatal Exception 02-Apr-2024 22:42:45 CEST-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 1 stream: 0
   [1] Running path 'dqmoffline_8_step'
   [2] Calling method for module L1TPhase2MuonOffline/'l1tPhase2MuonOffline'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for type: std::vector<l1t::SAMuon>
Looking for module label: l1tSAMuonsGmt
Looking for productInstanceName: promptSAMuons

   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "TryToContinue = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.

----- End Fatal Exception -------------------------------------------------

24900.0

----- Begin Fatal Exception 02-Apr-2024 22:43:02 CEST-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 1 stream: 0
   [1] Running path 'dqmoffline_8_step'
   [2] Calling method for module L1TPhase2MuonOffline/'l1tPhase2MuonOffline'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for type: std::vector<l1t::SAMuon>
Looking for module label: l1tSAMuonsGmt
Looking for productInstanceName: promptSAMuons

   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "TryToContinue = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.

----- End Fatal Exception -------------------------------------------------

24834.0

----- Begin Fatal Exception 02-Apr-2024 22:46:16 CEST-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 1 stream: 0
   [1] Running path 'dqmoffline_8_step'
   [2] Calling method for module L1TPhase2MuonOffline/'l1tPhase2MuonOffline'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for type: std::vector<l1t::SAMuon>
Looking for module label: l1tSAMuonsGmt
Looking for productInstanceName: promptSAMuons

   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "TryToContinue = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.

----- End Fatal Exception -------------------------------------------------

Expand to see more relval errors ...

Apr 02 '24 21:04 cmsbuild