Cleanup MINI/NANO modifiers

[vlimant]

PR description:

while looking for the proper modifier/era, in preparation for the 10.6 AOD reMINI-reNANO we stumbled upon many outdated process modifier and therefore cleaning them out

• run2_miniAOD_80XLegacy
• run2_miniAOD_94XFall17
• run2_miniAOD_devel
• run3_miniAOD_12X
• miniAOD_UL_preSummer20

[cmsbuild]

cms-bot internal usage

[vlimant]

is there an actual use of run2_miniAOD_80XLegacy run2_miniAOD_94XFall17 ?

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-45240/40620

• This PR adds an extra 24KB to repository

• There are other open Pull requests which might conflict with changes you have proposed:

  • File Configuration/StandardSequences/python/Eras.py modified in PR(s): #33532

[cmsbuild]

A new Pull Request was created by @vlimant for master.

It involves the following packages:

• Configuration/Eras (operations)
• Configuration/StandardSequences (operations)

@davidlange6, @cmsbuild, @fabiocos, @rappoccio, @antoniovilela can you please review it and eventually sign? Thanks. @rovere, @Martin-Grunewald, @slomeo, @mmusich, @VourMa, @dgulhan, @sameasy, @GiacomoSguazzoni, @JanFSchulte, @felicepantaleo, @mtosi, @makortel, @VinInn, @AnnikaStein, @fabiocos, @missirol this is something you requested to watch as well. @antoniovilela, @rappoccio, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

[vlimant]

interesting enough, there exists Configuration/ProcessModifiers/python/run2_miniAOD_UL_cff.py already with a similar purpose

This modifier is for common settings to run miniAOD on top of 
ultra-legacy (during LS2) Run-2 AOD

@cms-sw/orp-l2 what is the actual way to implement this : in Eras or ProcessModifiers ?

[vlimant]

could not attend last week ORP, although I left a question there. @cms-sw/orp-l2 is there a way forward here : Eras or ProcessModifiers ?

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-45240/40685

• This PR adds an extra 216KB to repository

• There are other open Pull requests which might conflict with changes you have proposed:

  • File Configuration/Eras/python/Modifier_run2_miniAOD_80XLegacy_cff.py modified in PR(s): #44368
  • File Configuration/Eras/python/Modifier_run2_miniAOD_94XFall17_cff.py modified in PR(s): #44368
  • File Configuration/Eras/python/Modifier_run2_miniAOD_devel_cff.py modified in PR(s): #44368
  • File Configuration/StandardSequences/python/Eras.py modified in PR(s): #33532
  • File RecoEgamma/EgammaPhotonProducers/python/reducedEgamma_cfi.py modified in PR(s): #42007

[cmsbuild]

Pull request #45240 was updated. @davidlange6, @mandrenguyen, @fabiocos, @cmsbuild, @hqucms, @ftorrresd, @antoniovilela, @vlimant, @jfernan2, @rappoccio can you please check and sign again.

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-45240/40686

• This PR adds an extra 216KB to repository

• Found files with invalid states:

  • Configuration/Eras/python/Modifier_run2_miniAOD_106XUL_cff.py:
    • Added: 8df01524addf509ca3a8e42324c8df2f0b0d188d
    • Deleted: 951a99cc1ce1a497bd062861e43b52903f599685

• There are other open Pull requests which might conflict with changes you have proposed:

  • File Configuration/Eras/python/Modifier_run2_miniAOD_80XLegacy_cff.py modified in PR(s): #44368
  • File Configuration/Eras/python/Modifier_run2_miniAOD_94XFall17_cff.py modified in PR(s): #44368
  • File Configuration/Eras/python/Modifier_run2_miniAOD_devel_cff.py modified in PR(s): #44368
  • File Configuration/Eras/python/Modifier_run3_miniAOD_12X_cff.py modified in PR(s): #44368
  • File Configuration/StandardSequences/python/Eras.py modified in PR(s): #33532
  • File RecoEgamma/EgammaPhotonProducers/python/reducedEgamma_cfi.py modified in PR(s): #42007

[cmsbuild]

Pull request #45240 was updated. @fabiocos, @davidlange6, @vlimant, @hqucms, @jfernan2, @ftorrresd, @antoniovilela, @rappoccio, @cmsbuild, @mandrenguyen can you please check and sign again.

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-45240/40687

• This PR adds an extra 212KB to repository

• Found files with invalid states:

  • Configuration/Eras/python/Modifier_run2_miniAOD_106XUL_cff.py:
    • Added: fa31f7af3dfdfc9a9fc676654408245c2aa43c00
    • Deleted: b5c3be5818740319218f0ad7bdddd11bfdfba7b4

• There are other open Pull requests which might conflict with changes you have proposed:

  • File Configuration/Eras/python/Modifier_run2_miniAOD_80XLegacy_cff.py modified in PR(s): #44368
  • File Configuration/Eras/python/Modifier_run2_miniAOD_94XFall17_cff.py modified in PR(s): #44368
  • File Configuration/Eras/python/Modifier_run2_miniAOD_devel_cff.py modified in PR(s): #44368
  • File Configuration/Eras/python/Modifier_run3_miniAOD_12X_cff.py modified in PR(s): #44368
  • File Configuration/StandardSequences/python/Eras.py modified in PR(s): #33532
  • File RecoEgamma/EgammaPhotonProducers/python/reducedEgamma_cfi.py modified in PR(s): #42007

[cmsbuild]

Pull request #45240 was updated. @rappoccio, @jfernan2, @hqucms, @cmsbuild, @davidlange6, @fabiocos, @vlimant, @ftorrresd, @mandrenguyen, @antoniovilela can you please check and sign again.

[vlimant]

please test

[cmsbuild]

-1

Failed Tests: UnitTests RelVals RelVals-INPUT AddOn Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-02dadf/40166/summary.html COMMIT: 37b609e2507df4def6d263cda5931a50de18eac6 CMSSW: CMSSW_14_1_X_2024-07-01-1100/el8_amd64_gcc12 User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/45240/40166/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found 11 errors in the following unit tests:

---> test testPromptPPSAlCaRecoProducer had ERRORS
---> test testExpressPPSAlCaRecoProducer had ERRORS
---> test TestConfigurationApplicationsConfigBuilder had ERRORS
and more ...

RelVals

• 136.7611136.7611_RunJetHT2016EreMINIAOD/step2_RunJetHT2016EreMINIAOD.log
• 136.8311136.8311_RunJetHT2017FreMINIAOD/step2_RunJetHT2017FreMINIAOD.log
• 136.88811136.88811_RunJetHT2018DreMINIAODUL/step2_RunJetHT2018DreMINIAODUL.log

Expand to see more relval errors ...

• 140.56
• 135.4
• 5.1
• 4.53
• 158.01
• 139.001
• 140.023
• 136.731
• 136.793
• 136.874
• 140.043
• 140.063
• 1000.0
• 1001.0
• 4.22
• 7.3
• 1306.0
• 8.0
• 13234.0
• 1330.0
• 141.042
• 9.0
• 25.0
• 14034.0
• 14234.0
• 141.044
• 141.046
• 312.0
• 2500.4
• 11634.0
• 12434.0
• 12834.0
• 25202.0
• 12846.0
• 12834.7
• 10224.0
• 29700.0
• 29696.0
• 29634.0
• 23234.0
• 13034.0
• 250202.181
• 29834.999
• 29634.911

RelVals-INPUT

• 4.64.6_MinimumBias2010A/step2_MinimumBias2010A.log
• 136.72411136.72411_RunJetHT2016B_reminiaodUL/step2_RunJetHT2016B_reminiaodUL.log
• 136.72412136.72412_RunJetHT2016B_reminiaodUL/step2_RunJetHT2016B_reminiaodUL.log

Expand to see more relval errors ...

• 136.7611
• 136.76111
• 136.7721
• 136.77211
• 136.7801
• 136.7802
• 136.7803
• 136.8311
• 136.83111
• 136.88811
• 138.4
• 138.5
• 140.201
• 140.202
• 140.56
• 140.57
• 140.5611
• 140.61
• 141.901
• 141.902
• 142.901
• 1325.5
• 1325.51
• 1325.516
• 1325.5161
• 1325.517
• 1325.518
• 158.01
• 159.01
• 142.902
• 134.0
• 144.6
• 1000.0
• 1001.0
• 1001.2
• 1001.3
• 1001.4
• 1002.0
• 1002.3
• 1002.4
• 1010.0
• 1005.0
• 1020.0
• 1003.0
• 1030.0
• 1040.1
• 1040.0
• 1041.0
• 1042.0
• 1043.0
• 1044.0
• 2500.0
• 1045.0
• 1052.1
• 1052.0
• 2500.001
• 2500.002
• 2500.1
• 2500.2
• 2500.01
• 2500.21
• 2500.011
• 2500.3
• 2500.211
• 2500.301
• 2500.012
• 2500.32
• 2500.321
• 2500.322
• 2500.323
• 2500.324
• 2500.325
• 2500.326
• 2500.327
• 2500.328
• 2500.4
• 2500.401
• 2500.402
• 2500.403
• 2500.404
• 2500.405
• 2500.406

AddOn Tests

• fastsim

[fastsim:1] cmsDriver.py TTbar_8TeV_TuneCUETP8M1_cfi  --conditions auto:run1_mc --fast  -n 100 --eventcontent AODSIM,DQM --relval 100000,1000 -s GEN,SIM,RECOBEFMIX,DIGI:pdigi_valid,L1,DIGI2RAW,L1Reco,RECO,VALIDATION  --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --datatier GEN-SIM-DIGI-RECO,DQMIO --beamspot Realistic8TeVCollision : FAILED - elapsed time: 19 sec (ended on Mon Jul  1 13:04:22 2024) - exit: 256

• fastsim1

[fastsim1:1] cmsDriver.py TTbar_13TeV_TuneCUETP8M1_cfi --conditions auto:run2_mc_l1stage1 --fast  -n 100 --eventcontent AODSIM,DQM --relval 100000,1000 -s GEN,SIM,RECOBEFMIX,DIGI:pdigi_valid,L1,DIGI2RAW,L1Reco,RECO,VALIDATION  --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --datatier GEN-SIM-DIGI-RECO,DQMIO --beamspot NominalCollision2015 --era Run2_25ns : FAILED - elapsed time: 14 sec (ended on Mon Jul  1 13:04:22 2024) - exit: 256

• fastsim2

[fastsim2:1] cmsDriver.py TTbar_13TeV_TuneCUETP8M1_cfi --conditions auto:run2_mc --fast  -n 100 --eventcontent AODSIM,DQM --relval 100000,1000 -s GEN,SIM,RECOBEFMIX,DIGI:pdigi_valid,L1,DIGI2RAW,L1Reco,RECO,VALIDATION  --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --datatier GEN-SIM-DIGI-RECO,DQMIO --beamspot NominalCollision2015 --era Run2_2016 : FAILED - elapsed time: 10 sec (ended on Mon Jul  1 13:04:23 2024) - exit: 256

[vlimant]

please test

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-45240/40792

• This PR adds an extra 32KB to repository

• Found files with invalid states:

  • Configuration/Eras/python/Modifier_run2_miniAOD_106XUL_cff.py:
    • Added: bf9d0372854677c4c22ff4dc4c11265b2711d1de
    • Deleted: 6a3cde6177fd35b0868d384c393f960dd2ea1406

• There are other open Pull requests which might conflict with changes you have proposed:

  • File Configuration/StandardSequences/python/Eras.py modified in PR(s): #33532, #45307
  • File PhysicsTools/PatAlgos/python/slimming/slimming_cff.py modified in PR(s): #45363
  • File RecoEgamma/EgammaPhotonProducers/python/reducedEgamma_cfi.py modified in PR(s): #42007

[cmsbuild]

Pull request #45240 was updated. @antoniovagnerini, @antoniovilela, @davidlange6, @fabiocos, @ftorrresd, @hqucms, @jfernan2, @mandrenguyen, @nothingface0, @rappoccio, @rvenditti, @syuvivida, @tjavaid, @vlimant can you please check and sign again.

[cmsbuild]

-1

Failed Tests: UnitTests RelVals RelVals-INPUT Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-02dadf/40196/summary.html COMMIT: 18958e1833a33b9f5b21ea70fb3a67d50caf1b02 CMSSW: CMSSW_14_1_X_2024-07-02-2300/el8_amd64_gcc12 User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/45240/40196/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found 6 errors in the following unit tests:

---> test TestDQMOfflineConfiguration_100 had ERRORS
---> test TestDQMOfflineConfiguration_140 had ERRORS
---> test test_MC_22_crosscheck had ERRORS
and more ...

RelVals

• 136.7611136.7611_RunJetHT2016EreMINIAOD/step2_RunJetHT2016EreMINIAOD.log
• 136.8311136.8311_RunJetHT2017FreMINIAOD/step2_RunJetHT2017FreMINIAOD.log
• 136.88811136.88811_RunJetHT2018DreMINIAODUL/step2_RunJetHT2018DreMINIAODUL.log

Expand to see more relval errors ...

• 135.4
• 158.01

RelVals-INPUT

• 136.72411136.72411_RunJetHT2016B_reminiaodUL/step2_RunJetHT2016B_reminiaodUL.log
• 136.72412136.72412_RunJetHT2016B_reminiaodUL/step2_RunJetHT2016B_reminiaodUL.log
• 136.7611136.7611_RunJetHT2016EreMINIAOD/step2_RunJetHT2016EreMINIAOD.log

Expand to see more relval errors ...

• 136.76111
• 136.7721
• 136.77211
• 136.8311
• 136.83111
• 136.88811
• 140.201
• 140.202
• 140.5611
• 1325.5
• 1325.51
• 1325.516
• 1325.5161
• 1325.517
• 1325.518
• 158.01
• 2500.326
• 2500.328
• 2500.404
• 2500.406

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-45240/40798

• This PR adds an extra 216KB to repository

• Found files with invalid states:

  • Configuration/Eras/python/Modifier_run2_miniAOD_106XUL_cff.py:
    • Added: bf9d0372854677c4c22ff4dc4c11265b2711d1de
    • Deleted: 6a3cde6177fd35b0868d384c393f960dd2ea1406

• There are other open Pull requests which might conflict with changes you have proposed:

  • File Configuration/PyReleaseValidation/python/relval_standard.py modified in PR(s): #45055, #45307
  • File Configuration/StandardSequences/python/Eras.py modified in PR(s): #33532, #45307
  • File PhysicsTools/PatAlgos/python/slimming/slimming_cff.py modified in PR(s): #45363
  • File RecoEgamma/EgammaPhotonProducers/python/reducedEgamma_cfi.py modified in PR(s): #42007

[cmsbuild]

Pull request #45240 was updated. @AdrianoDee, @antoniovagnerini, @antoniovilela, @cmsbuild, @davidlange6, @fabiocos, @ftorrresd, @hqucms, @jfernan2, @kskovpen, @mandrenguyen, @miquork, @nothingface0, @rappoccio, @rvenditti, @srimanob, @subirsarkar, @sunilUIET, @syuvivida, @tjavaid, @vlimant can you please check and sign again.

[vlimant]

please test

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-45240/40800

• This PR adds an extra 216KB to repository

• Found files with invalid states:

  • Configuration/Eras/python/Modifier_run2_miniAOD_106XUL_cff.py:
    • Added: bf9d0372854677c4c22ff4dc4c11265b2711d1de
    • Deleted: 6a3cde6177fd35b0868d384c393f960dd2ea1406

• There are other open Pull requests which might conflict with changes you have proposed:

  • File Configuration/PyReleaseValidation/python/relval_standard.py modified in PR(s): #45055, #45307
  • File Configuration/StandardSequences/python/Eras.py modified in PR(s): #33532, #45307
  • File PhysicsTools/PatAlgos/python/slimming/slimming_cff.py modified in PR(s): #45363
  • File RecoEgamma/EgammaPhotonProducers/python/reducedEgamma_cfi.py modified in PR(s): #42007

[cmsbuild]

Pull request #45240 was updated. @AdrianoDee, @antoniovagnerini, @antoniovilela, @davidlange6, @fabiocos, @ftorrresd, @hqucms, @jfernan2, @kskovpen, @mandrenguyen, @miquork, @nothingface0, @rappoccio, @rvenditti, @srimanob, @subirsarkar, @sunilUIET, @syuvivida, @tjavaid, @vlimant can you please check and sign again.

[cmsbuild]

-1

Failed Tests: UnitTests RelVals RelVals-INPUT Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-02dadf/40201/summary.html COMMIT: a325ad16d1d875b68c3023f8d9038ae7ddbe1cd4 CMSSW: CMSSW_14_1_X_2024-07-03-1100/el8_amd64_gcc12 User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/45240/40201/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found 3 errors in the following unit tests:

---> test test_MC_22_crosscheck had ERRORS
---> test test_MC_23_crosscheck had ERRORS
---> test TestDQMOfflineConfiguration_170 had ERRORS

RelVals

• 135.4

----- Begin Fatal Exception 03-Jul-2024 17:41:08 CEST-----------------------
An exception of category 'NoProductResolverException' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 1 stream: 0
   [1] Running path 'MINIAODSIMoutput_step'
   [2] Prefetching for module PoolOutputModule/'MINIAODSIMoutput'
   [3] Calling method for module ReducedEGProducer/'reducedEgamma'
Exception Message:
No data of type "CaloTowerConstituentsMap" with label "" in record "CaloGeometryRecord"
 Please add an ESSource or ESProducer to your job which can deliver this data.
----- End Fatal Exception -------------------------------------------------

RelVals-INPUT

• 140.201140.201_RunJetMET2022D_reMINI/step2_RunJetMET2022D_reMINI.log
• 140.202140.202_RunJetMET2022D_reMINI/step2_RunJetMET2022D_reMINI.log
• 140.5611140.5611_RunHI2018reMINIAOD/step2_RunHI2018reMINIAOD.log

Expand to see more relval errors ...

• 2500.326
• 2500.328
• 2500.404
• 2500.406

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-45240/40807

• This PR adds an extra 216KB to repository

• Found files with invalid states:

  • Configuration/Eras/python/Modifier_run2_miniAOD_106XUL_cff.py:
    • Added: bf9d0372854677c4c22ff4dc4c11265b2711d1de
    • Deleted: 6a3cde6177fd35b0868d384c393f960dd2ea1406

• There are other open Pull requests which might conflict with changes you have proposed:

  • File Configuration/PyReleaseValidation/python/relval_standard.py modified in PR(s): #45055, #45307
  • File Configuration/PyReleaseValidation/scripts/runTheMatrix.py modified in PR(s): #45370
  • File Configuration/StandardSequences/python/Eras.py modified in PR(s): #33532, #45307
  • File PhysicsTools/PatAlgos/python/slimming/slimming_cff.py modified in PR(s): #45363
  • File RecoEgamma/EgammaPhotonProducers/python/reducedEgamma_cfi.py modified in PR(s): #42007

[cmsbuild]

Pull request #45240 was updated. @AdrianoDee, @antoniovagnerini, @antoniovilela, @cmsbuild, @davidlange6, @fabiocos, @ftorrresd, @hqucms, @jfernan2, @kskovpen, @mandrenguyen, @miquork, @nothingface0, @rappoccio, @rvenditti, @srimanob, @subirsarkar, @sunilUIET, @syuvivida, @tjavaid, @vlimant can you please check and sign again.

[vlimant]

type jetmet

[vlimant]

type egamma

[vlimant]

please test

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-45240/40810

• This PR adds an extra 216KB to repository

• Found files with invalid states:

  • Configuration/Eras/python/Modifier_run2_miniAOD_106XUL_cff.py:
    • Added: bf9d0372854677c4c22ff4dc4c11265b2711d1de
    • Deleted: 6a3cde6177fd35b0868d384c393f960dd2ea1406
  • RecoEgamma/EgammaTools/python/gedGsfElectronsTo106X_cff.py:
    • Deleted: 54f04cd37a103f070e0a0ce50e6ff25cb3d71784
    • Added: 829152c21c11cc26bf1c04cf32e4828feab1d8e9

• There are other open Pull requests which might conflict with changes you have proposed:

  • File Configuration/PyReleaseValidation/python/relval_standard.py modified in PR(s): #45055, #45307
  • File Configuration/PyReleaseValidation/scripts/runTheMatrix.py modified in PR(s): #45370
  • File Configuration/StandardSequences/python/Eras.py modified in PR(s): #33532, #45307
  • File PhysicsTools/PatAlgos/python/slimming/slimming_cff.py modified in PR(s): #45363
  • File RecoEgamma/EgammaPhotonProducers/python/reducedEgamma_cfi.py modified in PR(s): #42007