cmssw 2025 Wfs Skeleton

PR description:

This PR proposes the addition of 2025 wfs to the matrix. For the moment they are a copy of what we run currently for 2024 with the idea that once 2025 parameters (conditions, HLT) come we will simply need to fill the dummy ones (keeping them in sync one with the other). Here I add also:

@relval2025 pointing to GRun as a copy of @relval2024;
auto:phase1_2025_* symbolic GTs;

The era is Era_2025 that for now is simply a copy of Era_2024. A TTbar 2025 noPU wf is added to the limited matrix to be tested with PRs.

The rationale is that I would like to avoid (with some anticipation) that we fall again in a situation such as https://github.com/cms-sw/cmssw/issues/44028 or https://github.com/cms-sw/cmssw/issues/41410.

PR validation:

New wfs run.

Oct 18 '24 14:10 AdrianoDee

@AdrianoDee, CMSSW_14_2_X branch is closed for direct updates. cms-bot is going to move this PR to master branch. In future, please use cmssw master branch to submit your changes.

Oct 18 '24 14:10 cmsbuild

cms-bot internal usage

Oct 18 '24 14:10 cmsbuild

please test

Oct 18 '24 14:10 AdrianoDee

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-46445/42304

There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/PyReleaseValidation/python/relval_steps.py modified in PR(s): #46355
- File Configuration/PyReleaseValidation/python/upgradeWorkflowComponents.py modified in PR(s): #33532, #45117, #46001

Oct 18 '24 14:10 cmsbuild

A new Pull Request was created by @AdrianoDee for master.

It involves the following packages:

Configuration/AlCa (alca)
Configuration/HLT (hlt)
Configuration/PyReleaseValidation (upgrade, pdmv)

@AdrianoDee, @Martin-Grunewald, @Moanwar, @atpathak, @consuegs, @kskovpen, @miquork, @mmusich, @perrotta, @srimanob, @subirsarkar, @sunilUIET can you please review it and eventually sign? Thanks. @Martin-Grunewald, @fabiocos, @makortel, @missirol, @mmusich, @rsreds, @silviodonato, @slomeo, @tocheng, @yuanchao this is something you requested to watch as well. @antoniovilela, @mandrenguyen, @rappoccio, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

Oct 18 '24 14:10 cmsbuild

-1

Failed Tests: RelVals Size: This PR adds an extra 92KB to repository Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-758440/42298/summary.html COMMIT: e6ad2e957b3156c48f9cd15aa060d195ac7a1cc5 CMSSW: CMSSW_14_2_X_2024-10-17-2300/el8_amd64_gcc12 User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/46445/42298/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals

ValueError: Undefined workflows: 16834.0

Oct 18 '24 15:10 cmsbuild

please test

Oct 19 '24 17:10 AdrianoDee

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-46445/42314

There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/PyReleaseValidation/python/relval_steps.py modified in PR(s): #46355
- File Configuration/PyReleaseValidation/python/upgradeWorkflowComponents.py modified in PR(s): #33532, #45117, #46001

Oct 19 '24 17:10 cmsbuild

Pull request #46445 was updated. @AdrianoDee, @Martin-Grunewald, @Moanwar, @atpathak, @consuegs, @kskovpen, @miquork, @mmusich, @perrotta, @srimanob, @subirsarkar, @sunilUIET can you please check and sign again.

Oct 19 '24 17:10 cmsbuild

-1

Failed Tests: UnitTests RelVals RelVals-INPUT AddOn Size: This PR adds an extra 44KB to repository Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-758440/42306/summary.html COMMIT: 49a82d85a1be3c07b4a6c13bd06b6144d77ff1b1 CMSSW: CMSSW_14_2_X_2024-10-19-1100/el8_amd64_gcc12 User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/46445/42306/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found 54 errors in the following unit tests:

---> test test_PrepareInputDb had ERRORS
---> test test_MilleZmm had ERRORS
---> test TestDQMOnlineClient-ecalgpu_dqm_sourceclient had ERRORS
and more ...

RelVals

4.534.53_RunPhoton2012B/step2_RunPhoton2012B.log
9.09.0_Higgs200ChargedTaus/step1_Higgs200ChargedTaus.log
7.37.3_CosmicsSPLoose2018/step1_CosmicsSPLoose2018.log

Expand to see more relval errors ...

RelVals-INPUT

4.174.17_RunMinBias2011A/step2_RunMinBias2011A.log
4.264.26_ZMuSkim2011A/step2_ZMuSkim2011A.log
4.234.23_ValSkim2011A/step2_ValSkim2011A.log

Expand to see more relval errors ...

AddOn Tests

fastsim

[fastsim:1] cmsDriver.py TTbar_8TeV_TuneCUETP8M1_cfi  --conditions auto:run1_mc --fast  -n 100 --eventcontent AODSIM,DQM --relval 100000,1000 -s GEN,SIM,RECOBEFMIX,DIGI:pdigi_valid,L1,DIGI2RAW,L1Reco,RECO,VALIDATION  --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --datatier GEN-SIM-DIGI-RECO,DQMIO --beamspot Realistic8TeVCollision : FAILED - elapsed time: 9 sec (ended on Sat Oct 19 20:13:05 2024) - exit: 256

fastsim1

[fastsim1:1] cmsDriver.py TTbar_13TeV_TuneCUETP8M1_cfi --conditions auto:run2_mc_l1stage1 --fast  -n 100 --eventcontent AODSIM,DQM --relval 100000,1000 -s GEN,SIM,RECOBEFMIX,DIGI:pdigi_valid,L1,DIGI2RAW,L1Reco,RECO,VALIDATION  --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --datatier GEN-SIM-DIGI-RECO,DQMIO --beamspot NominalCollision2015 --era Run2_25ns : FAILED - elapsed time: 9 sec (ended on Sat Oct 19 20:13:08 2024) - exit: 256

fastsim2

[fastsim2:1] cmsDriver.py TTbar_13TeV_TuneCUETP8M1_cfi --conditions auto:run2_mc --fast  -n 100 --eventcontent AODSIM,DQM --relval 100000,1000 -s GEN,SIM,RECOBEFMIX,DIGI:pdigi_valid,L1,DIGI2RAW,L1Reco,RECO,VALIDATION  --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --datatier GEN-SIM-DIGI-RECO,DQMIO --beamspot NominalCollision2015 --era Run2_2016 : FAILED - elapsed time: 9 sec (ended on Sat Oct 19 20:13:14 2024) - exit: 256

Expand to see more addon errors ...

Oct 19 '24 18:10 cmsbuild

please test

Oct 22 '24 07:10 AdrianoDee

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-46445/42329

There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/PyReleaseValidation/python/relval_steps.py modified in PR(s): #46355
- File Configuration/PyReleaseValidation/python/upgradeWorkflowComponents.py modified in PR(s): #33532, #45117, #46001, #46453

Oct 22 '24 07:10 cmsbuild

Pull request #46445 was updated. @AdrianoDee, @Martin-Grunewald, @Moanwar, @atpathak, @consuegs, @kskovpen, @miquork, @mmusich, @perrotta, @srimanob, @subirsarkar, @sunilUIET can you please check and sign again.

Oct 22 '24 07:10 cmsbuild

+1

Size: This PR adds an extra 16KB to repository Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-758440/42335/summary.html COMMIT: 03611e20d226776550112cddc7291d08226518a6 CMSSW: CMSSW_14_2_X_2024-10-21-2300/el8_amd64_gcc12 User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/46445/42335/install.sh to create a dev area with all the needed externals and cmssw changes.

DAS Queries: The DAS query tests failed, see the summary page for details.

Comparison Summary

Summary:

You potentially added 2649 lines to the logs
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 3573 differences found in the comparisons
DQMHistoTests: Total files compared: 46
DQMHistoTests: Total histograms compared: 3295620
DQMHistoTests: Total failures: 100302
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3195298
DQMHistoTests: Total skipped: 20
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -505683.53 KiB( 45 files compared)
DQMHistoSizes: changed ( 12834.0,... ): -41024.718 KiB HLT/JME
DQMHistoSizes: changed ( 12834.0,... ): -7833.350 KiB HLT/BTV
DQMHistoSizes: changed ( 12834.0,... ): -7723.324 KiB HLT/EXO
DQMHistoSizes: changed ( 12834.0,... ): -2888.263 KiB HLT/TOP
DQMHistoSizes: changed ( 12834.0,... ): -2705.012 KiB HLT/HLTEgammaValidation
DQMHistoSizes: changed ( 12834.0,... ): -2518.712 KiB HLT/SUSY
DQMHistoSizes: changed ( 12834.0,... ): -1757.367 KiB HLT/EGM
DQMHistoSizes: changed ( 12834.0,... ): -1724.938 KiB HLT/Objects
DQMHistoSizes: changed ( 12834.0,... ): -1681.417 KiB HLT/HIG
DQMHistoSizes: changed ( 12834.0,... ): -1506.888 KiB HLT/B2G
DQMHistoSizes: changed ( 12834.0 ): ...
Checked 201 log files, 171 edm output root files, 46 DQM output files
TriggerResults: found differences in 7 / 44 workflows

Oct 22 '24 14:10 cmsbuild

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-46445/42335

There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/PyReleaseValidation/python/relval_steps.py modified in PR(s): #46355
- File Configuration/PyReleaseValidation/python/upgradeWorkflowComponents.py modified in PR(s): #33532, #45117, #46001, #46453

Oct 22 '24 14:10 cmsbuild

Pull request #46445 was updated. @AdrianoDee, @Martin-Grunewald, @Moanwar, @atpathak, @cmsbuild, @consuegs, @kskovpen, @miquork, @mmusich, @perrotta, @srimanob, @subirsarkar, @sunilUIET can you please check and sign again.

Oct 22 '24 14:10 cmsbuild

please test

Oct 22 '24 14:10 AdrianoDee

+1

Size: This PR adds an extra 24KB to repository Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-758440/42344/summary.html COMMIT: ebf2c492caa1413aa472ad04429f8045e60b1dbd CMSSW: CMSSW_14_2_X_2024-10-21-2300/el8_amd64_gcc12 User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/46445/42344/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially added 2645 lines to the logs
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 3569 differences found in the comparisons
DQMHistoTests: Total files compared: 46
DQMHistoTests: Total histograms compared: 3295620
DQMHistoTests: Total failures: 100311
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3195289
DQMHistoTests: Total skipped: 20
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -505683.53 KiB( 45 files compared)
DQMHistoSizes: changed ( 12834.0,... ): -41024.718 KiB HLT/JME
DQMHistoSizes: changed ( 12834.0,... ): -7833.350 KiB HLT/BTV
DQMHistoSizes: changed ( 12834.0,... ): -7723.324 KiB HLT/EXO
DQMHistoSizes: changed ( 12834.0,... ): -2888.263 KiB HLT/TOP
DQMHistoSizes: changed ( 12834.0,... ): -2705.012 KiB HLT/HLTEgammaValidation
DQMHistoSizes: changed ( 12834.0,... ): -2518.712 KiB HLT/SUSY
DQMHistoSizes: changed ( 12834.0,... ): -1757.367 KiB HLT/EGM
DQMHistoSizes: changed ( 12834.0,... ): -1724.938 KiB HLT/Objects
DQMHistoSizes: changed ( 12834.0,... ): -1681.417 KiB HLT/HIG
DQMHistoSizes: changed ( 12834.0,... ): -1506.888 KiB HLT/B2G
DQMHistoSizes: changed ( 12834.0 ): ...
Checked 201 log files, 171 edm output root files, 46 DQM output files
TriggerResults: found differences in 7 / 44 workflows

Oct 22 '24 21:10 cmsbuild

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-46445/42341

There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/PyReleaseValidation/python/relval_steps.py modified in PR(s): #46355
- File Configuration/PyReleaseValidation/python/upgradeWorkflowComponents.py modified in PR(s): #33532, #45117, #46001, #46453

Oct 23 '24 07:10 cmsbuild

Pull request #46445 was updated. @AdrianoDee, @Martin-Grunewald, @Moanwar, @atpathak, @cmsbuild, @consuegs, @kskovpen, @miquork, @mmusich, @perrotta, @srimanob, @subirsarkar, @sunilUIET can you please check and sign again.

Oct 23 '24 07:10 cmsbuild

@AdrianoDee

Screenshot from 2024-10-23 09-45-15

| 141.044 	| Run2023D JetMET0 	| run3_hlt_relval 	| Run3_2023 	| HLT@relval2024 	|  
| 141.042 	| Run2023D ZeroBias 	| run3_hlt_relval 	| Run3_2023 	| HLT@relval2024 	|  	
| 141.046 	| Run2023D EGamma0 	| run3_hlt_relval 	| Run3_2023 	| HLT@relval2024 	|

Two things:

shall we move 2023 data relvals out of the short matrix and put the corresponding 2024 instead?
should these 2024 data relvals use @relval2025 to keep running a non-fake HLT menu?

Oct 23 '24 07:10 mmusich

Yes, I was thinking to do it in a separate PR. But I could use this chance indeed.

Oct 23 '24 07:10 AdrianoDee

please test

Oct 23 '24 07:10 AdrianoDee

+1

Size: This PR adds an extra 44KB to repository Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-758440/42350/summary.html COMMIT: bf2e350ac34e5832ee6fa3cbaf261155cbd2cc31 CMSSW: CMSSW_14_2_X_2024-10-22-2300/el8_amd64_gcc12 User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/46445/42350/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially added 1723 lines to the logs
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 3559 differences found in the comparisons
DQMHistoTests: Total files compared: 46
DQMHistoTests: Total histograms compared: 3224312
DQMHistoTests: Total failures: 74222
DQMHistoTests: Total nulls: 1
DQMHistoTests: Total successes: 3150069
DQMHistoTests: Total skipped: 20
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -904965.815 KiB( 45 files compared)
DQMHistoSizes: changed ( 12834.0,... ): -43568.391 KiB HLT/JME
DQMHistoSizes: changed ( 12834.0,... ): -17546.239 KiB HLT/Tracking
DQMHistoSizes: changed ( 12834.0,... ): -16130.179 KiB HLT/BTV
DQMHistoSizes: changed ( 12834.0,... ): -14352.967 KiB HLT/EGM
DQMHistoSizes: changed ( 12834.0,... ): -10467.421 KiB HLT/EXO
DQMHistoSizes: changed ( 12834.0,... ): -10390.469 KiB HLT/SiStrip
DQMHistoSizes: changed ( 12834.0,... ): -10237.354 KiB HLT/HCAL
DQMHistoSizes: changed ( 12834.0,... ): -9248.691 KiB HLT/Vertexing
DQMHistoSizes: changed ( 12834.0,... ): -9093.510 KiB HLT/Muon
DQMHistoSizes: changed ( 12834.0,... ): -9066.871 KiB HLT/Pixel
DQMHistoSizes: changed ( 12834.0 ): ...
Checked 201 log files, 171 edm output root files, 46 DQM output files
TriggerResults: found differences in 7 / 44 workflows

Oct 23 '24 14:10 cmsbuild

@AdrianoDee I assume that as soon as you sign this PR for pdmv also other groups can consider this as concluded from your part and can also sign it

Oct 25 '24 09:10 perrotta

+pdmv

@perrotta I think we were waiting each-other (:

Oct 25 '24 09:10 AdrianoDee

+alca

Oct 25 '24 09:10 perrotta

Yes, I was thinking to do it in a separate PR. But I could use this chance indeed.

so this won't happen here?

Oct 25 '24 10:10 mmusich

hold

Oct 25 '24 10:10 mmusich

Pull request has been put on hold by @mmusich They need to issue an unhold command to remove the hold state or L1 can unhold it for all

Oct 25 '24 10:10 cmsbuild