[12_4_X] Update BeamSpot in Run3 MC GTs + swap `Run3RoundOptics25ns13TeVLowSigmaZ` with `Realistic25ns13p6TeVEarly2022Collision` vtx smearing

[francescobrivio]

PR description:

Backport of #38760 This PR updates the BeamSpot for Run 3:

• [X] in all realistic MC GTs for Run 3
• [X] swap Run3RoundOptics25ns13TeVLowSigmaZ with Realistic25ns13p6TeVEarly2022Collision vtx smearing
• [ ] updates the GS file used for the relvals (need input from @cms-sw/pdmv-l2)

At the same time we update the Express GT to include a new DMD tag for the PPS PCL workflows.

The new tags are: BeamSpotObjects_Realistic25ns_13p6TeVCollisions_Early2022_v3_mc BeamSpotOnlineObjects_Realistic25ns_13p6TeVCollisions_Early2022_v2_mc DropBoxMetadata_v8.1_express

GT Diffs: Express: https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/124X_dataRun3_Express_Candidate_2022_08_05_17_14_24/124X_dataRun3_Express_v5 Express Frozen: https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/124X_dataRun3_Express_Candidate_2022_08_05_17_14_24/124X_dataRun3_Express_frozen_v5 Run3 MC: https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/124X_mcRun3_2022_realistic_v10/124X_mcRun3_2022_realistic_v11 Run3 cosmic MC: https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/124X_mcRun3_2022cosmics_realistic_deco_Candidate_2022_08_10_09_07_49/124X_mcRun3_2022cosmics_realistic_deco_v12 Run3 HI MC https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/124X_mcRun3_2022_realistic_HI_Candidate_2022_08_10_09_08_10/124X_mcRun3_2022_realistic_HI_v11 Run3 2023 MC https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/124X_mcRun3_2023_realistic_Candidate_2022_08_10_09_08_32/124X_mcRun3_2023_realistic_v12 Run3 2024 MC https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/124X_mcRun3_2024_realistic_Candidate_2022_08_10_09_09_01/124X_mcRun3_2024_realistic_v12

PR validation:

Code compiles

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

Backport of #38760

[cmsbuild]

A new Pull Request was created by @francescobrivio for CMSSW_12_4_X.

It involves the following packages:

• Configuration/AlCa (alca)
• Configuration/PyReleaseValidation (pdmv, upgrade)
• EventFilter/CSCRawToDigi (reconstruction)
• Geometry/CMSCommonData (geometry, upgrade)
• Geometry/HcalCommonData (geometry)
• Geometry/TrackerCommonData (geometry)
• Geometry/VeryForwardData (geometry)
• SimG4CMS/Calo (simulation)
• SimG4CMS/ShowerLibraryProducer (simulation)
• SimG4Core/Configuration (simulation)

@malbouis, @civanch, @yuanchao, @jordan-martins, @bsunanda, @makortel, @bbilin, @saumyaphor4252, @ianna, @mdhildreth, @cmsbuild, @AdrianoDee, @srimanob, @Dr15Jones, @clacaputo, @kskovpen, @jpata, @tvami, @ChrisMisan, @francescobrivio can you please review it and eventually sign? Thanks. @VourMa, @felicepantaleo, @kpedro88, @ghugo83, @Martin-Grunewald, @bsunanda, @grzanka, @trtomei, @slomeo, @venturia, @vargasa, @makortel, @JanFSchulte, @missirol, @simonepigazzini, @thomreis, @beaucero, @barvic, @fabferro, @rovere, @VinInn, @ptcox, @tocheng, @mmusich, @mtosi, @fabiocos this is something you requested to watch as well. @perrotta, @dpiparo, @qliphy, @rappoccio you are the release manager for this.

cms-bot commands are listed here

[mmusich]

Who'll be the primary user of this?

[tvami]

Who'll be the primary user of this?

HLT studies

[Martin-Grunewald]

Sorry, but I suppose this is for the 12_4 MC production, no?

[tvami]

Sorry, but I suppose this is for the 12_4 MC production, no?

Yes, this is used in the 12_4_X MC production. On the top of this change, we'll prove you @Martin-Grunewald et al special GTs that are including the JEC + PF for your autoCondHLT

[tvami]

@cmsbuild , please test

[tvami]

@cmsbuild , please abort

• I forgot that we need input from PdmV

[mmusich]

Sorry, but I suppose this is for the 12_4 MC production, no?

Why does it need to be in release? Production does not care

[mmusich]

hold

• until someone convinces me otherwise

[cmsbuild]

Pull request has been put on hold by @mmusich They need to issue an unhold command to remove the hold state or L1 can unhold it for all

[tvami]

Marco, this was agreed by PPD L1s, and ORP yesterday. Sal, Jordan, Kaori should comment...

[mmusich]

Please let's have a written justification. IIUC there are no minutes of the ORP meeting

[fwyzard]

As far as I kow the ORP meeting yesterday had been cancelled:

From: Qiang Li [email protected] To: [email protected] Subject: ORP[2022/08/09] cancelled

Dear all,

As the title says, we cancel this week's ORP meeting as many people are on vacations.

If you have any urgent requests, please fell free write to let us know. Otherwise we can also discuss relevant stuff at the joint Ops meeting this week.

Kindly reminds you the next milestone CMSSW_12_5_0_pre5: 2022/08/23 (last open pre-release)

Best, Andrea, Qiang

[rappoccio]

@mmusich we didn't have an ORP yesterday but we discussed privately with @qliphy and this was agreed to add for posterity and ease of use.

[mmusich]

@rappoccio,

ease of use

use for what reason? Why are we changing the physics output of a closed release?

[qliphy]

@mmusich

Please find the motivation from ALCA to backport this to 12_4_X:

At the same time we see two main advantages in backporting the BeamSpot to 12_4_X:

1. The release cycle would be made consistent with the ongoing big 12_4_X MC production
2. Integrating the BeamSpot (and other further updates) in 12_4_X seems strongly supported by TSG (especially the STORM group here in cc), as discussed in this thread: https://cms-talk.web.cern.ch/t/strategy-for-backporting-gt-updates-from-12-5-to-12-4/13587 Backporting the BeamSpot would require: A. New GTs --> this is on AlCa and easy to do B. Update the upgrade workflows --> this can be easily backported from Marco's PR#38760 [1] in master C. Update the input PU library used in relvals

[srimanob]

If we need this backported PR, we should also backport #39041 but the relvals need to be regenerated with 12_4 because in #39041, the relvals are produced with 12_5.

defaultDataSets['2021']='CMSSW_12_5_0_pre4-124X_mcRun3_2022_realistic_v10_BS2022-v'

So, we will need a full sets, not only PU library.

[srimanob]

Do we still this PR, or it can be closed? @francescobrivio

[tvami]

Well the reasons why we wanted to do it (from TSG) are still there... but to do it we need input from PdmV... so this is not really waiting on AlCa now

[srimanob]

Kindly ping @cms-sw/pdmv-l2 to converge on relvals submission, to be used with this PR.

[mmusich]

test parameters:

• pull_request = https://github.com/cms-sw/cmssw/pull/39548

[mmusich]

@francescobrivio

updates the GS file used for the relvals

I think you can tick off that box in the PR description, linking to https://github.com/cms-sw/cmssw/pull/39548

[francescobrivio]

@francescobrivio

updates the GS file used for the relvals

I think you can tick off that box in the PR description, linking to #39548

@mmusich thanks for the ping! In this PR I'm changing also the GTs for phase1_2022_realistic_hi, phase1_2023_realistic and phase1_2024_realistic so I guess those should be updated in #39548 as well, right? @kskovpen

[mmusich]

so I guess those should be updated in https://github.com/cms-sw/cmssw/pull/39548 as well, right?

I am still not 100% convinced that the update in 12_4_X for 2022 MC makes sense. Updating also the other Run-3 years and the HI simulation is straight out inappropriate IMHO.

[tvami]

I am still not 100% convinced that the update in 12_4_X for 2022 MC makes sense.

Currently the TSG is using special recipes to deal with the differences, if this is merged, everything will be much clearer and less error prone.

[cmsbuild]

Pull request #39019 was updated. @malbouis, @ChrisMisan, @yuanchao, @bsunanda, @makortel, @bbilin, @saumyaphor4252, @ianna, @mdhildreth, @cmsbuild, @AdrianoDee, @srimanob, @Dr15Jones, @clacaputo, @kskovpen, @civanch, @sunilUIET, @tvami, @mandrenguyen, @francescobrivio can you please check and sign again.

[cmsbuild]

Pull request #39019 was updated. @malbouis, @ChrisMisan, @yuanchao, @bsunanda, @makortel, @bbilin, @saumyaphor4252, @ianna, @mdhildreth, @cmsbuild, @AdrianoDee, @srimanob, @Dr15Jones, @clacaputo, @kskovpen, @civanch, @sunilUIET, @tvami, @mandrenguyen, @francescobrivio can you please check and sign again.

[cmsbuild]

Pull request #39019 was updated. @malbouis, @ChrisMisan, @yuanchao, @bsunanda, @makortel, @bbilin, @saumyaphor4252, @ianna, @mdhildreth, @cmsbuild, @AdrianoDee, @srimanob, @Dr15Jones, @clacaputo, @kskovpen, @civanch, @sunilUIET, @tvami, @mandrenguyen, @francescobrivio can you please check and sign again.

[francescobrivio]

@cmsbuild please test

• now that i have finally updated all the GTs and the description of the PR

[cmsbuild]

-1

Failed Tests: UnitTests RelVals-INPUT Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-fd62c0/28445/summary.html COMMIT: dfe1b000260e8359a29ef0b37e01c6254b9dcd47 CMSSW: CMSSW_12_4_X_2022-10-22-1100/el8_amd64_gcc10 User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/39019/28445/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found errors in the following unit tests:

---> test TestDQMOnlineClient-ctpps_dqm_sourceclient had ERRORS
---> test TestDQMOnlineClient-dt4ml_dqm_sourceclient had ERRORS
---> test TestDQMOnlineClient-dt_dqm_sourceclient had ERRORS
---> test TestDQMOnlineClient-beampixel_dqm_sourceclient had ERRORS
and more ...

RelVals-INPUT

• 11606.0DAS Error
• 11725.0DAS Error
• 11925.0DAS Error

Comparison Summary

Summary:

• No significant changes to the logs found
• ROOTFileChecks: Some differences in event products or their sizes found
• Reco comparison results: 22712 differences found in the comparisons
• DQMHistoTests: Total files compared: 50
• DQMHistoTests: Total histograms compared: 3675164
• DQMHistoTests: Total failures: 49204
• DQMHistoTests: Total nulls: 2
• DQMHistoTests: Total successes: 3625936
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 5.126 KiB( 49 files compared)
• DQMHistoSizes: changed ( 11834.0 ): 5.126 KiB SiStrip/MechanicalView
• Checked 208 log files, 45 edm output root files, 50 DQM output files
• TriggerResults: found differences in 7 / 49 workflows