cmssw icon indicating copy to clipboard operation
cmssw copied to clipboard
verified publisher icon cms-sw

Update beamspot in 12_4_X relval samples

Open kskovpen opened this issue 3 years ago • 26 comments

PR description:

Follows https://its.cern.ch/jira/browse/PDMVRELVALS-171. The new 12_4_X input samples for relval wfs are produced that use realistic beamspot, so that it is also consistent with #39041.

PR validation:

Need to wait for the production of samples.

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

Not a backport. This PR needs to go directly to 12_4_X for it to not interfere with the newer 12_5_X samples used in the master.

kskovpen avatar Sep 30 '22 07:09 kskovpen

A new Pull Request was created by @kskovpen for CMSSW_12_4_X.

It involves the following packages:

  • Configuration/PyReleaseValidation (pdmv, upgrade)

@bbilin, @cmsbuild, @AdrianoDee, @srimanob, @kskovpen, @sunilUIET can you please review it and eventually sign? Thanks. @makortel, @kpedro88, @fabiocos, @Martin-Grunewald, @missirol, @trtomei, @beaucero, @slomeo this is something you requested to watch as well. @perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

cmsbuild avatar Sep 30 '22 07:09 cmsbuild

Hi @kskovpen Do you also submit 11725.0 ? Or we need to comment it out in 12_4 also.

Best, Phat

srimanob avatar Sep 30 '22 07:09 srimanob

Hi @kskovpen Do you also submit 11725.0 ? Or we need to comment it out in 12_4 also.

Best, Phat

Hi @srimanob , yes, it has been submitted as well.

kskovpen avatar Sep 30 '22 07:09 kskovpen

Let's trigger the tests - many of them will fail for now. Will do it literately on a daily basis until we converge.

kskovpen avatar Sep 30 '22 07:09 kskovpen

please test

kskovpen avatar Sep 30 '22 07:09 kskovpen

Hi @kskovpen Do you also submit 11725.0 ? Or we need to comment it out in 12_4 also. Best, Phat

Hi @srimanob , yes, it has been submitted as well.

Thx. Can you do it for 12_5 also, so that we can bring it back to INPUT test.

srimanob avatar Sep 30 '22 07:09 srimanob

-1

Failed Tests: RelVals RelVals-INPUT Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-356284/27873/summary.html COMMIT: d669293badf810875172192783e086a121313a3c CMSSW: CMSSW_12_4_X_2022-09-29-2300/el8_amd64_gcc10 User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/39548/27873/install.sh to create a dev area with all the needed externals and cmssw changes.

  • DAS Queries: The DAS query tests failed, see the summary page for details.

RelVals

----- Begin Fatal Exception 30-Sep-2022 10:02:57 CEST-----------------------
An exception of category 'NoSecondaryFiles' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing module: class=MixingModule label='mix'
Exception Message:
RootEmbeddedFileSequence no input files specified for secondary input source.
----- End Fatal Exception -------------------------------------------------

RelVals-INPUT

  • 11601.0DAS Error
  • 11602.0DAS Error
  • 11603.0DAS Error
Expand to see more relval errors ...
  • 11604.0
  • 11605.0
  • 11606.0
  • 11607.0
  • 11608.0
  • 11609.0
  • 11630.0
  • 11634.0
  • 11634.1
  • 11634.15
  • 11634.24
  • 11634.5
  • 11634.501
  • 11634.505
  • 11634.511
  • 11634.521
  • 11634.601
  • 11634.7
  • 11634.91
  • 11640.0
  • 11643.0
  • 11646.0
  • 11650.0
  • 11650.501
  • 11650.505
  • 11723.17
  • 11725.0
  • 11834.0
  • 11834.13
  • 11834.21
  • 11834.24
  • 11834.99
  • 11846.0
  • 11925.0
  • 12034.0

cmsbuild avatar Sep 30 '22 08:09 cmsbuild

Hi @kskovpen Do you also submit 11725.0 ? Or we need to comment it out in 12_4 also. Best, Phat

Hi @srimanob , yes, it has been submitted as well.

Thx. Can you do it for 12_5 also, so that we can bring it back to INPUT test.

Submitted.

kskovpen avatar Sep 30 '22 09:09 kskovpen

Hi @kskovpen Can you not control nstreams to be 1 for https://dmytro.web.cern.ch/dmytro/cmsprodmon/workflows.php?campaign=CMSSW_12_5_0_pre4__fullsim_BS2022_14TeV-1664530270 ?

https://cms-pdmv.cern.ch/relval/relvals?status=submitted&batch_name=fullsim_BS2022_14TeV&cmssw_release=CMSSW_12_5_0_pre4&shown=1023&page=0&limit=50

srimanob avatar Oct 02 '22 16:10 srimanob

I recall that PU wfs run more efficiently with 2 streams.

kskovpen avatar Oct 04 '22 05:10 kskovpen

@cmsbuild please test

Normally, the noPU inputs should be available.

kskovpen avatar Oct 04 '22 05:10 kskovpen

please test

kskovpen avatar Oct 04 '22 05:10 kskovpen

@perrotta @rappoccio not sure if the bot is operational?

kskovpen avatar Oct 04 '22 05:10 kskovpen

@cmsbuild please test

perrotta avatar Oct 04 '22 05:10 perrotta

-1

Failed Tests: RelVals-INPUT Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-356284/27970/summary.html COMMIT: d669293badf810875172192783e086a121313a3c CMSSW: CMSSW_12_4_X_2022-10-03-2300/el8_amd64_gcc10 User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/39548/27970/install.sh to create a dev area with all the needed externals and cmssw changes.

  • DAS Queries: The DAS query tests failed, see the summary page for details.

RelVals-INPUT

  • 11606.0DAS Error
  • 11725.0DAS Error
  • 11925.0DAS Error

Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 3280 differences found in the comparisons
  • DQMHistoTests: Total files compared: 50
  • DQMHistoTests: Total histograms compared: 3677402
  • DQMHistoTests: Total failures: 8791
  • DQMHistoTests: Total nulls: 3
  • DQMHistoTests: Total successes: 3668586
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: -0.059 KiB( 49 files compared)
  • DQMHistoSizes: changed ( 11834.0 ): -0.063 KiB SiStrip/MechanicalView
  • DQMHistoSizes: changed ( 312.0 ): 0.004 KiB MessageLogger/Warnings
  • Checked 208 log files, 45 edm output root files, 50 DQM output files
  • TriggerResults: found differences in 1 / 49 workflows

cmsbuild avatar Oct 04 '22 10:10 cmsbuild

@cmsbuild , please test

tvami avatar Oct 04 '22 12:10 tvami

-1

Failed Tests: RelVals-INPUT Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-356284/27982/summary.html COMMIT: d669293badf810875172192783e086a121313a3c CMSSW: CMSSW_12_4_X_2022-10-04-1100/el8_amd64_gcc10 User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/39548/27982/install.sh to create a dev area with all the needed externals and cmssw changes.

  • DAS Queries: The DAS query tests failed, see the summary page for details.

RelVals-INPUT

  • 11606.0DAS Error
  • 11725.0DAS Error
  • 11925.0DAS Error

Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 3286 differences found in the comparisons
  • DQMHistoTests: Total files compared: 50
  • DQMHistoTests: Total histograms compared: 3677402
  • DQMHistoTests: Total failures: 8797
  • DQMHistoTests: Total nulls: 3
  • DQMHistoTests: Total successes: 3668580
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: -0.067 KiB( 49 files compared)
  • DQMHistoSizes: changed ( 11834.0 ): -0.063 KiB SiStrip/MechanicalView
  • DQMHistoSizes: changed ( 312.0 ): -0.004 KiB MessageLogger/Warnings
  • Checked 208 log files, 45 edm output root files, 50 DQM output files
  • TriggerResults: found differences in 1 / 49 workflows

cmsbuild avatar Oct 04 '22 17:10 cmsbuild

Looks like proxy/DAS errors. Let's try again.

kskovpen avatar Oct 04 '22 19:10 kskovpen

please test

kskovpen avatar Oct 04 '22 19:10 kskovpen

-1

Failed Tests: RelVals-INPUT Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-356284/27998/summary.html COMMIT: d669293badf810875172192783e086a121313a3c CMSSW: CMSSW_12_4_X_2022-10-04-1100/el8_amd64_gcc10 User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/39548/27998/install.sh to create a dev area with all the needed externals and cmssw changes.

  • DAS Queries: The DAS query tests failed, see the summary page for details.

RelVals-INPUT

  • 11606.0DAS Error
  • 11725.0DAS Error
  • 11925.0DAS Error

Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 3282 differences found in the comparisons
  • DQMHistoTests: Total files compared: 50
  • DQMHistoTests: Total histograms compared: 3677402
  • DQMHistoTests: Total failures: 8791
  • DQMHistoTests: Total nulls: 3
  • DQMHistoTests: Total successes: 3668586
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: -0.067 KiB( 49 files compared)
  • DQMHistoSizes: changed ( 11834.0 ): -0.063 KiB SiStrip/MechanicalView
  • DQMHistoSizes: changed ( 312.0 ): -0.004 KiB MessageLogger/Warnings
  • Checked 208 log files, 45 edm output root files, 50 DQM output files
  • TriggerResults: found differences in 1 / 49 workflows

cmsbuild avatar Oct 05 '22 00:10 cmsbuild

@cmsbuild , please test

tvami avatar Oct 05 '22 23:10 tvami

-1

Failed Tests: RelVals-INPUT Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-356284/28046/summary.html COMMIT: d669293badf810875172192783e086a121313a3c CMSSW: CMSSW_12_4_X_2022-10-05-2300/el8_amd64_gcc10 User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/39548/28046/install.sh to create a dev area with all the needed externals and cmssw changes.

  • DAS Queries: The DAS query tests failed, see the summary page for details.

RelVals-INPUT

  • 11606.0DAS Error
  • 11725.0DAS Error
  • 11925.0DAS Error

Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 3278 differences found in the comparisons
  • DQMHistoTests: Total files compared: 50
  • DQMHistoTests: Total histograms compared: 3677402
  • DQMHistoTests: Total failures: 8786
  • DQMHistoTests: Total nulls: 2
  • DQMHistoTests: Total successes: 3668592
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: -0.063 KiB( 49 files compared)
  • DQMHistoSizes: changed ( 11834.0 ): -0.063 KiB SiStrip/MechanicalView
  • Checked 208 log files, 45 edm output root files, 50 DQM output files
  • TriggerResults: found differences in 1 / 49 workflows

cmsbuild avatar Oct 06 '22 14:10 cmsbuild

The SingleMuPt1 inputs have been reproduced. However, the GluGluTo2Jets_14TeV ones are repeatedly failing due to GEN issues, example: https://cms-unified.web.cern.ch/cms-unified/showlog/?search=CMSSW_12_4_9_patch1__fullsim_noPU_2022_14TeV-GluGluTo2Jets_14TeV-00004. If @cms-sw/generators-l2 don't have an idea about a possible fix, probably we can remove 11725 and 11925 from IB tests in 12_4_X.

kskovpen avatar Oct 13 '22 10:10 kskovpen

Pull request #39548 was updated. @bbilin, @cmsbuild, @AdrianoDee, @srimanob, @kskovpen, @sunilUIET can you please check and sign again.

cmsbuild avatar Oct 13 '22 10:10 cmsbuild

please test

kskovpen avatar Oct 13 '22 12:10 kskovpen

-1

Failed Tests: RelVals-INPUT Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-356284/28229/summary.html COMMIT: 9bdc3c7de19ab34b4d789063c0f83b84d785bd18 CMSSW: CMSSW_12_4_X_2022-10-13-1100/el8_amd64_gcc10 User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/39548/28229/install.sh to create a dev area with all the needed externals and cmssw changes.

  • DAS Queries: The DAS query tests failed, see the summary page for details.

RelVals-INPUT

  • 11606.0DAS Error
  • 11725.0DAS Error
  • 11925.0DAS Error

Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 3280 differences found in the comparisons
  • DQMHistoTests: Total files compared: 50
  • DQMHistoTests: Total histograms compared: 3675164
  • DQMHistoTests: Total failures: 8795
  • DQMHistoTests: Total nulls: 3
  • DQMHistoTests: Total successes: 3666344
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: -0.059 KiB( 49 files compared)
  • DQMHistoSizes: changed ( 11834.0 ): -0.063 KiB SiStrip/MechanicalView
  • DQMHistoSizes: changed ( 312.0 ): 0.004 KiB MessageLogger/Warnings
  • Checked 208 log files, 45 edm output root files, 50 DQM output files
  • TriggerResults: found differences in 1 / 49 workflows

cmsbuild avatar Oct 13 '22 18:10 cmsbuild

test parameters:

  • pull request = #39019

francescobrivio avatar Oct 23 '22 13:10 francescobrivio

test parameters:

  • pull_request = #39019

francescobrivio avatar Oct 23 '22 13:10 francescobrivio

@cmsbuild please test

  • now that #39019 contains the correct GTs

francescobrivio avatar Oct 23 '22 13:10 francescobrivio

-1

Failed Tests: UnitTests RelVals-INPUT Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-356284/28446/summary.html COMMIT: 9bdc3c7de19ab34b4d789063c0f83b84d785bd18 CMSSW: CMSSW_12_4_X_2022-10-22-1100/el8_amd64_gcc10 User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/39548/28446/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found errors in the following unit tests:

---> test TestDQMOnlineClient-beam_dqm_sourceclient had ERRORS
---> test TestDQMOnlineClient-dt4ml_dqm_sourceclient had ERRORS
---> test TestDQMOnlineClient-ctpps_dqm_sourceclient had ERRORS
---> test TestDQMOnlineClient-beampixel_dqm_sourceclient had ERRORS
and more ...

RelVals-INPUT

The relvals timed out after 4 hours.

Comparison Summary

Summary:

  • No significant changes to the logs found
  • ROOTFileChecks: Some differences in event products or their sizes found
  • Reco comparison results: 22712 differences found in the comparisons
  • DQMHistoTests: Total files compared: 50
  • DQMHistoTests: Total histograms compared: 3675164
  • DQMHistoTests: Total failures: 49204
  • DQMHistoTests: Total nulls: 2
  • DQMHistoTests: Total successes: 3625936
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 5.126 KiB( 49 files compared)
  • DQMHistoSizes: changed ( 11834.0 ): 5.126 KiB SiStrip/MechanicalView
  • Checked 208 log files, 45 edm output root files, 50 DQM output files
  • TriggerResults: found differences in 7 / 49 workflows

cmsbuild avatar Oct 24 '22 03:10 cmsbuild