cmssw icon indicating copy to clipboard operation
cmssw copied to clipboard
verified publisher icon cms-sw

Extend NanoAOD for NGT Phase 2 Scouting with Tau and Gen information

Open mmusich opened this issue 5 months ago • 11 comments

PR description:

The goal of this PR is to extend support in the Phase 2 NGT Scouting NanoAODs (introduced at https://github.com/cms-sw/cmssw/pull/48091 and complemented at https://github.com/cms-sw/cmssw/pull/48394 and https://github.com/cms-sw/cmssw/pull/48295 ) for HPS Tau reconstructed at HLT (including DeepTau scores association), MET and Energy Sums as well as GenJets (including GEN flavour).

PR validation:

Run successfully the following command:

runTheMatrix.py --what upgrade -l 29634.759,29634.771,29634.772 -i all --ibeos

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

Not a backport, not meant to be backported.

mmusich avatar Jun 26 '25 15:06 mmusich

cms-bot internal usage

cmsbuild avatar Jun 26 '25 15:06 cmsbuild

test parameters:

  • workflows = 29634.759, 29634.771, 29634.772

mmusich avatar Jun 26 '25 15:06 mmusich

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-48425/45330

  • Found files with invalid states:

    • HLTrigger/NGTScouting/python/hltTausExtra_cfi.py:
      • Added: f879ab75d121c4ad85a387528a6b7300eea03a8a
      • Modified: 6e6db736eb4aee1f72b03ce516360f981a798073
      • Deleted: 82bcd1d0db61dca5ea71deb22af4a3e1c0225dac

  • There are other open Pull requests which might conflict with changes you have proposed:

    • File Configuration/PyReleaseValidation/python/relval_Run4.py modified in PR(s): #47859
    • File Configuration/PyReleaseValidation/python/upgradeWorkflowComponents.py modified in PR(s): #47090, #47859
    • File Configuration/PyReleaseValidation/scripts/runTheMatrix.py modified in PR(s): #47859

cmsbuild avatar Jun 26 '25 15:06 cmsbuild

A new Pull Request was created by @mmusich for master.

It involves the following packages:

  • Configuration/PyReleaseValidation (upgrade, pdmv)
  • HLTrigger/NGTScouting (hlt)
  • PhysicsTools/NanoAOD (xpog)
  • SimMuon/MCTruth (simulation)

@AdrianoDee, @Martin-Grunewald, @Moanwar, @civanch, @cmsbuild, @dickychant, @ftorrresd, @hqucms, @kpedro88, @mdhildreth, @miquork, @mmusich, @srimanob, @subirsarkar can you please review it and eventually sign? Thanks. @CeliaFernandez, @Fedespring, @HuguesBrun, @Martin-Grunewald, @abbiendi, @andrea21z, @battibass, @cericeci, @fabiocos, @gpetruc, @jhgoh, @makortel, @missirol, @slomeo, @trocino this is something you requested to watch as well. @antoniovilela, @mandrenguyen, @rappoccio, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

cmsbuild avatar Jun 26 '25 15:06 cmsbuild

@cmsbuild, please test

mmusich avatar Jun 26 '25 15:06 mmusich

type ngt

mmusich avatar Jun 26 '25 15:06 mmusich

-1

Failed Tests: UnitTests Size: This PR adds an extra 224KB to repository Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-7825b3/46938/summary.html COMMIT: 67b458e288004176c181cfe75a986a20a378fa1e CMSSW: CMSSW_15_1_X_2025-06-26-1100/el8_amd64_gcc12 User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/48425/46938/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found 1 errors in the following unit tests:

---> test test-das-selected-lumis had ERRORS

Comparison Summary

Summary:

  • You potentially added 1 lines to the logs
  • Reco comparison results: 2 differences found in the comparisons
  • DQMHistoTests: Total files compared: 50
  • DQMHistoTests: Total histograms compared: 4065619
  • DQMHistoTests: Total failures: 18
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 4065581
  • DQMHistoTests: Total skipped: 20
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
  • Checked 215 log files, 184 edm output root files, 50 DQM output files
  • TriggerResults: no differences found

cmsbuild avatar Jun 26 '25 17:06 cmsbuild

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-48425/45333

  • Found files with invalid states:

    • HLTrigger/NGTScouting/python/hltTausExtra_cfi.py:
      • Added: f879ab75d121c4ad85a387528a6b7300eea03a8a
      • Modified: 6e6db736eb4aee1f72b03ce516360f981a798073
      • Deleted: 82bcd1d0db61dca5ea71deb22af4a3e1c0225dac

  • There are other open Pull requests which might conflict with changes you have proposed:

    • File Configuration/PyReleaseValidation/python/relval_Run4.py modified in PR(s): #47859
    • File Configuration/PyReleaseValidation/python/upgradeWorkflowComponents.py modified in PR(s): #47090, #47859
    • File Configuration/PyReleaseValidation/scripts/runTheMatrix.py modified in PR(s): #47859

cmsbuild avatar Jun 26 '25 18:06 cmsbuild

Pull request #48425 was updated. @AdrianoDee, @Martin-Grunewald, @Moanwar, @civanch, @cmsbuild, @dickychant, @ftorrresd, @hqucms, @kpedro88, @mdhildreth, @miquork, @mmusich, @srimanob, @subirsarkar can you please check and sign again.

cmsbuild avatar Jun 26 '25 18:06 cmsbuild

@cmsbuild, please test

mmusich avatar Jun 26 '25 19:06 mmusich

+1

Size: This PR adds an extra 224KB to repository Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-7825b3/46940/summary.html COMMIT: d1c1816f1348e0e530263d12b4c2c56b4ae7fb4f CMSSW: CMSSW_15_1_X_2025-06-26-1100/el8_amd64_gcc12 User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/48425/46940/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • You potentially removed 1 lines from the logs
  • Reco comparison results: 6 differences found in the comparisons
  • DQMHistoTests: Total files compared: 50
  • DQMHistoTests: Total histograms compared: 4065619
  • DQMHistoTests: Total failures: 53
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 4065546
  • DQMHistoTests: Total skipped: 20
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
  • Checked 215 log files, 184 edm output root files, 50 DQM output files
  • TriggerResults: no differences found

cmsbuild avatar Jun 26 '25 21:06 cmsbuild

enable nano

ftorrresd avatar Jun 27 '25 13:06 ftorrresd

please test

ftorrresd avatar Jun 27 '25 13:06 ftorrresd

+1

Size: This PR adds an extra 16KB to repository Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-7825b3/46966/summary.html COMMIT: d1c1816f1348e0e530263d12b4c2c56b4ae7fb4f CMSSW: CMSSW_15_1_X_2025-06-26-2300/el8_amd64_gcc12 Additional Tests: NANO User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/48425/46966/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

NANO Comparison Summary

Summary:

  • You potentially added 396 lines to the logs
  • ROOTFileChecks: Some differences in event products or their sizes found
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 22
  • DQMHistoTests: Total histograms compared: 89100
  • DQMHistoTests: Total failures: 0
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 89100
  • DQMHistoTests: Total skipped: 0
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 21 files compared)
  • Checked 134 log files, 80 edm output root files, 22 DQM output files
  • TriggerResults: no differences found

Nano size comparison Summary:

workflow id kb/ev ref kb/ev diff kb/ev ev/s/thd ref ev/s/thd diff rate mem/thd ref mem/thd
NANOmc106Xul16v2 2500.001 3.118 3.118 0.000 ( +0.0% ) 3.68 3.82 -3.6% 2.684 2.757
NANOmc106Xul17v2 2500.002 3.177 3.177 0.000 ( +0.0% ) 3.36 3.54 -5.2% 3.130 2.776
NANOmc106Xul18v2 2500.003 3.128 3.128 0.000 ( +0.0% ) 3.49 3.61 -3.3% 3.113 2.793
NANOdata106Xul16v2 2500.011 1.560 1.560 0.000 ( +0.0% ) 6.07 6.35 -4.3% 2.751 2.544
NANOdata106Xul17v2 2500.012 1.948 1.948 0.000 ( +0.0% ) 4.07 4.26 -4.5% 2.876 2.595
NANOdata106Xul18v2 2500.013 1.914 1.914 0.000 ( +0.0% ) 4.90 5.12 -4.3% 2.867 2.525
NANOmcUL16APVreMINI 2500.021 0.022 0.022 0.000 ( +0.0% ) 1.62 1.70 -4.7% 2.637 2.973
NANOmcUL16reMINI 2500.022 0.022 0.022 0.000 ( +0.0% ) 1.51 1.64 -8.2% 2.626 2.966
NANOmcUL17reMINI 2500.023 0.022 0.022 0.000 ( +0.0% ) 1.50 1.59 -5.9% 2.586 2.697
NANOmcUL18reMINI 2500.024 0.022 0.022 0.000 ( +0.0% ) 1.25 1.32 -4.9% 2.776 3.101
NANOdataUL16APVreMINI 2500.031 0.035 0.035 0.000 ( +0.0% ) 1.26 1.35 -6.5% 2.711 2.901
NANOdataUL16reMINI 2500.032 0.036 0.036 0.000 ( +0.0% ) 1.42 1.51 -6.0% 2.637 2.918
NANOdataUL17reMINI 2500.033 0.037 0.037 0.000 ( +0.0% ) 1.26 1.33 -5.5% 2.743 2.918
NANOdataUL18reMINI 2500.034 0.036 0.036 0.000 ( +0.0% ) 1.24 1.33 -6.7% 2.731 2.909
NANOmc130X 2500.101 3.220 3.220 0.000 ( +0.0% ) 4.21 4.39 -4.1% 3.051 2.618
NANOdata130Xrun3 2500.111 1.799 1.799 0.000 ( +0.0% ) 6.21 6.49 -4.3% 2.702 2.799
ScoutingNANOdata130Xrun3 2500.131 0.771 0.771 0.000 ( +0.0% ) 32.73 33.93 -3.5% 1.554 1.640
muPOGNANOmc140X 2500.221 2.065 2.065 0.000 ( +0.0% ) 5.45 5.66 -3.7% 2.244 2.296
EGMNANOmc140X 2500.222 3.876 3.876 0.000 ( +0.0% ) 3.93 4.05 -2.9% 2.523 2.640
BTVNANOmc140X 2500.223 10.607 10.607 0.000 ( +0.0% ) 2.45 2.61 -6.1% 2.707 2.860
jmeNANOmc140X 2500.224 6.666 6.666 0.000 ( +0.0% ) 1.12 1.17 -4.7% 2.581 2.711
lepTrackInfoNANOmc140X 2500.226 3.570 3.570 0.000 ( +0.0% ) 3.87 4.09 -5.3% 2.507 2.644
ScoutingNANOmc140X 2500.227 1.486 1.486 0.000 ( +0.0% ) 15.20 15.45 -1.6% 1.964 1.995
ScoutingNANOwithPromptmc140X 2500.228 4.474 4.474 0.000 ( +0.0% ) 3.30 3.43 -3.8% 2.866 2.859
muPOGNANO140Xrun3 2500.231 1.469 1.469 0.000 ( +0.0% ) 7.42 7.89 -6.0% 2.475 2.522
EGMNANOdata140Xrun3 2500.232 2.623 2.623 0.000 ( +0.0% ) 5.61 5.79 -3.2% 2.845 3.028
BTVNANOdata140Xrun3 2500.233 5.612 5.612 0.000 ( +0.0% ) 3.51 3.76 -6.6% 2.921 3.224
jmeNANOdata140Xrun3 2500.234 3.976 3.976 0.000 ( +0.0% ) 1.44 1.48 -2.9% 2.878 2.483
lepTrackInfoNANOdata140Xrun3 2500.236 2.413 2.413 0.000 ( +0.0% ) 5.48 5.81 -5.8% 2.879 3.028
ScoutingNANOdata140Xrun3 2500.237 1.101 1.101 0.000 ( +0.0% ) 29.00 29.72 -2.4% 1.535 1.681
ScoutingNANOwithPromptdata140Xrun3 2500.238 2.783 2.783 0.000 ( +0.0% ) 5.06 5.18 -2.3% 3.072 3.153
l1DPGNANO140Xrun3 2500.241 9.404 9.404 0.000 ( +0.0% ) 6.73 7.49 -10.1% 1.847 1.946
muDPGNANO140Xrun3 2500.242 10.331 10.331 0.000 ( +0.0% ) 1.64 1.71 -4.2% 1.674 1.745
muDPGNANOBkg140Xrun3 2500.243 2.712 2.712 0.000 ( +0.0% ) 15.26 15.49 -1.5% 1.073 1.071
hcalDPGNANO140Xrun3 2500.244 486.016 486.016 0.000 ( +0.0% ) 1.14 1.15 -1.7% 1.676 1.729
hcalDPGCalibNANO140Xrun3 2500.245 826.413 826.413 0.000 ( +0.0% ) 1.52 1.56 -2.4% 1.726 1.730
hcalDPGMCNANO140Xrun3 2500.251 645.333 645.333 0.000 ( +0.0% ) 1.65 1.67 -1.2% 1.691 1.830
NANOmc2024reMINI 2500.261 0.021 0.021 0.000 ( +0.0% ) 1.46 1.53 -4.2% 2.813 3.177
NANOdata2024reMINI 2500.271 0.036 0.036 0.000 ( +0.0% ) 1.28 1.37 -6.4% 2.685 2.933
NANOmc150X 2500.301 3.218 3.218 0.000 ( +0.0% ) 17.42 18.24 -4.5% 2.683 2.774
NANOdata150X 2500.311 2.144 2.144 0.000 ( +0.0% ) 32.76 35.50 -7.7% 2.456 2.506
NANOdata150X 2500.312 2.630 2.630 0.000 ( +0.0% ) 23.00 23.54 -2.3% 2.496 2.131
muPOGNANOmc150X 2500.321 2.255 2.255 0.000 ( +0.0% ) 15.09 15.54 -2.9% 2.488 2.659
EGMNANOmc150X 2500.322 4.110 4.110 0.000 ( +0.0% ) 13.70 14.09 -2.8% 2.579 2.759
BTVNANOmc150X 2500.323 10.551 10.551 0.000 ( +0.0% ) 4.65 4.87 -4.6% 2.808 2.967
jmeNANOmc150X 2500.324 6.760 6.760 0.000 ( +0.0% ) 1.35 1.39 -2.7% 2.886 2.306
jmeNANOrePuppimc150X 2500.325 6.762 6.762 0.000 ( +0.0% ) 1.26 1.30 -3.0% 3.206 2.558
lepTrackInfoNANOmc150X 2500.326 3.720 3.720 0.000 ( +0.0% ) 14.42 11.67 +23.6% 2.576 2.419
ScoutingNANOmc150X 2500.327 1.562 1.562 0.000 ( +0.0% ) 18.73 19.10 -1.9% 2.438 1.979
ScoutingNANOwithPromptmc150X 2500.328 4.773 4.773 0.000 ( +0.0% ) 10.43 10.64 -2.0% 2.865 3.000
BPHNANOmc150X 2500.329 4.127 4.127 0.000 ( +0.0% ) 16.70 17.25 -3.2% 2.637 2.837
muPOGNANO150Xrun3 2500.331 1.125 1.125 0.000 ( +0.0% ) 23.83 16.68 +42.9% 2.150 2.409
EGMNANOdata150Xrun3 2500.332 2.985 2.985 0.000 ( +0.0% ) 21.64 23.11 -6.4% 2.291 2.011
BTVNANOdata150Xrun3 2500.333 7.971 7.971 0.000 ( +0.0% ) 5.68 5.28 +7.6% 2.572 2.261
jmeNANOdata150Xrun3 2500.334 4.349 4.349 0.000 ( +0.0% ) 1.61 1.67 -3.4% 2.403 2.381
jmeNANOrePuppidata150Xrun3 2500.335 4.332 4.332 0.000 ( +0.0% ) 1.40 1.46 -4.1% 2.619 2.652
lepTrackInfoNANOdata150Xrun3 2500.336 2.625 2.625 0.000 ( +0.0% ) 23.78 25.13 -5.4% 2.204 2.476
ScoutingNANOdata150Xrun3 2500.337 1.737 1.737 0.000 ( +0.0% ) 30.62 31.33 -2.3% 1.528 1.689
ScoutingNANOwithPromptdata150Xrun3 2500.338 2.601 2.601 0.000 ( +0.0% ) 17.23 17.52 -1.7% 2.566 2.632
BPHNANOdata150Xrun3 2500.339 2.632 2.632 0.000 ( +0.0% ) 31.95 32.46 -1.6% 2.537 2.245
TTbarMINIAOD10.6_UL18v2 2500.901 1.819 1.819 0.000 ( +0.0% ) 29.00 29.00 -0.0% 1.920 1.733
TTbarMINIAOD14.0 2500.902 1.665 1.665 0.000 ( +0.0% ) 28.27 28.25 +0.1% 1.592 1.438
DYToLL_M-50_13TeV_pythia8 2500.911 14.345 14.345 0.000 ( +0.0% ) 8.41 7.96 +5.7% 1.110 1.110
DYToll01234Jets_5f_LO_MLM_Madgraph_LHE_13TeV 2500.912 0.438 0.240 0.198 ( +82.7% ) 3.23 3.25 -0.6% 0.859 0.855
TTbar_Pow_LHE_13TeV 2500.913 0.110 0.110 0.000 ( +0.0% ) 2.64 2.60 +1.3% 0.862 0.859

cmsbuild avatar Jun 27 '25 17:06 cmsbuild

+hlt

mmusich avatar Jun 28 '25 07:06 mmusich

+1

civanch avatar Jun 28 '25 11:06 civanch

+1

ftorrresd avatar Jun 30 '25 21:06 ftorrresd

kindly pinging @cms-sw/upgrade-l2 and @cms-sw/pdmv-l2 for the changes in Configuration/PyReleaseValidation.

mmusich avatar Jul 02 '25 06:07 mmusich

+pdmv

AdrianoDee avatar Jul 02 '25 07:07 AdrianoDee

+Upgrade

Moanwar avatar Jul 02 '25 13:07 Moanwar

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @sextonkennedy, @rappoccio, @antoniovilela, @mandrenguyen (and backports should be raised in the release meeting by the corresponding L2)

cmsbuild avatar Jul 02 '25 13:07 cmsbuild

+1

mandrenguyen avatar Jul 02 '25 13:07 mandrenguyen