cmssw icon indicating copy to clipboard operation
cmssw copied to clipboard
verified publisher icon cms-sw

[L1T] Update L1T MET and Add JUMP algorithm (Phase 2)

Open JunwonTomOh opened this issue 6 months ago • 9 comments

PR description: This PR introduces two new L1 MET calculation algorithms and related implementations:

A new MET emulator using polynomial interpolation has been added. This algorithm was not present in the existing emulator framework and enables smoother, resource-efficient computation of MET using a fitted interpolation curve.

A corresponding extension was made to the existing MET producer to include this new interpolation-based method. This allows users to evaluate and deploy the interpolated MET algorithm directly in CMSSW workflows.

The PR also adds a new algorithm named JUMP (Jet Uncertainty-aware MET Prediction), which uses jet energy resolution information to correct MET in high pileup conditions. Both the producer and the emulator for JUMP were developed from scratch. DPS Note publicly available on CDS: CMS DP-2025/023

These additions aim to enhance MET performance in challenging pileup scenarios and facilitate hardware emulation and validation of new MET strategies.

PR validation: All new algorithms have been tested using the FastPUPPI Used /eos/cms/store/cmst3/group/l1tr/FastPUPPI/14_2_X/fpinputs_140X/v0/TTToSemileptonic_PU200

To support the validation of these algorithms, reference plots prepared by @thesps can be found here: https://ssummers.web.cern.ch/CMS/TauJetsMET/V44JUMP/ba40f8a/

@thesps

JunwonTomOh avatar Jun 13 '25 06:06 JunwonTomOh

cms-bot internal usage

cmsbuild avatar Jun 13 '25 06:06 cmsbuild

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-48308/45177

Code check has found code style and quality issues which could be resolved by applying following patch(s)

  • code-checks: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-48308/45177/code-checks.patch e.g. curl -k https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-48308/45177/code-checks.patch | patch -p1 You can also run scram build code-checks to apply code checks directly

  • code-format: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-48308/45177/code-format.patch e.g. curl -k https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-48308/45177/code-format.patch | patch -p1 You can also run scram build code-format to apply code format directly

cmsbuild avatar Jun 13 '25 06:06 cmsbuild

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-48308/45179

cmsbuild avatar Jun 13 '25 07:06 cmsbuild

A new Pull Request was created by @JunwonTomOh for master.

It involves the following packages:

  • L1Trigger/Phase2L1ParticleFlow (l1, upgrade)

@BenjaminRS, @Moanwar, @cmsbuild, @quinnanm, @srimanob, @subirsarkar can you please review it and eventually sign? Thanks. @Martin-Grunewald, @missirol, @mmusich this is something you requested to watch as well. @antoniovilela, @mandrenguyen, @rappoccio, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

cmsbuild avatar Jun 13 '25 07:06 cmsbuild

test parameters:

  • workflow = 29634.78
  • relvals_opt = --what upgrade

BenjaminRS avatar Jun 16 '25 13:06 BenjaminRS

please test

BenjaminRS avatar Jun 16 '25 13:06 BenjaminRS

@JunwonTomOh can you update the title of this PR please, following the L1T recommendations, to: [L1T] Update L1T MET and Add JUMP algorithm (Phase 2)

BenjaminRS avatar Jun 16 '25 14:06 BenjaminRS

@BenjaminRS Thank you for the comments. I've just changed.

JunwonTomOh avatar Jun 16 '25 15:06 JunwonTomOh

+1

Size: This PR adds an extra 16KB to repository Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-ab62aa/46762/summary.html COMMIT: 070951de521a48b6f556f761ab0de0162d670261 CMSSW: CMSSW_15_1_X_2025-06-16-1100/el8_amd64_gcc12 User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/48308/46762/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

cmsbuild avatar Jun 16 '25 15:06 cmsbuild

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-48308/45856

cmsbuild avatar Aug 21 '25 02:08 cmsbuild

Pull request #48308 was updated. @BenjaminRS, @Moanwar, @cmsbuild, @quinnanm, @srimanob, @subirsarkar can you please check and sign again.

cmsbuild avatar Aug 21 '25 02:08 cmsbuild

Hello @quinnanm , sorry for the delay. I made a PR on cms-data to add the factors saved by .json files, and modified the emulator codes to read the factors from there.

JunwonTomOh avatar Aug 21 '25 02:08 JunwonTomOh

test parameters:

workflow = 29634.78 relvals_opt = --what upgrade

quinnanm avatar Aug 21 '25 18:08 quinnanm

please test

quinnanm avatar Aug 21 '25 18:08 quinnanm

-1

Failed Tests: RelVals Size: This PR adds an extra 16KB to repository Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-ab62aa/47825/summary.html COMMIT: 104769c67b15f252baaa6864962d051a686c6ccf CMSSW: CMSSW_15_1_X_2025-08-21-1100/el8_amd64_gcc12 User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/48308/47825/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals

----- Begin Fatal Exception 21-Aug-2025 21:29:42 CEST-----------------------
An exception of category 'FileInPathError' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 1 stream: 0
   [1] Running path 'Phase2L1GTProducer'
   [2] Prefetching for module L1GTProducer/'l1tGTProducer'
   [3] Calling method for module L1MetPfProducer/'l1tMETPFProducer'
Exception Message:
edm::FileInPath unable to find file L1Trigger/Phase2L1ParticleFlow/data/met/l1met_ptphi2pxpy_poly2_v1.json anywhere in the search path.
The search path is defined by: CMSSW_SEARCH_PATH
${CMSSW_SEARCH_PATH} is: /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/48308/47825/CMSSW_15_1_X_2025-08-21-1100/poison:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/48308/47825/CMSSW_15_1_X_2025-08-21-1100/src:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/48308/47825/CMSSW_15_1_X_2025-08-21-1100/external/el8_amd64_gcc12/data:/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02903/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_15_1_X_2025-08-21-1100/poison:/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02903/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_15_1_X_2025-08-21-1100/src:/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02903/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_15_1_X_2025-08-21-1100/external/el8_amd64_gcc12/data
Current directory is: /data/cmsbld/jenkins/workspace/ib-run-pr-relvals/matrix-results/29634.75_TTbar_14TeV+Run4D110_HLT75e33Timing
----- End Fatal Exception -------------------------------------------------

----- Begin Fatal Exception 21-Aug-2025 21:34:53 CEST-----------------------
An exception of category 'FileInPathError' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 1 stream: 0
   [1] Running path 'Phase2L1GTProducer'
   [2] Prefetching for module L1GTProducer/'l1tGTProducer'
   [3] Calling method for module L1MetPfProducer/'l1tMETPFProducer'
Exception Message:
edm::FileInPath unable to find file L1Trigger/Phase2L1ParticleFlow/data/met/l1met_ptphi2pxpy_poly2_v1.json anywhere in the search path.
The search path is defined by: CMSSW_SEARCH_PATH
${CMSSW_SEARCH_PATH} is: /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/48308/47825/CMSSW_15_1_X_2025-08-21-1100/poison:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/48308/47825/CMSSW_15_1_X_2025-08-21-1100/src:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/48308/47825/CMSSW_15_1_X_2025-08-21-1100/external/el8_amd64_gcc12/data:/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02903/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_15_1_X_2025-08-21-1100/poison:/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02903/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_15_1_X_2025-08-21-1100/src:/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02903/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_15_1_X_2025-08-21-1100/external/el8_amd64_gcc12/data
Current directory is: /data/cmsbld/jenkins/workspace/ib-run-pr-relvals/matrix-results/29834.999_TTbar_14TeV+Run4D110PU_PMXS1S2PR
----- End Fatal Exception -------------------------------------------------

----- Begin Fatal Exception 21-Aug-2025 21:37:53 CEST-----------------------
An exception of category 'FileInPathError' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 1 stream: 0
   [1] Running path 'Phase2L1GTProducer'
   [2] Prefetching for module L1GTProducer/'l1tGTProducer'
   [3] Calling method for module L1MetPfProducer/'l1tMETPFProducer'
Exception Message:
edm::FileInPath unable to find file L1Trigger/Phase2L1ParticleFlow/data/met/l1met_ptphi2pxpy_poly2_v1.json anywhere in the search path.
The search path is defined by: CMSSW_SEARCH_PATH
${CMSSW_SEARCH_PATH} is: /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/48308/47825/CMSSW_15_1_X_2025-08-21-1100/poison:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/48308/47825/CMSSW_15_1_X_2025-08-21-1100/src:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/48308/47825/CMSSW_15_1_X_2025-08-21-1100/external/el8_amd64_gcc12/data:/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02903/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_15_1_X_2025-08-21-1100/poison:/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02903/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_15_1_X_2025-08-21-1100/src:/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02903/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_15_1_X_2025-08-21-1100/external/el8_amd64_gcc12/data
Current directory is: /data/cmsbld/jenkins/workspace/ib-run-pr-relvals/matrix-results/24834.911_TTbar_14TeV+Run4D98_DD4hep
----- End Fatal Exception -------------------------------------------------
Expand to see more relval errors ...

cmsbuild avatar Aug 21 '25 21:08 cmsbuild

test parameters:

  • workflow = 29634.78
  • pull_request = cms-data/L1Trigger-Phase2L1ParticleFlow#9
  • relvals_opt = --what upgrade

BenjaminRS avatar Aug 22 '25 08:08 BenjaminRS

please test

BenjaminRS avatar Aug 22 '25 08:08 BenjaminRS

+1

Size: This PR adds an extra 16KB to repository Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-ab62aa/47828/summary.html COMMIT: 104769c67b15f252baaa6864962d051a686c6ccf CMSSW: CMSSW_15_1_X_2025-08-21-2300/el8_amd64_gcc12 User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/48308/47828/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

  • @alexandertuna cms-sw/cmssw#48750
  • @Dr15Jones cms-sw/cmssw#48779

You can see more details here: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-ab62aa/47828/git-recent-commits.json https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-ab62aa/47828/git-merge-result

Comparison Summary

Summary:

  • You potentially added 17 lines to the logs
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 51
  • DQMHistoTests: Total histograms compared: 4209453
  • DQMHistoTests: Total failures: 17
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 4209416
  • DQMHistoTests: Total skipped: 20
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 50 files compared)
  • Checked 220 log files, 188 edm output root files, 51 DQM output files
  • TriggerResults: no differences found

cmsbuild avatar Aug 22 '25 11:08 cmsbuild

@BenjaminRS Thank you for testing my code! Could you confirm both this and cms-data?

JunwonTomOh avatar Sep 01 '25 06:09 JunwonTomOh

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-48308/45951

Code check has found code style and quality issues which could be resolved by applying following patch(s)

  • code-format: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-48308/45951/code-format.patch e.g. curl -k https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-48308/45951/code-format.patch | patch -p1 You can also run scram build code-format to apply code format directly

cmsbuild avatar Sep 01 '25 13:09 cmsbuild

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-48308/45952

cmsbuild avatar Sep 01 '25 13:09 cmsbuild

Pull request #48308 was updated. @BenjaminRS, @Moanwar, @cmsbuild, @quinnanm, @srimanob, @subirsarkar can you please check and sign again.

cmsbuild avatar Sep 01 '25 13:09 cmsbuild

@cmsbuild please test

Moanwar avatar Sep 01 '25 15:09 Moanwar

+1

Size: This PR adds an extra 24KB to repository Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-ab62aa/47934/summary.html COMMIT: 88d6bf5ea35b98871e09d3f72abf0ac0acba3a41 CMSSW: CMSSW_15_1_X_2025-09-01-1100/el8_amd64_gcc12 User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/48308/47934/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

  • @fwyzard cms-sw/cmsdist#10034
  • @Annnnya cms-sw/cmsdist#10039
  • @cms-sw cms-sw/cmsdist#10051

You can see more details here: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-ab62aa/47934/git-recent-commits.json https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-ab62aa/47934/git-merge-result

Comparison Summary

Summary:

  • You potentially added 18 lines to the logs
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 51
  • DQMHistoTests: Total histograms compared: 4250721
  • DQMHistoTests: Total failures: 35
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 4250666
  • DQMHistoTests: Total skipped: 20
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 50 files compared)
  • Checked 220 log files, 188 edm output root files, 51 DQM output files
  • TriggerResults: no differences found

cmsbuild avatar Sep 02 '25 04:09 cmsbuild

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-48308/46034

cmsbuild avatar Sep 09 '25 15:09 cmsbuild

Pull request #48308 was updated. @BenjaminRS, @Moanwar, @cmsbuild, @quinnanm, @srimanob, @subirsarkar can you please check and sign again.

cmsbuild avatar Sep 09 '25 15:09 cmsbuild

please test

BenjaminRS avatar Sep 09 '25 20:09 BenjaminRS

+1

Size: This PR adds an extra 32KB to repository Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-ab62aa/48039/summary.html COMMIT: af82bf2ca97c7bb94e56f486bafb654a450f9bb9 CMSSW: CMSSW_15_1_X_2025-09-09-1100/el8_amd64_gcc12 User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/48308/48039/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

  • @tihsu99 cms-sw/cmssw#48850

You can see more details here: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-ab62aa/48039/git-recent-commits.json https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-ab62aa/48039/git-merge-result

Comparison Summary

Summary:

  • You potentially added 14 lines to the logs
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 51
  • DQMHistoTests: Total histograms compared: 4248921
  • DQMHistoTests: Total failures: 8
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 4248893
  • DQMHistoTests: Total skipped: 20
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 50 files compared)
  • Checked 220 log files, 188 edm output root files, 51 DQM output files
  • TriggerResults: no differences found

cmsbuild avatar Sep 09 '25 22:09 cmsbuild

Milestone for this pull request has been moved to CMSSW_16_0_X. Please open a backport if it should also go in to CMSSW_15_1_X.

cmsbuild avatar Sep 10 '25 06:09 cmsbuild

+l1

BenjaminRS avatar Sep 10 '25 13:09 BenjaminRS