cms-sw
track selection DNN update and relative working points
PR description:
- the track selection DNN is updated with a retraining under more recent Run3 conditions (CMSSW 12_5_0_pre5) and relative working points
- two trainings are performed: mkFit for 7 iterations + CKF for the rest and CKF for all the iterations
- for the current tracking, where mkFit is used in 4 iterations + InitialStepPreSplitting (5), the first DNN is used for the 4 mkFit iterations (initial , high pt triplet, detached quad and triplet) and the iterations where both trainings use CKF (pixel pair, mixed triplet, tobtec, jetCore), while the CKF training and working points is used as a fallback for pixelLess, low pT quad and triplet, trained with mkFit in the first training.
- the update was presented at the TRK POG -> link
PR validation:
the MTV plots for TTbar and QCD high pT (1800-2400) are linked below
-
default tracking TTbar default QCD default
NB: the cand DNN cut for mkFit high pT triplet is kept the same, as it doesn't change the performance (same as before retraining)
for testing one needs to include this PR to cms-data https://github.com/cms-data/RecoTracker-FinalTrackSelectors/pull/12
+code-checks
Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-39715/32547
- This PR adds an extra 24KB to repository
A new Pull Request was created by @leonardogiannini for master.
It involves the following packages:
- RecoTracker/FinalTrackSelectors (reconstruction)
- RecoTracker/IterativeTracking (reconstruction)
@cmsbuild, @mandrenguyen, @clacaputo can you please review it and eventually sign? Thanks. @VourMa, @felicepantaleo, @GiacomoSguazzoni, @JanFSchulte, @rovere, @VinInn, @missirol, @ebrondol, @gpetruc, @mmusich, @mtosi, @dgulhan this is something you requested to watch as well. @perrotta, @dpiparo, @rappoccio you are the release manager for this.
cms-bot commands are listed here
test parameters:
- pull_request = cms-data/RecoTracker-FinalTrackSelectors#12
+1
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-06172f/28219/summary.html
COMMIT: bef89f3fc712b2cfb2bfe94da31bea382a8649b5
CMSSW: CMSSW_12_6_X_2022-10-12-2300/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/39715/28219/install.sh to create a dev area with all the needed externals and cmssw changes.
The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:
- @bsunanda cms-sw/cmssw#39687
You can see more details here: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-06172f/28219/git-recent-commits.json https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-06172f/28219/git-merge-result
Comparison Summary
@slava77 comparisons for the following workflows were not done due to missing matrix map:
- /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-06172f/20834.0_TTbar_14TeV+2026D88+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14+DigiTrigger+RecoGlobal+HARVESTGlobal
- /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-06172f/20834.75_TTbar_14TeV+2026D88_HLT75e33+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14+DigiTrigger+RecoGlobal+HLT75e33+HARVESTGlobal
- /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-06172f/20896.0_CloseByPGun_CE_E_Front_120um+2026D88+CE_E_Front_120um_GenSimHLBeamSpotHGCALCloseBy+DigiTrigger+RecoGlobal+HARVESTGlobal
- /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-06172f/20900.0_CloseByPGun_CE_H_Coarse_Scint+2026D88+CE_H_Coarse_Scint_GenSimHLBeamSpotHGCALCloseBy+DigiTrigger+RecoGlobal+HARVESTGlobal
- /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-06172f/21034.999_TTbar_14TeV+2026D88PU_PMXS1S2PR+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14+PREMIX_PremixHLBeamSpot14PU+DigiTriggerPU+RecoGlobalPU+HARVESTGlobalPU
- /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-06172f/23234.0_TTbar_14TeV+2026D94+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14+DigiTrigger+RecoGlobal+HARVESTGlobal
Summary:
- No significant changes to the logs found
- Reco comparison results: 20456 differences found in the comparisons
- DQMHistoTests: Total files compared: 49
- DQMHistoTests: Total histograms compared: 3392309
- DQMHistoTests: Total failures: 111888
- DQMHistoTests: Total nulls: 2
- DQMHistoTests: Total successes: 3280397
- DQMHistoTests: Total skipped: 22
- DQMHistoTests: Total Missing objects: 0
- DQMHistoSizes: Histogram memory added: -0.016 KiB( 48 files compared)
- DQMHistoSizes: changed ( 138.5 ): -0.016 KiB JetMET/SUSYDQM
- Checked 204 log files, 49 edm output root files, 49 DQM output files
- TriggerResults: found differences in 2 / 48 workflows
Hi, I'm checking the reco differences and they are minimal. I see a ~1% decrease in the number of general tracks, maybe it is expected, could you please confirm it?
Hi, I'm checking the reco differences and they are minimal. I see a ~1% decrease in the number of general tracks, maybe it is expected, could you please confirm it?
which workflow is this? In events with PU the number of fake tracks is expected to be lower, as shown in the slides linked in the PR description. A link to the internal validation (available in the slides), somewhat representative to the inclusive distribution you have picked up is in http://uaf-10.t2.ucsd.edu/~legianni/DNN-retraining-eval/MKWP/TTbar_epoch4-4/plots_ootb/distPtEtaPhi.pdf
which workflow is this?
Sorry, it's
11834p0
OK, then my link is supposed to be equivalent (ttbar with PU in run3)
which workflow is this?
Sorry, it's
11834p0OK, then my link is supposed to be equivalent (ttbar with PU in run3)
Indeed, I can see the decrease in number of tracks, thanks
+reconstruction
- track selection DNN update
- new model in https://github.com/cms-data/RecoTracker-FinalTrackSelectors/pull/12
- reco changes expected
This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)
@clacaputo do you mind also signing the external (https://github.com/cms-data/RecoTracker-FinalTrackSelectors/pull/12) and then this can be merged.
