cms-sw
[14_1_X] Added more ZDC functionality to DQM
Code reformatted, emulator updates, and histogram title change
PR description:
This is a backport to CMSSW_14_1_X of existing PR here: https://github.com/cms-sw/cmssw/pull/45948
The PR adds to the DQM for ZDC specific histograms, via ZDCQIE10Task
The backport is updated relative to the main PR, mostly fixing build issue, retitling some histograms, and updating python such that emulated ZDC sums fill. It is also rebased to 14_1_X for the backport.
PR validation:
PR is tested in CMSSW_14_1_X_2024-10-15-2300, after running a merge-topic to incorporate the PR into a fresh area scram b ran successfully, validating the build
Following test command produces the expected output:
cmsRun DQM/Integration/python/clients/hcal_dqm_sourceclient-live_cfg.py inputFiles=/eos/cms/store/group/dpg_hcal/comm_hcal/PFG/backup_raw/HIHLTPhysics-HIRun2023A-RAW-v1-375754.root runkey=hi_run
under the upload directory. Inspected histograms inside the DQM file appear reasonable for ZDC sums.
In addition, the following set of commands are run scram b runtests scram build code-checks scram build code-format
code-checks revealed no issues, code-format suggestions were adopted. runtests had multiple failures, detailed below - however, every failure existed already with runtests in a fresh area of CMSSW_14_1_X_2024-10-15-2300 with the same packages checked out (before merge-topic), so it is unclear it is the result of the PR
>> Local Products Rules ..... started
>> Local Products Rules ..... done
Creating test log file logs/el9_amd64_gcc12/testing.log
Fail 26s ... DQM/Integration/TestDQMOnlineClient-beam_dqm_sourceclient
Pass 38s ... DQM/Integration/TestDQMOnlineClient-beamhlt_dqm_sourceclient
Fail 6s ... DQM/Integration/TestDQMOnlineClient-beampixel_dqm_sourceclient
Fail 49s ... DQM/Integration/TestDQMOnlineClient-csc_dqm_sourceclient
Fail 7s ... DQM/Integration/TestDQMOnlineClient-ctpps_dqm_sourceclient
Fail 6s ... DQM/Integration/TestDQMOnlineClient-dt4ml_dqm_sourceclient
Fail 6s ... DQM/Integration/TestDQMOnlineClient-dt_dqm_sourceclient
Fail 6s ... DQM/Integration/TestDQMOnlineClient-ecal_dqm_sourceclient
Pass 17s ... DQM/Integration/TestDQMOnlineClient-ecalgpu_dqm_sourceclient
Fail 7s ... DQM/Integration/TestDQMOnlineClient-es_dqm_sourceclient
Fail 6s ... DQM/Integration/TestDQMOnlineClient-fed_dqm_sourceclient
Fail 7s ... DQM/Integration/TestDQMOnlineClient-gem_dqm_sourceclient
Fail 6s ... DQM/Integration/TestDQMOnlineClient-hcal_dqm_sourceclient
Pass 10s ... DQM/Integration/TestDQMOnlineClient-hcalgpu_dqm_sourceclient
Fail 6s ... DQM/Integration/TestDQMOnlineClient-hcalreco_dqm_sourceclient
Fail 6s ... DQM/Integration/TestDQMOnlineClient-hlt_dqm_clientPB
Fail 6s ... DQM/Integration/TestDQMOnlineClient-hlt_dqm_sourceclient
Fail 5s ... DQM/Integration/TestDQMOnlineClient-info_dqm_sourceclient
Fail 6s ... DQM/Integration/TestDQMOnlineClient-l1tstage2_dqm_sourceclient
Fail 6s ... DQM/Integration/TestDQMOnlineClient-l1tstage2emulator_dqm_sourceclient
Fail 6s ... DQM/Integration/TestDQMOnlineClient-mutracking_dqm_sourceclient
Pass 9s ... DQM/Integration/TestDQMOnlineClient-onlinebeammonitor_dqm_sourceclient
Pass 7s ... DQM/Integration/TestDQMOnlineClient-pfgpu_dqm_sourceclient
Fail 6s ... DQM/Integration/TestDQMOnlineClient-pixel_dqm_sourceclient
Pass 10s ... DQM/Integration/TestDQMOnlineClient-pixelgpu_dqm_sourceclient
Fail 5s ... DQM/Integration/TestDQMOnlineClient-pixellumi_dqm_sourceclient
Fail 6s ... DQM/Integration/TestDQMOnlineClient-rpc_dqm_sourceclient
Fail 5s ... DQM/Integration/TestDQMOnlineClient-scal_dqm_sourceclient
Pass 33s ... DQM/Integration/TestDQMOnlineClient-sistrip_approx_dqm_sourceclient
Fail 6s ... DQM/Integration/TestDQMOnlineClient-sistrip_dqm_sourceclient
Fail 6s ... DQM/Integration/TestDQMOnlineClient-visualization
Fail 6s ... DQM/Integration/TestDQMOnlineClient-visualization_secondInstance
Pass 172s ... DQM/Integration/TestDQMOnlineClients-compilation_commissioning_run
Pass 142s ... DQM/Integration/TestDQMOnlineClients-compilation_cosmic_run
Pass 145s ... DQM/Integration/TestDQMOnlineClients-compilation_cosmic_run_stage1
Pass 155s ... DQM/Integration/TestDQMOnlineClients-compilation_hi_run
Pass 152s ... DQM/Integration/TestDQMOnlineClients-compilation_hpu_run
Pass 149s ... DQM/Integration/TestDQMOnlineClients-compilation_pp_run
Pass 131s ... DQM/Integration/TestDQMOnlineClients-compilation_pp_run_stage1
>> Test sequence completed for CMSSW CMSSW_14_1_X_2024-10-15-2300
If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:
As mentioned above, PR is a backport (with modification) of https://github.com/cms-sw/cmssw/pull/45948, intended for CMSSW_14_1_X
One note - close inspection of the changes to the python file would be welcome - the changes were made as part of testing the full package but unclear to me all should be incorporated into the branch as is. Guidance appreciated.
@hjbossi @mandrenguyen
A new Pull Request was created by @cfmcginn for CMSSW_14_1_X.
It involves the following packages:
- DQM/HcalTasks (dqm)
- DQM/Integration (dqm)
@antoniovagnerini, @cmsbuild, @nothingface0, @rvenditti, @syuvivida, @tjavaid can you please review it and eventually sign? Thanks. @DryRun, @abdoulline, @batinkov, @bsunanda, @denizsun, @francescobrivio, @salimcerci, @threus this is something you requested to watch as well. @antoniovilela, @mandrenguyen, @rappoccio, @sextonkennedy you are the release manager for this.
cms-bot commands are listed here
- Backported from #46478
+1
Size: This PR adds an extra 24KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-a9b9d8/42239/summary.html
COMMIT: 2b258ab4b455afa1fd93792acfda48a2563459a2
CMSSW: CMSSW_14_1_X_2024-10-16-1100/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/46407/42239/install.sh to create a dev area with all the needed externals and cmssw changes.
Comparison Summary
Summary:
- You potentially added 1 lines to the logs
- Reco comparison results: 11 differences found in the comparisons
- DQMHistoTests: Total files compared: 44
- DQMHistoTests: Total histograms compared: 3331166
- DQMHistoTests: Total failures: 6
- DQMHistoTests: Total nulls: 0
- DQMHistoTests: Total successes: 3331140
- DQMHistoTests: Total skipped: 20
- DQMHistoTests: Total Missing objects: 0
- DQMHistoSizes: Histogram memory added: 0.0 KiB( 43 files compared)
- Checked 191 log files, 161 edm output root files, 44 DQM output files
- TriggerResults: no differences found
Just to add another tag for relevant reviewers: @denizsun, @salimcerci, @JoshPBR2, @abdoulline. This is a modification of of the original PR, which we intend to update with the changes once @JoshPBR2 is available to do so. The current version of the release is 14_1_3, built yesterday. If possible, we would love for this to be included the next release, i.e. 14_1_4.
As @cfmcginn mentions, it would be good to understand what is most appropriate to do for the source client file. The reason why we have questions is that I am not entirely sure what is used to create the plots online vs. local tests. Please let us know what you think!
This PR is different from the master one: #45948, thus it's not a backport. You need to close that one and open another one that contains the same changes as this one in the master (CMSSW_14_2_X) cycle.
Hi @mmusich, thanks for your comment! The plan is to update the PR to master (CMSSW_14_2_X) cycle to have the same changes as the one here, i.e. this would be a true backport. Unfortunately, we've been unable to get ahold of Josh (I believe he might be traveling to CERN), and he is needed in order to make this change. We wanted to give ample time for review of these changes, so we decided to open the PR to 14_1_X as soon as possible. Let me know what you think of this strategy.
Let me know what you think of this strategy.
well, abandoning the unattended PR and opening a new one from scratch will get you merged faster. This one cannot be integrated until the master one is merged.
Let me know what you think of this strategy.
well, abandoning the unattended PR and opening a new one from scratch will get you merged faster. This one cannot be integrated until the master one is merged.
Okay I see, in that case we will see if Josh gets back to us today, and if not, we will close the master PR and open a new one.
Let me know what you think of this strategy.
well, abandoning the unattended PR and opening a new one from scratch will get you merged faster. This one cannot be integrated until the master one is merged.
Okay I see, in that case we will see if Josh gets back to us today, and if not, we will close the master PR and open a new one.
@hjbossi any updates on the comments by Marco? At this stage, I would also recommend closing the master https://github.com/cms-sw/cmssw/pull/45948 and making a new one that matches the backport.
Let me know what you think of this strategy.
well, abandoning the unattended PR and opening a new one from scratch will get you merged faster. This one cannot be integrated until the master one is merged.
Okay I see, in that case we will see if Josh gets back to us today, and if not, we will close the master PR and open a new one.
@hjbossi any updates on the comments by Marco? At this stage, I would also recommend closing the master #45948 and making a new one that matches the backport.
I intend to open the new PR to main today; I don't know that I have power to close the old, but that should follow
Looking at the master PR https://github.com/cms-sw/cmssw/pull/45948, I don't think I have permissions to close it, which means that must be done by @JoshPBR2. Although maybe someone with less restricted permissions can do so?
Pull request #46407 was updated. @antoniovagnerini, @cmsbuild, @nothingface0, @rvenditti, @syuvivida, @tjavaid can you please check and sign again.
Not all of the requested changes in https://github.com/cms-sw/cmssw/pull/46478 have been incorporated here (because of the history of how this was setup) - I will incorporate them now (just the last commit including some header removals and reversion of a bool to false, can provide more details as needed)
Pull request #46407 was updated. @antoniovagnerini, @cmsbuild, @nothingface0, @rvenditti, @syuvivida, @tjavaid can you please check and sign again.
Backport is now up-to-date w/ main merge, ready for testing. Note commits are not squashed, can squash at request
+1
Size: This PR adds an extra 16KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-a9b9d8/42396/summary.html
COMMIT: 61c0a4381c4ea9d40cc21d7806bfa34cee0b4b60
CMSSW: CMSSW_14_1_X_2024-10-25-1100/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/46407/42396/install.sh to create a dev area with all the needed externals and cmssw changes.
Comparison Summary
Summary:
- You potentially added 1 lines to the logs
- Reco comparison results: 0 differences found in the comparisons
- DQMHistoTests: Total files compared: 46
- DQMHistoTests: Total histograms compared: 3562835
- DQMHistoTests: Total failures: 0
- DQMHistoTests: Total nulls: 0
- DQMHistoTests: Total successes: 3562815
- DQMHistoTests: Total skipped: 20
- DQMHistoTests: Total Missing objects: 0
- DQMHistoSizes: Histogram memory added: 0.0 KiB( 45 files compared)
- Checked 199 log files, 169 edm output root files, 46 DQM output files
- TriggerResults: no differences found
This pull request is fully signed and it will be integrated in one of the next CMSSW_14_1_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_14_2_X is complete. This pull request will now be reviewed by the release team before it's merged. @mandrenguyen, @rappoccio, @sextonkennedy, @antoniovilela (and backports should be raised in the release meeting by the corresponding L2)