cmssw Refactored how begin/end/Job/Stream are implemented

PR description:

Moved transition calls from Worker to ModuleHolder
The implementation is now done using stand-alone functions using ModuleRegistry
Cleaned up some unused functions

The only changes expected are

some framework internal modules now have ActivityRegistry signals happen for them
the order in which modules are called during begin/end/Job/Stream may have changed

PR validation:

Code compiles. All framework unit tests pass.

Jun 25 '25 20:06 Dr15Jones

@wddgit please review

Jun 25 '25 20:06 Dr15Jones

cms-bot internal usage

Jun 25 '25 20:06 cmsbuild

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-48416/45315

There are other open Pull requests which might conflict with changes you have proposed:
- File FWCore/Integration/plugins/ExceptionThrowingProducer.cc modified in PR(s): #44372

Code check has found code style and quality issues which could be resolved by applying following patch(s)

code-format: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-48416/45315/code-format.patch e.g. curl -k https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-48416/45315/code-format.patch | patch -p1 You can also run scram build code-format to apply code format directly

Jun 25 '25 20:06 cmsbuild

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-48416/45316

There are other open Pull requests which might conflict with changes you have proposed:
- File FWCore/Integration/plugins/ExceptionThrowingProducer.cc modified in PR(s): #44372

Jun 25 '25 21:06 cmsbuild

A new Pull Request was created by @Dr15Jones for master.

It involves the following packages:

FWCore/Framework (core)
FWCore/Integration (core)
FWCore/ServiceRegistry (core)
FWCore/TestProcessor (core)
Mixing/Base (simulation)

@Dr15Jones, @civanch, @cmsbuild, @kpedro88, @makortel, @mdhildreth, @smuzaffar can you please review it and eventually sign? Thanks. @fabiocos, @fwyzard, @makortel, @missirol, @wddgit this is something you requested to watch as well. @antoniovilela, @mandrenguyen, @rappoccio, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

Jun 25 '25 21:06 cmsbuild

I'll start reviewing this tomorrow.

Jun 25 '25 21:06 wddgit

please test

Jun 25 '25 21:06 Dr15Jones

please abort test

Jun 25 '25 21:06 Dr15Jones

test parameters:

full = true

Jun 25 '25 21:06 Dr15Jones

please test

Jun 25 '25 21:06 Dr15Jones

-1

Failed Tests: ClangBuild Size: This PR adds an extra 476KB to repository Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-1d8536/46925/summary.html COMMIT: a5e25c665ba166e6b402332e98a0d89398509e00 CMSSW: CMSSW_15_1_X_2025-06-25-1100/el8_amd64_gcc12 User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/48416/46925/install.sh to create a dev area with all the needed externals and cmssw changes.

Clang Build

I found compilation warning while trying to compile with clang. Command used:

USER_CUDA_FLAGS='--expt-relaxed-constexpr' USER_CXXFLAGS='-Wno-register -fsyntax-only' /usr/bin/time -v scram build -k -j 32 COMPILER='llvm compile'

See details on the summary page.

Jun 25 '25 23:06 cmsbuild

please test

Jun 26 '25 13:06 Dr15Jones

please abort test

Jun 26 '25 13:06 Dr15Jones

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-48416/45324

There are other open Pull requests which might conflict with changes you have proposed:
- File FWCore/Integration/plugins/ExceptionThrowingProducer.cc modified in PR(s): #44372

Jun 26 '25 13:06 cmsbuild

Pull request #48416 was updated. @Dr15Jones, @civanch, @cmsbuild, @kpedro88, @makortel, @mdhildreth, @smuzaffar can you please check and sign again.

Jun 26 '25 13:06 cmsbuild

test parameters:

full = true

Jun 26 '25 13:06 Dr15Jones

please test

Jun 26 '25 13:06 Dr15Jones

-1

Failed Tests: UnitTests Size: This PR adds an extra 16KB to repository Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-1d8536/46934/summary.html COMMIT: 2c3bcc42237efc1d7e89e4f2daca2b0606b4f35c CMSSW: CMSSW_15_1_X_2025-06-26-1100/el8_amd64_gcc12 User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/48416/46934/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found 1 errors in the following unit tests:

---> test TestPerfToolsModuleAllocMonitor had ERRORS

Comparison Summary

Summary:

You potentially removed 160 lines from the logs
Reco comparison results: 10 differences found in the comparisons
DQMHistoTests: Total files compared: 50
DQMHistoTests: Total histograms compared: 4065619
DQMHistoTests: Total failures: 51
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 4065548
DQMHistoTests: Total skipped: 20
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
Checked 215 log files, 184 edm output root files, 50 DQM output files
TriggerResults: no differences found

Jun 26 '25 18:06 cmsbuild

please test

Jun 26 '25 18:06 Dr15Jones

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-48416/45332

There are other open Pull requests which might conflict with changes you have proposed:
- File FWCore/Integration/plugins/ExceptionThrowingProducer.cc modified in PR(s): #44372

Jun 26 '25 18:06 cmsbuild

Pull request #48416 was updated. @Dr15Jones, @civanch, @kpedro88, @makortel, @mdhildreth, @smuzaffar can you please check and sign again.

Jun 26 '25 18:06 cmsbuild

+1

Size: This PR adds an extra 20KB to repository Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-1d8536/46939/summary.html COMMIT: b00dec551c0656e9a8f6091025fef75c27a49113 CMSSW: CMSSW_15_1_X_2025-06-26-1100/el8_amd64_gcc12 User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/48416/46939/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially removed 159 lines from the logs
Reco comparison results: 10 differences found in the comparisons
DQMHistoTests: Total files compared: 50
DQMHistoTests: Total histograms compared: 4065619
DQMHistoTests: Total failures: 51
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 4065548
DQMHistoTests: Total skipped: 20
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
Checked 215 log files, 184 edm output root files, 50 DQM output files
TriggerResults: no differences found

Jun 27 '25 00:06 cmsbuild

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-48416/45342

There are other open Pull requests which might conflict with changes you have proposed:
- File FWCore/Integration/plugins/ExceptionThrowingProducer.cc modified in PR(s): #44372

Jun 27 '25 13:06 cmsbuild

Pull request #48416 was updated. @Dr15Jones, @civanch, @cmsbuild, @kpedro88, @makortel, @mdhildreth, @smuzaffar can you please check and sign again.

Jun 27 '25 13:06 cmsbuild

@wddgit if you could just double check the exception handling changes to ModuleRegistryUtilities.cc that would be great.

Jun 27 '25 13:06 Dr15Jones

please test

Jun 27 '25 13:06 Dr15Jones

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-48416/45352

There are other open Pull requests which might conflict with changes you have proposed:
- File FWCore/Integration/plugins/ExceptionThrowingProducer.cc modified in PR(s): #44372

Jun 27 '25 17:06 cmsbuild

Pull request #48416 was updated. @Dr15Jones, @civanch, @cmsbuild, @kpedro88, @makortel, @mdhildreth, @smuzaffar can you please check and sign again.

Jun 27 '25 17:06 cmsbuild

please test

Jun 27 '25 17:06 Dr15Jones

+1

Logic looks good. Just the one minor comment I just submitted. Overall, looks good now.

Jun 27 '25 18:06 wddgit