Matti Kortelainen
Matti Kortelainen
assign geometry, upgrade (let's start with these, since the problem seems to be specific to Phase 2 DD4Hep workflow)
The UBSAN log points to this line https://github.com/cms-sw/cmssw/blob/a35549c2f1deeec2f9bb8be7d39baa4f39f4f76b/Geometry/HcalCommonData/src/HcalDDDRecConstants.cc#L173
Occurred in * CMSSW_12_6_X_2022-09-19-1100 el8_amd64_gcc10 as a timeout in step 2 https://cmssdt.cern.ch/SDT/cgi-bin/logreader/el8_amd64_gcc10/CMSSW_12_6_X_2022-09-19-1100/pyRelValMatrixLogs/run/39434.911_TTbar_14TeV+2026D88_DD4hep+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14+DigiTrigger+RecoGlobal+HARVESTGlobal/step2_TTbar_14TeV+2026D88_DD4hep+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14+DigiTrigger+RecoGlobal+HARVESTGlobal.log#/ * CMSSW_12_6_X_2022-09-19-2300 el8_amd64_gcc10 in step 3 https://cmssdt.cern.ch/SDT/cgi-bin/logreader/el8_amd64_gcc10/CMSSW_12_6_X_2022-09-19-2300/pyRelValMatrixLogs/run/39434.911_TTbar_14TeV+2026D88_DD4hep+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14+DigiTrigger+RecoGlobal+HARVESTGlobal/step3_TTbar_14TeV+2026D88_DD4hep+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14+DigiTrigger+RecoGlobal+HARVESTGlobal.log#/ both pointing to `HcalDDDRecConstants::getHCID()`.
ASAN gives something helpful in CMSSW_12_6_ASAN_X_2022-09-19-1100 step 2 ``` ==139802==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x606007e5311c at pc 0x2b5a2eb95034 bp 0x2b5a51fedbf0 sp 0x2b5a51fedbe8 READ of size 4 at 0x606007e5311c thread T2...
> CMSSW_12_6_X_2022-09-15-2300 seems to be the first IB where these crashes appeared. The `HcalDDDRecConstants::getHCID()` seems to play some role in most of them. None of the PRs merged in CMSSW_12_6_X_2022-09-15-2300...
In normal IBs the crashes occur randomly, so you may have to try to run many times or try to load the machine. Maybe try to run on an ASAN...
This problem got fixed by https://github.com/cms-sw/cmssw/pull/39967
@cmsbuild, please close
assign core,xpog
Given that by default a GPU would be used if it is available, maybe it would be time to make the loading of `Configuration.StandardSequecens.Accelerators_cff` unconditional https://github.com/cms-sw/cmssw/blob/284681e89ac822d328cf54cfe57866c475fae9e4/Configuration/StandardSequences/python/Services_cff.py#L11-L18 either by unconditional `process.load("Configuration.StandardSequecens.Accelerators_cff")`...