cmssw
cmssw copied to clipboard
segmentation faults in hcalIsoTrkAnalyzer_cff.py
one segmentation fault occurs when including the 'run3_data' global tag. This is the output ---
03-Jul-2022 11:00:32 CEST Initiating request to open file root://eoscms.cern.ch//eos/cms/store/data/Run2022A/Commissioning/ALCARECO/HcalCalIsoTrk-PromptReco-v1/000/352/322/00000/51b3d367-ab3f-4a1b-b157-31fe3e76eedf.root %MSG-w XrdAdaptor: file_open 03-Jul-2022 11:00:34 CEST pre-events Data is served from cern.ch instead of original site eoscms %MSG 03-Jul-2022 11:00:34 CEST Successfully opened file root://eoscms.cern.ch//eos/cms/store/data/Run2022A/Commissioning/ALCARECO/HcalCalIsoTrk-PromptReco-v1/000/352/322/00000/51b3d367-ab3f-4a1b-b157-31fe3e76eedf.root Labels used InputTag: label = hltTriggerSummaryAOD, instance = , process = HLT InputTag: label = TriggerResults, instance = , process = HLT offlineBeamSpot offlinePrimaryVertices generalTracks InputTag: label = ecalRecHit, instance = EcalRecHitsEB InputTag: label = ecalRecHit, instance = EcalRecHitsEE hbhereco towerMaker muons Parameters read from config file minPt 1 theTrackQuality highPurity minQuality 2 maxDxyPV 0.02 maxDzPV 0.02 maxChi2 5 maxDpOverP 0.1 minOuterHit 4 minLayerCrossed 8 maxInMiss 0 maxOutMiss 0 a_coneR 34.98:44.98:64.98 a_charIsoR 63.88 a_mipR 14 a_mipR2 18 a_mipR3 20 a_mipR4 22 a_mipR5 24 pTrackMin_ 10 eEcalMax_ 2 maxRestrictionP_ 8 slopeRestrictionP_ 0.050905 eIsolateStrong_ 2 eIsolateSoft_ 10 hcalScale_ 1 momentumLow_ 40 prescaleLow_ 1 momentumHigh_ 60 prescaleHigh_ 1 useRaw_ 0 ignoreTrigger_ 1 useL1Trigegr_ 0 dataType_ 0 mode_ 11 unCorrect_ 0 collapseDepth_ 0 L1TrigName_ L1_SingleJet60 Threshold flag used 1 value for EB 0.18 EE -206.074:357.671:-204.978:39.033:1.25:10 and 0 events to be debugged Process HLT L1Filter: L2Filter:L2Filter L3Filter:Filter 0 DetIDs to be corrected with HEP17 flag:0 EtaDist 0.6 0.3 PhiDist 0.628319 0.314159 phibin_[0] -2.82743 phibin_[1] -2.19911 phibin_[2] -1.5708 phibin_[3] -0.942478 phibin_[4] -0.314159 phibin_[5] 0.314159 phibin_[6] 0.942478 phibin_[7] 1.5708 phibin_[8] 2.19911 phibin_[9] 2.82743 etabin_[0] -2.1 etabin_[1] -1.5 etabin_[2] -0.9 etabin_[3] -0.3 etabin_[4] 0.3 etabin_[5] 0.9 etabin_[6] 1.5 etabin_[7] 2.1 03-Jul-2022 11:00:36 CEST Closed file root://eoscms.cern.ch//eos/cms/store/data/Run2022A/Commissioning/ALCARECO/HcalCalIsoTrk-PromptReco-v1/000/352/322/00000/51b3d367-ab3f-4a1b-b157-31fe3e76eedf.root 03-Jul-2022 11:00:36 CEST Initiating request to open file root://eoscms.cern.ch//eos/cms/store/data/Run2022A/Commissioning/ALCARECO/HcalCalIsoTrk-PromptReco-v1/000/352/336/00000/170c5bf1-cdca-421d-9305-2180291699e7.root %MSG-w XrdAdaptor: file_open 03-Jul-2022 11:00:38 CEST PostStreamBeginLumi Data is served from cern.ch instead of original site eoscms %MSG 03-Jul-2022 11:00:38 CEST Successfully opened file root://eoscms.cern.ch//eos/cms/store/data/Run2022A/Commissioning/ALCARECO/HcalCalIsoTrk-PromptReco-v1/000/352/336/00000/170c5bf1-cdca-421d-9305-2180291699e7.root endRun[1] 352322 03-Jul-2022 11:00:39 CEST Closed file root://eoscms.cern.ch//eos/cms/store/data/Run2022A/Commissioning/ALCARECO/HcalCalIsoTrk-PromptReco-v1/000/352/336/00000/170c5bf1-cdca-421d-9305-2180291699e7.root 03-Jul-2022 11:00:39 CEST Initiating request to open file root://eoscms.cern.ch//eos/cms/store/data/Run2022A/Commissioning/ALCARECO/HcalCalIsoTrk-PromptReco-v1/000/352/352/00000/6700274f-20c8-4c07-abf9-5c39d6f0d6ec.root %MSG-w XrdAdaptor: file_open 03-Jul-2022 11:00:41 CEST PostStreamBeginLumi Data is served from cern.ch instead of original site eoscms %MSG 03-Jul-2022 11:00:41 CEST Successfully opened file root://eoscms.cern.ch//eos/cms/store/data/Run2022A/Commissioning/ALCARECO/HcalCalIsoTrk-PromptReco-v1/000/352/352/00000/6700274f-20c8-4c07-abf9-5c39d6f0d6ec.root endRun[2] 352336 03-Jul-2022 11:00:41 CEST Closed file root://eoscms.cern.ch//eos/cms/store/data/Run2022A/Commissioning/ALCARECO/HcalCalIsoTrk-PromptReco-v1/000/352/352/00000/6700274f-20c8-4c07-abf9-5c39d6f0d6ec.root 03-Jul-2022 11:00:41 CEST Initiating request to open file root://eoscms.cern.ch//eos/cms/store/data/Run2022A/Commissioning/ALCARECO/HcalCalIsoTrk-PromptReco-v1/000/352/371/00000/89ea2fbb-03e5-49fa-bcd0-bb6b2557c3ec.root %MSG-w XrdAdaptor: file_open 03-Jul-2022 11:00:43 CEST PostStreamBeginLumi Data is served from cern.ch instead of original site eoscms %MSG 03-Jul-2022 11:00:43 CEST Successfully opened file root://eoscms.cern.ch//eos/cms/store/data/Run2022A/Commissioning/ALCARECO/HcalCalIsoTrk-PromptReco-v1/000/352/371/00000/89ea2fbb-03e5-49fa-bcd0-bb6b2557c3ec.root endRun[3] 352352 03-Jul-2022 11:00:43 CEST Closed file root://eoscms.cern.ch//eos/cms/store/data/Run2022A/Commissioning/ALCARECO/HcalCalIsoTrk-PromptReco-v1/000/352/371/00000/89ea2fbb-03e5-49fa-bcd0-bb6b2557c3ec.root 03-Jul-2022 11:00:43 CEST Initiating request to open file root://eoscms.cern.ch//eos/cms/store/data/Run2022A/Commissioning/ALCARECO/HcalCalIsoTrk-PromptReco-v1/000/352/373/00000/d10fe6e5-fb66-4a4d-9456-6ce1c3be3484.root %MSG-w XrdAdaptor: file_open 03-Jul-2022 11:00:45 CEST PostStreamBeginLumi Data is served from cern.ch instead of original site eoscms %MSG 03-Jul-2022 11:00:45 CEST Successfully opened file root://eoscms.cern.ch//eos/cms/store/data/Run2022A/Commissioning/ALCARECO/HcalCalIsoTrk-PromptReco-v1/000/352/373/00000/d10fe6e5-fb66-4a4d-9456-6ce1c3be3484.root endRun[4] 352371 03-Jul-2022 11:00:46 CEST Closed file root://eoscms.cern.ch//eos/cms/store/data/Run2022A/Commissioning/ALCARECO/HcalCalIsoTrk-PromptReco-v1/000/352/373/00000/d10fe6e5-fb66-4a4d-9456-6ce1c3be3484.root 03-Jul-2022 11:00:46 CEST Initiating request to open file root://eoscms.cern.ch//eos/cms/store/data/Run2022A/Commissioning/ALCARECO/HcalCalIsoTrk-PromptReco-v1/000/352/415/00000/6b0aa2d3-c041-4d34-bcf0-895c6d17df10.root %MSG-w XrdAdaptor: file_open 03-Jul-2022 11:00:48 CEST PostStreamBeginLumi Data is served from cern.ch instead of original site eoscms %MSG 03-Jul-2022 11:00:48 CEST Successfully opened file root://eoscms.cern.ch//eos/cms/store/data/Run2022A/Commissioning/ALCARECO/HcalCalIsoTrk-PromptReco-v1/000/352/415/00000/6b0aa2d3-c041-4d34-bcf0-895c6d17df10.root endRun[5] 352373 03-Jul-2022 11:00:48 CEST Closed file root://eoscms.cern.ch//eos/cms/store/data/Run2022A/Commissioning/ALCARECO/HcalCalIsoTrk-PromptReco-v1/000/352/415/00000/6b0aa2d3-c041-4d34-bcf0-895c6d17df10.root 03-Jul-2022 11:00:48 CEST Initiating request to open file root://eoscms.cern.ch//eos/cms/store/data/Run2022A/Commissioning/ALCARECO/HcalCalIsoTrk-PromptReco-v1/000/352/416/00000/33820bb3-8a73-465f-b098-cd61edcf256e.root %MSG-w XrdAdaptor: file_open 03-Jul-2022 11:00:50 CEST PostStreamBeginLumi Data is served from cern.ch instead of original site eoscms %MSG 03-Jul-2022 11:00:50 CEST Successfully opened file root://eoscms.cern.ch//eos/cms/store/data/Run2022A/Commissioning/ALCARECO/HcalCalIsoTrk-PromptReco-v1/000/352/416/00000/33820bb3-8a73-465f-b098-cd61edcf256e.root endRun[6] 352415 03-Jul-2022 11:00:50 CEST Closed file root://eoscms.cern.ch//eos/cms/store/data/Run2022A/Commissioning/ALCARECO/HcalCalIsoTrk-PromptReco-v1/000/352/416/00000/33820bb3-8a73-465f-b098-cd61edcf256e.root 03-Jul-2022 11:00:50 CEST Initiating request to open file root://eoscms.cern.ch//eos/cms/store/data/Run2022A/Commissioning/ALCARECO/HcalCalIsoTrk-PromptReco-v1/000/352/417/00000/b5fbc8d1-02b5-4b30-aeca-b841f69a18e0.root %MSG-w XrdAdaptor: file_open 03-Jul-2022 11:00:52 CEST PostStreamBeginLumi Data is served from cern.ch instead of original site eoscms %MSG 03-Jul-2022 11:00:52 CEST Successfully opened file root://eoscms.cern.ch//eos/cms/store/data/Run2022A/Commissioning/ALCARECO/HcalCalIsoTrk-PromptReco-v1/000/352/417/00000/b5fbc8d1-02b5-4b30-aeca-b841f69a18e0.root endRun[7] 352416 03-Jul-2022 11:00:53 CEST Closed file root://eoscms.cern.ch//eos/cms/store/data/Run2022A/Commissioning/ALCARECO/HcalCalIsoTrk-PromptReco-v1/000/352/417/00000/b5fbc8d1-02b5-4b30-aeca-b841f69a18e0.root 03-Jul-2022 11:00:53 CEST Initiating request to open file root://eoscms.cern.ch//eos/cms/store/data/Run2022A/Commissioning/ALCARECO/HcalCalIsoTrk-PromptReco-v1/000/352/422/00000/3cc8f9d0-458b-4ff5-b80d-827a25b1af2e.root %MSG-w XrdAdaptor: file_open 03-Jul-2022 11:00:55 CEST PostStreamBeginLumi Data is served from cern.ch instead of original site eoscms %MSG 03-Jul-2022 11:00:55 CEST Successfully opened file root://eoscms.cern.ch//eos/cms/store/data/Run2022A/Commissioning/ALCARECO/HcalCalIsoTrk-PromptReco-v1/000/352/422/00000/3cc8f9d0-458b-4ff5-b80d-827a25b1af2e.root endRun[8] 352417 03-Jul-2022 11:00:55 CEST Closed file root://eoscms.cern.ch//eos/cms/store/data/Run2022A/Commissioning/ALCARECO/HcalCalIsoTrk-PromptReco-v1/000/352/422/00000/3cc8f9d0-458b-4ff5-b80d-827a25b1af2e.root 03-Jul-2022 11:00:55 CEST Initiating request to open file root://eoscms.cern.ch//eos/cms/store/data/Run2022A/Commissioning/ALCARECO/HcalCalIsoTrk-PromptReco-v1/000/352/425/00000/7f70dad2-b949-43f5-9400-e6bae647fec8.root %MSG-w XrdAdaptor: file_open 03-Jul-2022 11:00:57 CEST PostStreamBeginLumi Data is served from cern.ch instead of original site eoscms %MSG 03-Jul-2022 11:00:57 CEST Successfully opened file root://eoscms.cern.ch//eos/cms/store/data/Run2022A/Commissioning/ALCARECO/HcalCalIsoTrk-PromptReco-v1/000/352/425/00000/7f70dad2-b949-43f5-9400-e6bae647fec8.root endRun[9] 352422 Begin processing the 1st record. Run 352425, Event 16806351, LumiSection 18 on stream 0 at 03-Jul-2022 11:01:03.013 CEST PersistencyIO INFO +++ Set Streamer to dd4hep::OpaqueDataBlock DD4hep WARN ++ Using globally Geant4 unit system (mm,ns,MeV) DD4CMS INFO +++ Processing the CMS detector description xml-memory-buffer Detector INFO *********** Created World volume with size: 101000 101000 450000 Detector INFO +++ Patching names of anonymous shapes.... DDDefinition INFO +++ Finished processing xml-memory-buffer Run 352425 Event 16806351 Track 34 p 14.3353 03-Jul-2022 11:01:13 CEST Closed file root://eoscms.cern.ch//eos/cms/store/data/Run2022A/Commissioning/ALCARECO/HcalCalIsoTrk-PromptReco-v1/000/352/425/00000/7f70dad2-b949-43f5-9400-e6bae647fec8.root 03-Jul-2022 11:01:13 CEST Initiating request to open file root://eoscms.cern.ch//eos/cms/store/data/Run2022A/Commissioning/ALCARECO/HcalCalIsoTrk-PromptReco-v1/000/352/452/00000/c7dadd0c-16d8-4592-9917-4e10f9b47d4a.root %MSG-w XrdAdaptor: file_open 03-Jul-2022 11:01:15 CEST PostStreamBeginLumi Data is served from cern.ch instead of original site eoscms %MSG 03-Jul-2022 11:01:15 CEST Successfully opened file root://eoscms.cern.ch//eos/cms/store/data/Run2022A/Commissioning/ALCARECO/HcalCalIsoTrk-PromptReco-v1/000/352/452/00000/c7dadd0c-16d8-4592-9917-4e10f9b47d4a.root endRun[10] 352425 03-Jul-2022 11:01:15 CEST Closed file root://eoscms.cern.ch//eos/cms/store/data/Run2022A/Commissioning/ALCARECO/HcalCalIsoTrk-PromptReco-v1/000/352/452/00000/c7dadd0c-16d8-4592-9917-4e10f9b47d4a.root 03-Jul-2022 11:01:15 CEST Initiating request to open file root://eoscms.cern.ch//eos/cms/store/data/Run2022A/Commissioning/ALCARECO/HcalCalIsoTrk-PromptReco-v1/000/352/457/00000/633921aa-b676-4500-a65b-6e76b266c3fc.root %MSG-w XrdAdaptor: file_open 03-Jul-2022 11:01:17 CEST PostStreamBeginLumi Data is served from cern.ch instead of original site eoscms %MSG 03-Jul-2022 11:01:17 CEST Successfully opened file root://eoscms.cern.ch//eos/cms/store/data/Run2022A/Commissioning/ALCARECO/HcalCalIsoTrk-PromptReco-v1/000/352/457/00000/633921aa-b676-4500-a65b-6e76b266c3fc.root endRun[11] 352452 03-Jul-2022 11:01:17 CEST Closed file root://eoscms.cern.ch//eos/cms/store/data/Run2022A/Commissioning/ALCARECO/HcalCalIsoTrk-PromptReco-v1/000/352/457/00000/633921aa-b676-4500-a65b-6e76b266c3fc.root 03-Jul-2022 11:01:17 CEST Initiating request to open file root://eoscms.cern.ch//eos/cms/store/data/Run2022A/Commissioning/ALCARECO/HcalCalIsoTrk-PromptReco-v1/000/352/462/00000/541465e5-9e7c-47c0-8028-40804e40a0c0.root %MSG-w XrdAdaptor: file_open 03-Jul-2022 11:01:19 CEST PostStreamBeginLumi Data is served from cern.ch instead of original site eoscms %MSG 03-Jul-2022 11:01:19 CEST Successfully opened file root://eoscms.cern.ch//eos/cms/store/data/Run2022A/Commissioning/ALCARECO/HcalCalIsoTrk-PromptReco-v1/000/352/462/00000/541465e5-9e7c-47c0-8028-40804e40a0c0.root endRun[12] 352457 03-Jul-2022 11:01:20 CEST Closed file root://eoscms.cern.ch//eos/cms/store/data/Run2022A/Commissioning/ALCARECO/HcalCalIsoTrk-PromptReco-v1/000/352/462/00000/541465e5-9e7c-47c0-8028-40804e40a0c0.root 03-Jul-2022 11:01:20 CEST Initiating request to open file root://eoscms.cern.ch//eos/cms/store/data/Run2022A/Commissioning/ALCARECO/HcalCalIsoTrk-PromptReco-v1/000/352/463/00000/dead4d57-e843-48ef-885b-7e1fc2c14fe2.root %MSG-w XrdAdaptor: file_open 03-Jul-2022 11:01:22 CEST PostStreamBeginLumi Data is served from cern.ch instead of original site eoscms %MSG 03-Jul-2022 11:01:22 CEST Successfully opened file root://eoscms.cern.ch//eos/cms/store/data/Run2022A/Commissioning/ALCARECO/HcalCalIsoTrk-PromptReco-v1/000/352/463/00000/dead4d57-e843-48ef-885b-7e1fc2c14fe2.root endRun[13] 352462 03-Jul-2022 11:01:22 CEST Closed file root://eoscms.cern.ch//eos/cms/store/data/Run2022A/Commissioning/ALCARECO/HcalCalIsoTrk-PromptReco-v1/000/352/463/00000/dead4d57-e843-48ef-885b-7e1fc2c14fe2.root 03-Jul-2022 11:01:22 CEST Initiating request to open file root://eoscms.cern.ch//eos/cms/store/data/Run2022A/Commissioning/ALCARECO/HcalCalIsoTrk-PromptReco-v1/000/352/488/00000/defcab21-452f-4f4b-8ac5-95782eb42e84.root %MSG-w XrdAdaptor: file_open 03-Jul-2022 11:01:24 CEST PostStreamBeginLumi Data is served from cern.ch instead of original site eoscms %MSG 03-Jul-2022 11:01:24 CEST Successfully opened file root://eoscms.cern.ch//eos/cms/store/data/Run2022A/Commissioning/ALCARECO/HcalCalIsoTrk-PromptReco-v1/000/352/488/00000/defcab21-452f-4f4b-8ac5-95782eb42e84.root endRun[14] 352463 03-Jul-2022 11:01:24 CEST Closed file root://eoscms.cern.ch//eos/cms/store/data/Run2022A/Commissioning/ALCARECO/HcalCalIsoTrk-PromptReco-v1/000/352/488/00000/defcab21-452f-4f4b-8ac5-95782eb42e84.root 03-Jul-2022 11:01:24 CEST Initiating request to open file root://eoscms.cern.ch//eos/cms/store/data/Run2022A/Commissioning/ALCARECO/HcalCalIsoTrk-PromptReco-v1/000/352/493/00000/233a2412-338c-45d0-9b97-2a33a8bc9eb7.root %MSG-w XrdAdaptor: file_open 03-Jul-2022 11:01:26 CEST PostStreamBeginLumi Data is served from cern.ch instead of original site eoscms %MSG 03-Jul-2022 11:01:26 CEST Successfully opened file root://eoscms.cern.ch//eos/cms/store/data/Run2022A/Commissioning/ALCARECO/HcalCalIsoTrk-PromptReco-v1/000/352/493/00000/233a2412-338c-45d0-9b97-2a33a8bc9eb7.root endRun[15] 352488 03-Jul-2022 11:01:26 CEST Closed file root://eoscms.cern.ch//eos/cms/store/data/Run2022A/Commissioning/ALCARECO/HcalCalIsoTrk-PromptReco-v1/000/352/493/00000/233a2412-338c-45d0-9b97-2a33a8bc9eb7.root 03-Jul-2022 11:01:26 CEST Initiating request to open file root://eoscms.cern.ch//eos/cms/store/data/Run2022A/Commissioning/ALCARECO/HcalCalIsoTrk-PromptReco-v1/000/352/499/00000/b30e8af1-0c42-411e-9eca-244a9cc02a08.root %MSG-w XrdAdaptor: file_open 03-Jul-2022 11:01:28 CEST PostStreamBeginLumi Data is served from cern.ch instead of original site eoscms %MSG 03-Jul-2022 11:01:28 CEST Successfully opened file root://eoscms.cern.ch//eos/cms/store/data/Run2022A/Commissioning/ALCARECO/HcalCalIsoTrk-PromptReco-v1/000/352/499/00000/b30e8af1-0c42-411e-9eca-244a9cc02a08.root endRun[16] 352493 03-Jul-2022 11:01:29 CEST Closed file root://eoscms.cern.ch//eos/cms/store/data/Run2022A/Commissioning/ALCARECO/HcalCalIsoTrk-PromptReco-v1/000/352/499/00000/b30e8af1-0c42-411e-9eca-244a9cc02a08.root 03-Jul-2022 11:01:29 CEST Initiating request to open file root://eoscms.cern.ch//eos/cms/store/data/Run2022A/Commissioning/ALCARECO/HcalCalIsoTrk-PromptReco-v1/000/352/503/00000/dd5178ad-2478-4c53-8b2c-fcb39012e040.root %MSG-w XrdAdaptor: file_open 03-Jul-2022 11:01:31 CEST PostStreamBeginLumi Data is served from cern.ch instead of original site eoscms %MSG 03-Jul-2022 11:01:31 CEST Successfully opened file root://eoscms.cern.ch//eos/cms/store/data/Run2022A/Commissioning/ALCARECO/HcalCalIsoTrk-PromptReco-v1/000/352/503/00000/dd5178ad-2478-4c53-8b2c-fcb39012e040.root endRun[17] 352499 03-Jul-2022 11:01:31 CEST Closed file root://eoscms.cern.ch//eos/cms/store/data/Run2022A/Commissioning/ALCARECO/HcalCalIsoTrk-PromptReco-v1/000/352/503/00000/dd5178ad-2478-4c53-8b2c-fcb39012e040.root 03-Jul-2022 11:01:31 CEST Initiating request to open file root://eoscms.cern.ch//eos/cms/store/data/Run2022A/Commissioning/ALCARECO/HcalCalIsoTrk-PromptReco-v1/000/352/505/00000/16a12bdc-729d-4432-bc97-d96a45d95d7a.root %MSG-w XrdAdaptor: file_open 03-Jul-2022 11:01:33 CEST PostStreamBeginLumi Data is served from cern.ch instead of original site eoscms %MSG 03-Jul-2022 11:01:33 CEST Successfully opened file root://eoscms.cern.ch//eos/cms/store/data/Run2022A/Commissioning/ALCARECO/HcalCalIsoTrk-PromptReco-v1/000/352/505/00000/16a12bdc-729d-4432-bc97-d96a45d95d7a.root endRun[18] 352503
A fatal system signal has occurred: segmentation violation The following is the call stack containing the origin of the signal.
Sun Jul 3 11:01:34 CEST 2022
Thread 7 (Thread 0x7faaeed90700 (LWP 29776) "cmsRun"):
#0 0x00007fab206b1b3b in do_futex_wait.constprop () from /lib64/libpthread.so.0
#1 0x00007fab206b1bcf in __new_sem_wait_slow.constprop.0 () from /lib64/libpthread.so.0
#2 0x00007fab206b1c6b in sem_wait@@GLIBC_2.2.5 () from /lib64/libpthread.so.0
#3 0x00007fab1a829616 in XrdCl::JobManager::RunJobs() () from /cvmfs/cms.cern.ch/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_3_0_pre6/external/slc7_amd64_gcc10/lib/libXrdCl.so.3
#4 0x00007fab1a8296d9 in RunRunnerThread () from /cvmfs/cms.cern.ch/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_3_0_pre6/external/slc7_amd64_gcc10/lib/libXrdCl.so.3
#5 0x00007fab206abea5 in start_thread () from /lib64/libpthread.so.0
#6 0x00007fab203d4b0d in clone () from /lib64/libc.so.6
Thread 6 (Thread 0x7faaef591700 (LWP 29775) "cmsRun"):
#0 0x00007fab206b1b3b in do_futex_wait.constprop () from /lib64/libpthread.so.0
#1 0x00007fab206b1bcf in __new_sem_wait_slow.constprop.0 () from /lib64/libpthread.so.0
#2 0x00007fab206b1c6b in sem_wait@@GLIBC_2.2.5 () from /lib64/libpthread.so.0
#3 0x00007fab1a829616 in XrdCl::JobManager::RunJobs() () from /cvmfs/cms.cern.ch/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_3_0_pre6/external/slc7_amd64_gcc10/lib/libXrdCl.so.3
#4 0x00007fab1a8296d9 in RunRunnerThread () from /cvmfs/cms.cern.ch/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_3_0_pre6/external/slc7_amd64_gcc10/lib/libXrdCl.so.3
#5 0x00007fab206abea5 in start_thread () from /lib64/libpthread.so.0
#6 0x00007fab203d4b0d in clone () from /lib64/libc.so.6
Thread 5 (Thread 0x7faaefd92700 (LWP 29774) "cmsRun"):
#0 0x00007fab206b1b3b in do_futex_wait.constprop () from /lib64/libpthread.so.0
#1 0x00007fab206b1bcf in __new_sem_wait_slow.constprop.0 () from /lib64/libpthread.so.0
#2 0x00007fab206b1c6b in sem_wait@@GLIBC_2.2.5 () from /lib64/libpthread.so.0
#3 0x00007fab1a829616 in XrdCl::JobManager::RunJobs() () from /cvmfs/cms.cern.ch/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_3_0_pre6/external/slc7_amd64_gcc10/lib/libXrdCl.so.3
#4 0x00007fab1a8296d9 in RunRunnerThread () from /cvmfs/cms.cern.ch/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_3_0_pre6/external/slc7_amd64_gcc10/lib/libXrdCl.so.3
#5 0x00007fab206abea5 in start_thread () from /lib64/libpthread.so.0
#6 0x00007fab203d4b0d in clone () from /lib64/libc.so.6
Thread 4 (Thread 0x7faaf0593700 (LWP 29773) "cmsRun"):
#0 0x00007fab206b2e9d in nanosleep () from /lib64/libpthread.so.0
#1 0x00007fab1a91608f in XrdSysTimer::Wait(int) () from /cvmfs/cms.cern.ch/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_3_0_pre6/external/slc7_amd64_gcc10/lib/libXrdUtils.so.3
#2 0x00007fab1a798fa3 in XrdCl::TaskManager::RunTasks() () from /cvmfs/cms.cern.ch/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_3_0_pre6/external/slc7_amd64_gcc10/lib/libXrdCl.so.3
#3 0x00007fab1a7990f9 in RunRunnerThread () from /cvmfs/cms.cern.ch/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_3_0_pre6/external/slc7_amd64_gcc10/lib/libXrdCl.so.3
#4 0x00007fab206abea5 in start_thread () from /lib64/libpthread.so.0
#5 0x00007fab203d4b0d in clone () from /lib64/libc.so.6
Thread 3 (Thread 0x7faaf0d94700 (LWP 29772) "cmsRun"):
#0 0x00007fab203d50e3 in epoll_wait () from /lib64/libc.so.6
#1 0x00007fab1a9104b2 in XrdSys::IOEvents::PollE::Begin(XrdSysSemaphore*, int&, char const**) () from /cvmfs/cms.cern.ch/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_3_0_pre6/external/slc7_amd64_gcc10/lib/libXrdUtils.so.3
#2 0x00007fab1a90cc3d in XrdSys::IOEvents::BootStrap::Start(void*) () from /cvmfs/cms.cern.ch/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_3_0_pre6/external/slc7_amd64_gcc10/lib/libXrdUtils.so.3
#3 0x00007fab1a915798 in XrdSysThread_Xeq () from /cvmfs/cms.cern.ch/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_3_0_pre6/external/slc7_amd64_gcc10/lib/libXrdUtils.so.3
#4 0x00007fab206abea5 in start_thread () from /lib64/libpthread.so.0
#5 0x00007fab203d4b0d in clone () from /lib64/libc.so.6
Thread 2 (Thread 0x7faafc5a5700 (LWP 29761) "cmsRun"):
#0 0x00007fab206b31d9 in waitpid () from /lib64/libpthread.so.0
#1 0x00007fab19fc8fd7 in edm::service::cmssw_stacktrace_fork() () from /cvmfs/cms.cern.ch/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_3_0_pre6/lib/slc7_amd64_gcc10/pluginFWCoreServicesPlugins.so
#2 0x00007fab19fc9b4a in edm::service::InitRootHandlers::stacktraceHelperThread() () from /cvmfs/cms.cern.ch/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_3_0_pre6/lib/slc7_amd64_gcc10/pluginFWCoreServicesPlugins.so
#3 0x00007fab20caaf90 in std::execute_native_thread_routine (__p=0x7fab0e25ed10) at ../../../../../libstdc++-v3/src/c++11/thread.cc:80
#4 0x00007fab206abea5 in start_thread () from /lib64/libpthread.so.0
#5 0x00007fab203d4b0d in clone () from /lib64/libc.so.6
Thread 1 (Thread 0x7fab1e529540 (LWP 29742) "cmsRun"):
#0 0x00007fab203c9ddd in poll () from /lib64/libc.so.6
#1 0x00007fab19fc928f in full_read.constprop () from /cvmfs/cms.cern.ch/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_3_0_pre6/lib/slc7_amd64_gcc10/pluginFWCoreServicesPlugins.so
#2 0x00007fab19fc9c1c in edm::service::InitRootHandlers::stacktraceFromThread() () from /cvmfs/cms.cern.ch/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_3_0_pre6/lib/slc7_amd64_gcc10/pluginFWCoreServicesPlugins.so
#3 0x00007fab19fcc56b in sig_dostack_then_abort () from /cvmfs/cms.cern.ch/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_3_0_pre6/lib/slc7_amd64_gcc10/pluginFWCoreServicesPlugins.so
#4
Current Modules:
Module: none (crashed)
A fatal system signal has occurred: segmentation violation Segmentation fault (core dumped)
My configuration file is -- /afs/cern.ch/user/d/dasgupsu/public/hcal_calibration/alcareco_treemaker/collision_900GeV/CMSSW_12_3_0_pre6/src/Calibration/HcalCalibAlgos/test/python/isoTrackAlCaAnalysis_cfg.py inpul file list is --- /afs/cern.ch/user/d/dasgupsu/public/hcal_calibration/alcareco_treemaker/collision_900GeV/CMSSW_12_3_0_pre6/src/Calibration/HcalCalibAlgos/test/python/inputfiles.py
A new Issue was created by @dasguptasuman .
@Dr15Jones, @perrotta, @dpiparo, @makortel, @smuzaffar, @qliphy can you please review it and eventually sign/assign? Thanks.
cms-bot commands are listed here
Assign geometry
New categories assigned: geometry
@cvuosalo,@mdhildreth,@ianna,@Dr15Jones,@makortel,@civanch you have been requested to review this Pull request/Issue and eventually sign? Thanks
FYI @cms-sw/alca-l2 @cms-sw/hcal-dpg-l2
@bsunanda can you please have a look?
It could be interesting to know if the segmentation faults reproduces also for some more recent release, other than in the old pre-release CMSSW_12_3_0_pre6.
It runs successfully without segmentation violation for the release CMSSW_12_4_0_pre4.
It runs successfully without segmentation violation for the release CMSSW_12_4_0_pre4.
Thank you @dasguptasuman What about 12_3_6 (e.g.), i.e. a full release in the 12_3 serie? If it works also over there, I wouldn't bother that much about what happened in an old pre-release, not intended to be used for any production or analyisis
It does not work for the CMSSW_12_3_6 version.
ahh I'm re-reading this now
one segmentation fault occurs when including the 'run3_data' global tag.
why are you using that GT? That's meant for a ReReco campaign. Which is planned in about 1 year from now.
https://twiki.cern.ch/twiki/bin/view/CMS/LatestOnlineGTs
--> what if you try with 123X_dataRun3_Prompt_v12
?
assign alca
New categories assigned: alca
@yuanchao,@francescobrivio,@malbouis,@tvami,@ChrisMisan you have been requested to review this Pull request/Issue and eventually sign? Thanks
unassign geometry
Same issue is also observed for "123X_dataRun3_Prompt_v12" global Tag.
This is path of the config file: /afs/cern.ch/user/d/dasgupsu/public/hcal_calibration/alcareco_treemaker/collision_900GeV/CMSSW_12_3_6/src/Calibration/HcalCalibAlgos/test/python/isoTrackAlCaAnalysis_cfg.py
This is path of the log file:/afs/cern.ch/user/d/dasgupsu/public/hcal_calibration/alcareco_treemaker/collision_900GeV/CMSSW_12_3_6/src/Calibration/HcalCalibAlgos/test/python/output_alca.log
cmsRun isoTrackAlCaAnalysis_cfg.py
leads to
----- Begin Fatal Exception 05-Jul-2022 18:50:15 CEST-----------------------
An exception of category 'ConfigFileReadError' occurred while
[0] Processing the python configuration file named isoTrackAlCaAnalysis_cfg.py
Exception Message:
unknown python problem occurred.
ModuleNotFoundError: No module named 'inputfiles'
but after fixing that, I can reproduce the problem. I'll try to fix it tomorrow. @bsunanda if you find something obvious please let us know
Sorry I do not know the issue. I shall take a look. Is it only for CMSSW_12_3_X or is it true for CMSSW_12_5_X and CMSSW_12_4_X?
From: Tamas Vami @.*** Sent: 05 July 2022 19:07 To: cms-sw/cmssw Cc: Sunanda Banerjee; Mention Subject: Re: [cms-sw/cmssw] segmentation faults in hcalIsoTrkAnalyzer_cff.py (Issue #38586)
cmsRun isoTrackAlCaAnalysis_cfg.py
leads to
----- Begin Fatal Exception 05-Jul-2022 18:50:15 CEST----------------------- An exception of category 'ConfigFileReadError' occurred while [0] Processing the python configuration file named isoTrackAlCaAnalysis_cfg.py Exception Message: unknown python problem occurred. ModuleNotFoundError: No module named 'inputfiles'
but after fixing that, I can reproduce the problem. I'll try to fix it tomorrow. @bsunandahttps://github.com/bsunanda if you find something obvious please let us know
— Reply to this email directly, view it on GitHubhttps://github.com/cms-sw/cmssw/issues/38586#issuecomment-1175300583, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABGMZOXF5J7ZDNVFBVLE373VSRTU3ANCNFSM52QMQR5A. You are receiving this because you were mentioned.Message ID: @.***>
I have checked for these two version CMSSW_12_5_0_pre2 and CMSSW_12_4_1 . Same issue is observed for this global tag ' 123X_dataRun3_Prompt_v12' .
Something seems to be going wrong at the run transitions,
If I run on
'/store/data/Run2022A/Commissioning/ALCARECO/HcalCalIsoTrk-PromptReco-v1/000/352/568/00000/7ab7e89a-6925-4bfa-9e0f-e9c74fc4a4a3.root')
alone, it works, if I run on
'/store/data/Run2022A/Commissioning/ALCARECO/HcalCalIsoTrk-PromptReco-v1/000/352/567/00000/f40e87a2-1eb2-45b4-9893-17c67d8a3273.root')
alone, it also works. But when both of them are inputs, it segfaults
looking at the code
https://github.com/cms-sw/cmssw/blob/942cf0fb51630f6e9a514e34824c1b309ddcc463/Calibration/HcalCalibAlgos/plugins/HcalIsoTrkAnalyzer.cc#L300-L307
tok_ddrec_
has a edm::Transition::BeginRun
Does the IOV of IdealMagneticFieldRecord
change between runs 352567 and 352568?
Looking at its definition
https://github.com/cms-sw/cmssw/blob/86937de21338aa5477a0405b3ebb082811bcbd7e/MagneticField/Records/interface/IdealMagneticFieldRecord.h#L11-L13
RunInfoRcd
sounds like something whose IOV could change for every Run.
Could it be that in some specific situations (since the log in the issue description had several successful Run transitions before the crash) re-creating the (magnetic field) geometry can fail?
MFGeometryFileRcd --> a single IOV from 1 to inf MagFieldConfigRcd --> 2 IOVs, they change at 220641 RunInfoRcd --> this is huge, one of our biggest tags, 98580 IOVs, as expected, changes for every run, so yes, that changes from 352567 and 352568
We just noticed what one of the AlCa harvestors is also having a very similar behaviour, I opened another issue for that https://github.com/cms-sw/cmssw/issues/38624
assign geometry
- I never should have removed it, sorry
New categories assigned: geometry
@mdhildreth,@ianna,@Dr15Jones,@makortel,@bsunanda,@civanch you have been requested to review this Pull request/Issue and eventually sign? Thanks
FYI @bsunanda
@cms-sw/geometry-l2 @makortel this issue is resolved, shall it be closed?
Fine with me
@cmsbuild, please close