LDMX-Software
[Discussion] Cleanup detectors (v9, v12, v13)
Is your feature request related to a problem? Please describe.
Maybe not related but my compilation takes a while and I see all these all detectors "builds" take upto 24% of the compilation progress bar
Actual compilation printouts:
[ 0%] Built target ldmx-ti-v9_tagger.gdml
[ 0%] Built target ldmx-det-v12_ecal.gdml
[ 0%] Built target ldmx-det-v12_magnet.gdml
[ 0%] Built target ldmx-det-v12_recoil.gdml
[ 0%] Built target ldmx-det-v12_constants.gdml
[ 1%] Built target ldmx-det-v12_detector.gdml
[ 1%] Built target ldmx-det-v12_hcal.gdml
[ 2%] Built target Framework_Exception
[ 2%] Built target PerfDict
[ 2%] Built target SimCore_Event
[ 2%] Built target ldmx-det-v12_scoring_planes.gdml
[ 3%] Built target ldmx-det-v12_tagger.gdml
[ 3%] Built target ldmx-det-v13_detector.gdml
[ 3%] Built target ldmx-det-v13_constants.gdml
[ 3%] Built target ldmx-det-v12_target.gdml
[ 3%] Built target ldmx-det-v13_ecal.gdml
[ 3%] Built target ldmx-det-v13_hcal.gdml
[ 4%] Built target Recon_Event
[ 5%] Built target ldmx-det-v13_magnet.gdml
[ 5%] Built target ldmx-det-v13_recoil.gdml
[ 5%] Built target ldmx-det-v13_scoring_planes.gdml
[ 5%] Built target ldmx-det-v13_target.gdml
[ 5%] Built target ldmx-det-v13_tagger.gdml
[ 5%] Built target ldmx-det-v14_constants.gdml
[ 5%] Built target ldmx-det-v14_ecal_motherboard5_assembly.gdml
[ 5%] Built target ldmx-det-v14_detector.gdml
[ 5%] Built target ldmx-det-v14_ecal.gdml
[ 6%] Built target ldmx-det-v14_ecal_support_box.gdml
[ 8%] Built target G4DarkBreM
[ 8%] Built target ldmx-det-v14_ecal_motherboard6_assembly.gdml
[ 8%] Built target ldmx-det-v14_hcal.gdml
[ 8%] Built target ldmx-det-v14_magnet.gdml
[ 8%] Built target ldmx-det-v14_materials.gdml
[ 8%] Built target ldmx-det-v14_recoil.gdml
[ 8%] Built target ldmx-det-v14_target.gdml
[ 8%] Built target ldmx-det-v14_scoring_planes.gdml
[ 9%] Built target ldmx-det-v14_tagger.gdml
[ 9%] Built target ldmx-det-v14_trig_scint.gdml
[ 9%] Built target ldmx-det-v14-8gev_constants.gdml
[ 10%] Built target ldmx-det-v14-8gev_detector.gdml
[ 10%] Built target ldmx-det-v14-8gev_ecal_motherboard5_assembly.gdml
[ 10%] Built target ldmx-det-v14-8gev_ecal.gdml
[ 10%] Built target ldmx-det-v14-8gev_ecal_motherboard6_assembly.gdml
[ 10%] Built target ldmx-det-v14-8gev_ecal_support_box.gdml
[ 10%] Built target ldmx-det-v14-8gev_hcal.gdml
[ 11%] Built target ldmx-det-v14-8gev_magnet.gdml
[ 11%] Built target ldmx-det-v14-8gev_scoring_planes.gdml
[ 11%] Built target ldmx-det-v14-8gev_recoil.gdml
[ 11%] Built target ldmx-det-v14-8gev_materials.gdml
[ 11%] Built target ldmx-det-v14-8gev_tagger.gdml
[ 12%] Built target ldmx-det-v14-8gev_trig_scint.gdml
[ 12%] Built target ldmx-det-v14-8gev_target.gdml
[ 12%] Built target ldmx-det-v9_detector.gdml
[ 12%] Built target ldmx-det-v9_ecal.gdml
[ 13%] Built target ldmx-det-v9_hcal.gdml
[ 13%] Built target ldmx-det-v9_magnet.gdml
[ 13%] Built target ldmx-det-v9_recoil.gdml
[ 13%] Built target ldmx-det-v9_scoring_planes.gdml
Scanning dependencies of target EventDict
[ 13%] Built target ldmx-hcal-prototype-v1.0_constants.gdml
[ 13%] Built target ldmx-hcal-prototype-v1.0_hcal_physvol.gdml
Consolidate compiler generated dependencies of target EventDict
[ 14%] Built target ldmx-hcal-prototype-v1.0_detector.gdml
[ 14%] Built target ldmx-det-v9_tagger.gdml
[ 14%] Built target ldmx-hcal-prototype-v1.0_hcal_solids.gdml
[ 14%] Built target ldmx-hcal-prototype-v1.0_hcal_volumes.gdml
[ 14%] Built target ldmx-hcal-prototype-v1.0_trigger_physvol.gdml
[ 14%] Built target ldmx-hcal-prototype-v1.0_materials.gdml
[ 15%] Built target ldmx-hcal-prototype-v1.0_trigger_solids.gdml
[ 15%] Built target ldmx-hcal-prototype-v1.0_trigger_volumes.gdml
[ 15%] Built target ldmx-hcal-prototype-v1.0_userinfo.gdml
[ 15%] Built target ldmx-hcal-prototype-v2.0_TS_physvol.gdml
[ 15%] Building CXX object Framework/CMakeFiles/EventDict.dir/EventDict.cxx.o
[ 15%] Built target ldmx-hcal-prototype-v2.0_TS_solids.gdml
[ 15%] Built target ldmx-hcal-prototype-v2.0_detector.gdml
[ 15%] Built target ldmx-hcal-prototype-v2.0_TS_volumes.gdml
[ 15%] Built target ldmx-hcal-prototype-v2.0_frame_physvol.gdml
[ 16%] Built target ldmx-hcal-prototype-v2.0_constants.gdml
[ 16%] Built target ldmx-hcal-prototype-v2.0_frame_solids.gdml
[ 16%] Built target ldmx-hcal-prototype-v2.0_hcal_physvol.gdml
[ 16%] Built target ldmx-hcal-prototype-v2.0_frame_volumes.gdml
[ 17%] Built target ldmx-hcal-prototype-v2.0_hcal_solids.gdml
[ 17%] Built target ldmx-hcal-prototype-v2.0_materials.gdml
[ 17%] Built target ldmx-hcal-prototype-v2.0_userinfo.gdml
[ 17%] Built target ldmx-hcal-prototype-v2.0_hcal_volumes.gdml
[ 17%] Built target ldmx-lyso-r1-v14-8gev_detector.gdml
[ 17%] Built target ldmx-lyso-r1-v14-8gev_ecal_motherboard6_assembly.gdml
[ 18%] Built target ldmx-lyso-r1-v14-8gev_ecal.gdml
[ 18%] Built target ldmx-lyso-r1-v14-8gev_constants.gdml
[ 18%] Built target ldmx-lyso-r1-v14-8gev_ecal_support_box.gdml
[ 18%] Built target ldmx-lyso-r1-v14-8gev_ecal_motherboard5_assembly.gdml
[ 18%] Built target ldmx-lyso-r1-v14-8gev_hcal.gdml
[ 18%] Built target ldmx-lyso-r1-v14-8gev_magnet.gdml
[ 19%] Built target ldmx-lyso-r1-v14-8gev_materials.gdml
[ 19%] Built target ldmx-lyso-r1-v14-8gev_scoring_planes.gdml
[ 19%] Built target ldmx-lyso-r1-v14-8gev_recoil.gdml
[ 19%] Built target ldmx-lyso-r1-v14-8gev_tagger.gdml
[ 19%] Built target ldmx-lyso-r1-v14-8gev_trig_scint.gdml
[ 19%] Built target ldmx-lyso-r1-v14-8gev_target.gdml
[ 20%] Built target ldmx-lyso-v12_constants.gdml
[ 20%] Built target ldmx-lyso-v12_detector.gdml
[ 20%] Built target ldmx-lyso-v12_ecal.gdml
[ 20%] Built target ldmx-lyso-v12_hcal.gdml
[ 20%] Built target ldmx-lyso-v12_magnet.gdml
[ 20%] Built target ldmx-lyso-v12_recoil.gdml
[ 21%] Built target ldmx-lyso-v12_scoring_planes.gdml
[ 21%] Built target ldmx-lyso-v12_tagger.gdml
[ 21%] Built target ldmx-lyso-v12_target.gdml
[ 21%] Built target ldmx-reduced-v1_constants.gdml
[ 21%] Built target ldmx-reduced-v1_ecal.gdml
[ 21%] Built target ldmx-reduced-v1_detector.gdml
[ 22%] Built target ldmx-reduced-v1_ecal_motherboard5_assembly.gdml
[ 22%] Built target ldmx-reduced-v1_ecal_support_box.gdml
[ 22%] Built target ldmx-reduced-v1_ecal_motherboard6_assembly.gdml
[ 22%] Built target ldmx-reduced-v1_hcal.gdml
[ 22%] Built target ldmx-reduced-v1_materials.gdml
[ 22%] Built target ldmx-reduced-v1_recoil.gdml
Consolidate compiler generated dependencies of target DetDescr
[ 23%] Built target ldmx-reduced-v1_scoring_planes.gdml
[ 23%] Built target ldmx-reduced-v1_target.gdml
Consolidate compiler generated dependencies of target Ecal_Event
[ 23%] Built target ldmx-reduced-v1_tagger.gdml
[ 23%] Built target ldmx-reduced-v1_trig_scint.gdml
[ 23%] Built target ldmx-ti-v9_detector.gdml
Consolidate compiler generated dependencies of target Hcal_Event
[ 24%] Built target ldmx-ti-v9_hcal.gdml
[ 24%] Built target ldmx-ti-v9_magnet.gdml
[ 24%] Built target ldmx-ti-v9_recoil.gdml
[ 24%] Built target ldmx-ti-v9_ecal.gdml
Describe the solution you'd like
Maybe it's time for a cleanup? I see v1-v4 were tar zipped at one point, so I'm wondering if we could do the same for
ldmx-det-v9, ldmx-det-v12, ldmx-det-v13, ldmx-hcal-prototype-v1.0, ldmx-lyso-v12
? Maybe even ldmx-ti-v9? Is anybody using that, or any of the above? For LYSO we just made v14 version, for HCAL demo there is v2.
Describe alternatives you've considered
My alternative is even more aggressive: we could completely delete these and rely on github history. If somebody needs to go back to these geometry there is always a way to check out an older version of ldmx-sw, so the only reason to keep these is if we want to check something that's a development, but then developments would be done for the current geometry, right?
While I am in favor of removing old and unused detector versions, I also want to point out that the percentage is not fraction of time but fraction of build targets. The GDML files create a lot of build targets since the CMake stuff I wrote to replace the relative path references with full paths effectively creates a single build target for each of the GDML files within the Detectors data directory.
To be clear, I am not in favor of your more aggressive approach since the code development can often interact with the detector development. The best example I have of this is when we were comparing v12 and v14 of the ECal - we wanted to use the newer software for both so having them both readily available in the same git commit is preferable.
we were comparing v12 and v14 of the ECal
Yeah sure that's a good example where this would be needed. But we are done with those comparisons, right?
What's your take on the tarring solution? If somebody needs a prev geometry they can still untar and make the comparison. I'd just like to understand if that's nowadays the main activity of any group or we can consider that to be the secondary, and thus support it by requiring the user to untar.
I also changed the issue title to include [Discussion], and maybe let me ping people directly @therwig @EinarElen @vdutta @bryngemark @wesketchum (especially regarding the ti-v9)
i can provide the information that ever since some backwards compatibility breaking changes a few years ago, we are not able to run on v9 without some real work on the code. i'd be in favor of tarring that. btw since it's the geometry version used for both our actual papers, we should make extra sure that anyone/future us would know how to find it again.
i agree with @tom that v12 is too recent to remove from the list of readily available geometries. i am almost willing to bet that when we get to the deeper end of BDT development/multielectron studies, people will want to know "what did that look like in v12?"
v13 only introduces the rotated ecal, right? as such it could help with answering similar questions.
this touches upon a related matter: are we/should we be testing code development on all geometries? this could perhaps determine when we want to demote a geometry from active to tarred.
should we be testing code development on all geometries
I think we really should do this! I hope the github actions will be able to take a few more workflows.
Speaking of "broken" geometries, I think ldmx-lyso-v12 has an unclosed loop and so it will probably not run as it is.
For now my understanding is that we should run tests that use
ldmx-det-v12
ldmx-det-v13
ldmx-det-v14 --> this is being run, but planned to move to ldmx-det-v14-8gev
ldmx-det-v14-8gev --> this is what we run now
ldmx-hcal-prototype-v2.0
ldmx-lyso-r1-v14-8gev
ldmx-reduced-v1
and to tar the followings
ldmx-det-9
ldmx-hcal-prototype-v1.0 ??
ldmx-lyso-v12
ldmx-ti-v9
Should ldmx-hcal-prototype-v1.0 be also one of those active geometries with the tests or that can be tarred? Tagging @EinarElen @cmantill