celeritas-project
Add Celeritas optical-only integration with existing Geant4
In the future, due to ongoing work by @whokion , we expect a new Geant4 feature to allow G4 optical generation processes to emit distributions of photons rather than photons. These distributions can then be offloaded to Opticks, Celeritas, etc. and will be supported in the long term by Geant4.
However, in the meantime for validation and for compatibility with older frameworks, we need to support an independent way to integrate Celertias' optical physics into a Geant4 application. The obvious way to do this is with an approach like Opticks, having a custom G4VProcess that replaces Cerenkov for example.
Unlike Opticks, this will not be just a copy of G4Cerenkov/G4Scintillation. Instead, it will convert Geant4 data into Celeritas types and use the Celeritas CherenkovOffload class to add generator distributions to the Celeritas optical loop. This will allow validation of more of Celeritas from inside a Geant4 context.
Note that this concept is similar to https://github.com/celeritas-project/celeritas/issues/1396, but this will be for optical processes rather than EM. Some technical challenges may be shared: for example, whether we can both pull data from and replace a process.
Proposed steps:
- Make a G4 example with optical physics; figure out how to cleanly replace the cerenkov/scintillatoin with Celeritas (custom modular physics list?)
- Add adapter classes for Geant4 classes that can be reused in geant importer, primary construction, hit processing etc (@sethrj )
- Add adapter class for HEP RNG (@sethrj )
- Create generator distributions in a new G4VProcess subclass using celeritas data
- Add an optical-only mode for setting up celeritas whose "flush" at end of event will run optical photons
I'm not sure we'll actually need to create custom Cherenkov/Scintillation processes for this. I think we should be able to set the "stacking" flag to false, so the number of photons is sampled in the Geant4 processes but the secondaries aren't generated. Then we can retrieve the number via GetNumPhotons().
@amandalund Great idea, but we may want to consider both options since with a custom process the Celeritas imported materials and dn/dx calculator are used rather than Geant4 (could be good for validation).
True, that could be useful for validation. We might be able to get something working more quickly starting with the existing processes (wouldn't have to worry about changing the physics list or replacing the processes when integrating with an external application).
Discussion from Slack on 15 July
@whokion:
In principle, we can implement custom optical generation processes for scintillation and Cherenkov processes, replacing the standard G4Scintillation and G4Cerenkov processes in the G4OpticalPhysics constructor with new ones in a custom optical physics list. However, this still requires to introduce metadata (similar to GenStep in Opticks) to store the information needed to generate optical photons at a later stage (ie., to provide corresponding input to Celeritas optical generators. The question is how to store this metadata during the standard Geant4 stepping/tracking loop. One possible approach is to create a single secondary track and attach the necessary auxiliary track information to it. This is actually quite similar to the mechanism I’m currently adding to Geant4—extending the existing G4Scintillation and G4Cerenkov processes without introducing new custom version of them. In any case, I’ll experiment with this alternative approach using a custom implementation, without relying on any new Geant4 features as parallel as for now. (edited)
@amandalund:
if we were to use the existing Geant4 Cherenkov and scintillation processes rather than creating custom processes (and set "stacking" to false, so the sampled number of photons is stored but no photons are created), then I think we could use a stepping action to retrieve the number of photons from the process and buffer the distribution data
@whokion:
We still need additional step/track dependent information such as parent charge, material id, step length, velocity, position etc to generate secondary optical photons in the later or offloaded stage. Of course, one can retrieve or reconstruct all necessary information from a user stepping action, which is sort of equivalent to reimplement the early part of the G4Cerekov and G4Scintillation processes (i.e., without the optical photon generation loop).
@sethrj:
I’m confused about the metadata storage requirement; wouldn’t we be pushing distributions to the celeritas optical queue just like we push primaries in EM?
@whokion:
So, do you want to directly call celeritas optical distribution from those custom processes? (which may be, I think, technically possible).
@sethrj:
Exactly
Sorry for the delay on this task. I still believe the cleaner solution is to send an optical photon track along with the relevant metadata (e.g., user track information such as material Id, energy deposition, etc., required for Celeritas optical distributions) to the user tracking action. There, we can select optical tracks and create the corresponding Celeritas optical distributions for offloading. This approach decouples the custom Geant4 processes and user physics list from the user application and its actions—similar to the offloading pipeline already used for EM particles.
An advantage is that the Celeritas import data can be initialized in the user (run) action and shared at the application level for both Cerenkov and scintillation. Optical distributions can then be generated using the imported data along with minimal metadata passed via the Geant4 optical photon track from the custom generation processes. The overhead should be negligible, since only a small amount of data is passed to the Geant4 tracking manager and the number of primary steps for optical generation processes are relatively small.
Alternatively, we could generate optical distributions directly within the processes and store them (in each process or in a static vector) for later transfer to a Celeritas user action. It is not who owns and handles these distribution data and how/where to hand it over to Celeritas transport during the standard Geant4 process and tracking loop. That seems like a less typical and more tightly coupled with the Geant4 kernel itself.
Thanks @whokion for your thoughts!
I still believe the cleaner solution is to send an optical photon track along with the relevant metadata (e.g., user track information such as material Id, energy deposition, etc., required for Celeritas optical distributions) to the user tracking action. There, we can select optical tracks and create the corresponding Celeritas optical distributions for offloading. This approach decouples the custom Geant4 processes and user physics list from the user application and its actions—similar to the offloading pipeline already used for EM particles.
It sounds like this approach will still use Geant4 to generate the optical tracks with its own scintillation/cherenkov process, and we'd have to copy all those photons to our own pinned memory and then to GPU. This tracking action (or alternatively a custom tracking manager) is different than the stepping action (or physics process) that would be necessary to implement @amandalund 's idea, right?
An advantage is that the Celeritas import data can be initialized in the user (run) action and shared at the application level for both Cerenkov and scintillation. Optical distributions can then be generated using the imported data along with minimal metadata passed via the Geant4 optical photon track from the custom generation processes.
I am not sure how the difference between a custom process, tracking action, or stepping action would change initialization, except that a custom process replacing G4Cherenkov would complicate the data import.
Alternatively, we could generate optical distributions directly within the processes and store them (in each process or in a static vector) for later transfer to a Celeritas user action. It is not who owns and handles these distribution data and how/where to hand it over to Celeritas transport during the standard Geant4 process and tracking loop. That seems like a less typical and more tightly coupled with the Geant4 kernel itself.
I really think we do want to generate the distributions. Wouldn't it be easy to have a PushCherenkov(GeneratorData const&) function in the LocalTransporter that buffers the distributions to send to GPU?
I interpreted "send an optical photon track along with the relevant metadata" to mean rather than generating all the photons, the Cherenkov/scintillation process would instead generate one optical secondary with some attached user track into which would contain all the data we need to create the distribution (but I may be misunderstanding). If that's the case I guess we'd still need our own custom Geant4 processes. Is it straightforward to replace a Geant4 process in a user's physics list with our own, or would that need to be done on the user's side?
@amandalund Your understanding is correct (i.e., only one optical photon is created from each generation process). Regarding the custom generation processes, we still need to implement them (i.e. celeritas scintillation and Cerenkov processes)—straightforward but collecting the minimal metadata that will be attached to a secondary optical photon via user track information—and add them to a Celeritas physics list. We will neither use the G4OpticalPhysics constructor nor replace any physics Geant4 process, as no standard Geant4 optical processes will be involved in the offloading workflow.
I still don't understand why one stacked secondary per step (especially since it sounds like it has to "abuse" the user data) is preferable to directly offloading to celeritas. Is it just that it's a similar process to the one you've got
I still don't understand why one stacked secondary per step (especially since it sounds like it has to "abuse" the user data) is preferable to directly offloading to celeritas. Is it just that it's a similar process to the one you're writing to incorporate into the next version of geant4?
But if we're integrating with a user application and they have G4Cherenkov or G4Scintillation in their physics list (or even their own custom Cherenkov/scintillation processes), will we be able to replace those processes with our custom generation processes? Or will they need to change their physics list to use the Celeritas processes?
To answer @sethrj's second point above: If we generate and push the distribution from each custom process, the imported data would need to be accessed individually by those processes. This approach is somewhat awkward, as it requires interfacing with Celeritas code and using data structures within Geant4 processes (even though there is no technical issue for doing that). Instead, we can handle this more cleanly by accessing the data centrally at the application level using a user tracking action—ruling out a custom tracking manager as this approach is meant to support older versions of Geant4 as well as a unnecessary user stepping action.
But if we're integrating with a user application and they have
G4CherenkovorG4Scintillationin their physics list (or even their own custom Cherenkov/scintillation processes), will we be able to replace those processes with our custom generation processes? Or will they need to change their physics list to use the Celeritas processes?
Even though a user physics list has both G4Cherenkov and G4Scintillation in their physics list, we can always deactivate them from the celeritas side (then, no optical photons will be generated from the user application, so no Geant4 optical transport processes will be called or relevant either).
This approach is somewhat awkward, as it requires interfacing with Celeritas code and using data structures within Geant4 processes (even though there is no technical issue for doing that)
Rather than being awkward, I think this will be the first instance of a powerful validation technique: as discussed in https://github.com/celeritas-project/celeritas/issues/1396 it would help validation to be able to plug Celeritas physics into Geant4 applications. We'll be able to get the optical data from the SharedParams and LocalTransporter at any time during the run.
When playing around with this last week I sketched out https://github.com/sethrj/celeritas/commit/e8364d2ac6c078f03b6a095e70d29c01f3665a8e . I think a clean interface for converting between Celeritas and Geant4 will be a plus for people trying to integrate the benefits of both. For example, we can use https://github.com/sethrj/celeritas/commit/2fc04470a41b6bb6dc3b307cbbe83149b719794e to import particle data into Celeritas or to pass data into the Cherenkov offload.
Sounds and looks good! One potential downside, however, is that any user-defined stacking actions will be bypassed, since no actual secondary photons are created - in the user track information approach, a single optical photon still follows user stacking actions equivalently representing the entire secondary photons that would have been generated in the standard loop.
See https://github.com/celeritas-project/celeritas/issues/886#issuecomment-1875974225
- This issue is the Opticks-like approach where we send "gensteps" from a stepping action
- Another issue could be added for @whokion 's Geant4 11.4 QuasiPhoton integration; would this use a specialized tracking manager?
- Another way (@Rashika-Gupta could do this) is having a tracking manager for optical photons (PDG -22) that sends directly to our optical photon loop
All those are different than what we're currently doing which is to only send EM tracks, and only create optical photons from offloaded EM tracks