Castro
Castro copied to clipboard
get rid of global / managed runtime parameters
GPU compilers seem to have trouble with managed-memory runtime parameters, so we should remove them.
The idea is that we will have the current python script define a set of structs for each namespace and then a parent struct, params
that has each namespace struct. E.g.:
struct hydro_params {
int time_integration_method{};
int ppm_type{};
};
struct gravity_params{
int do_grav{};
int grav_source_type{};
};
struct params {
hydro_params hydro;
gravity_params gravity;
};
We will then make params run_params
a static member variable of the Castro
class. This would allow us to access it anywhere.
If we want to simplify things, we could do something like:
auto & hp = run_params.hydro;
to just access the hydro parameters.
Currently all of the Castro runtime parameters are parmparsed in separate headers, like castro_queries.H
that are all included in intiialize_cpp_runparams()
.
We can implement this a bit at a time. Here's an outline:
- [ ] modify
parse_castro_params.py
to create the struct's for each namespace and the parent struct. These can all be in a single header. This is done in PR #2688. - [ ] ~~modify
parse_castro_params.py
to also create a "sync" function that initializes the runtime parameter structs by setting the value from the global version. This is temporary.~~ In the*_queries.H
, also set the struct version of the parameter. This is done in PR #2688. - [ ] Wherever we reset runtime parameters in
read_params()
orCastro_setup.cpp
, also update the struct - [ ] Convert all of the Castro functions to access the parameters from the new struct instead of the global header. We can do this a bit at a time.
- [ ] Once all of the code is updated, remove the global managed variables
We will need to separately do the same for Microphysics, but we can do that after this.
Note: for GPU kernels, we will need to pass the parameter struct as a lambda-captured copy down through any GPU functions we call in each kernel.
actually, it looks like in Castro_setup.cpp
we do some modifications to the parameters, so we should move the sync to there
most of the infrastructure is now in place for this.
we will be able to do this on Intel / SYCL: https://github.com/AMReX-Codes/amrex/pull/4056
we will still want to do #2701