PSyclone icon indicating copy to clipboard operation
PSyclone copied to clipboard

get_kernel_schedule() fails for some LFRic kernels

Open sergisiso opened this issue 1 year ago • 2 comments

There are multiple issues:

  1. Can not find the name of the subroutine when doing the lookup in the kernel file (e.g. with mixed precision)

  2. The validate_kernel_code_args() fails. For example in:

psyclone -api dynamo0.3 -l all -d working \
             -s /home/sergi/workspace/stfc/LFRic/trunk/gungho/optimisation/meto-spice//global.py \
             -opsy working/lfric_xios_setup_mod_psy.f90 -oalg working/lfric_xios_setup_mod.f90 working/lfric_xios_setup_mod.x90
Transforming invoke 'invoke_0_nodal_xyz_coordinates_kernel_type' ...
Generation Error: Generator: script file '/home/sergi/workspace/stfc/LFRic/trunk/gungho/optimisation/meto-spice//global.py'
raised the following exception during execution ...
{
      File "/home/sergi/workspace/stfc/LFRic/trunk/gungho/optimisation/meto-spice/global.py", line 72, in trans
    inline_trans.apply(kernel)
      File "/home/sergi/workspace/stfc/PSyclone/src/psyclone/transformations.py", line 1466, in apply
    self.validate(node, options)
      File "/home/sergi/workspace/stfc/PSyclone/src/psyclone/transformations.py", line 141, in validate
    kernel_schedule = kern.get_kernel_schedule()
      File "/home/sergi/workspace/stfc/PSyclone/src/psyclone/dynamo0p3.py", line 8448, in get_kernel_schedule
    self.validate_kernel_code_args()
      File "/home/sergi/workspace/stfc/PSyclone/src/psyclone/dynamo0p3.py", line 8471, in validate_kernel_code_args
    interface_info.generate()
      File "/home/sergi/workspace/stfc/PSyclone/src/psyclone/domain/lfric/kernel_interface.py", line 117, in generate
    super(KernelInterface, self).generate(var_accesses=var_accesses)
      File "/home/sergi/workspace/stfc/PSyclone/src/psyclone/domain/lfric/arg_ordering.py", line 292, in generate
    self.basis(unique_fs, var_accesses=var_accesses)
      File "/home/sergi/workspace/stfc/PSyclone/src/psyclone/domain/lfric/kernel_interface.py", line 536, in basis
    self._create_basis(function_space, self.basis_mapping,
      File "/home/sergi/workspace/stfc/PSyclone/src/psyclone/domain/lfric/kernel_interface.py", line 774, in _create_basis
    raise NotImplementedError(
    NotImplementedError: Evaluator shapes not implemented in kernel_interface class.

or in:

psyclone -api dynamo0.3 -l all -d working \
             -s /home/sergi/workspace/stfc/LFRic/trunk/gungho/optimisation/meto-spice//global.py \
             -opsy working/algorithm/galerkin_projection_algorithm_mod_psy.f90 -oalg working/algorithm/galerkin_projection_algorithm_mod.f90 working/algorithm/galerkin_projection_algorithm_mod.x90
Transforming invoke 'invoke_0' ...
Found 1 setval calls
Transforming invoke 'invoke_1' ...
Found 2 setval calls
Transforming invoke 'invoke_2_gp_rhs_kernel_type' ...
Generation Error: Generator: script file '/home/sergi/workspace/stfc/LFRic/trunk/gungho/optimisation/meto-spice//global.py'
raised the following exception during execution ...
{
      File "/home/sergi/workspace/stfc/PSyclone/src/psyclone/dynamo0p3.py", line 8448, in get_kernel_schedule
    self.validate_kernel_code_args()
      File "/home/sergi/workspace/stfc/PSyclone/src/psyclone/dynamo0p3.py", line 8471, in validate_kernel_code_args
    interface_info.generate()
      File "/home/sergi/workspace/stfc/PSyclone/src/psyclone/domain/lfric/kernel_interface.py", line 117, in generate
    super(KernelInterface, self).generate(var_accesses=var_accesses)
      File "/home/sergi/workspace/stfc/PSyclone/src/psyclone/domain/lfric/arg_ordering.py", line 292, in generate
    self.basis(unique_fs, var_accesses=var_accesses)
      File "/home/sergi/workspace/stfc/PSyclone/src/psyclone/domain/lfric/kernel_interface.py", line 536, in basis
    self._create_basis(function_space, self.basis_mapping,
      File "/home/sergi/workspace/stfc/PSyclone/src/psyclone/domain/lfric/kernel_interface.py", line 736, in _create_basis
    basis_tag, [int(first_dim_value_func(function_space)),
      File "/home/sergi/workspace/stfc/PSyclone/src/psyclone/dynamo0p3.py", line 4578, in basis_first_dim_value
    raise GenerationError(
    psyclone.errors.GenerationError: Generation Error: Unsupported space for basis function, expecting one of ['w3', 'wtheta', 'w2v', 'w2vtrace', 'w2broken', 'w0', 'w1', 'w2', 'w2trace', 'w2h', 'w2htrace', 'any_w2', 'wchi'] but found 'any_space_1'
}

sergisiso avatar Aug 03 '22 13:08 sergisiso

Currently the mixed-precision support in PSyclone only has to worry about using the correct precision when declaring variables in the PSy layer - we rely on the standard Fortran functionality for the compiler to choose the correct kernel routine according to the type of the arguments. However, when we come to module inline then either we have to do what the Fortran compiler would do (and figure out which version of the kernel is being called) or we inline all versions of the kernel and add a suitable interface block.

arporter avatar Sep 28 '22 07:09 arporter

Determining the correct routine won't be easy because the various kernel routines are normally implemented in terms of explicit 32- and 64-bit datatypes while the algorithm and PSy-layer use the LFRic type definitions (r_def, r_solver, etc.). We would need a mapping between LFRic types and their actual precision for a given compilation. Despite that, it's still probably more immediately do-able than adding an interface block since we have no PSyIR support for such a construct.

arporter avatar Sep 28 '22 07:09 arporter