pangeo-forge-recipes icon indicating copy to clipboard operation
pangeo-forge-recipes copied to clipboard

Port EOOffshore CMEMS ASCAT REP and NRT recipes to use `beam-refactor` branch

Open derekocallaghan opened this issue 2 years ago • 2 comments

Following approach in #445, this will port the CMEMS ASCAT REP and NRT recipes to use the beam-refactor branch. It will involve writing at least one custom PTransform for required Dataset preprocessing.

derekocallaghan avatar Dec 22 '22 10:12 derekocallaghan

It looks like two PTransform classes will be required:

  • Similar to the CCMP recipe, the first transform will preprocess each Dataset, including replacing the provided time coordinates.
    • For now, the preprocess function can be wrapped with pangeo_forge_recipes.transforms._add_keys().
  • The second transform will sort the PCollection using the updated time coordinates. I still need to confirm the best approach here, but it's likely that it will update the Position in each Index, something similar to what currently occurs in pangeo_forge_recipes.transforms.IndexItems.

derekocallaghan avatar Dec 22 '22 11:12 derekocallaghan