staged-recipes icon indicating copy to clipboard operation
staged-recipes copied to clipboard

Add CMEMS SLA recipe

Open ktyle opened this issue 2 years ago • 5 comments

This recipe's inent is to supersede the initial efforts of @cisaacstern 's #55.

Since this is my first attempt at a recipe, I have a few questions:

  1. What is the current best practice to deal with sites such as Copernicus which require a user id and password to access their FTP service?
  2. In meta.yaml, what should be the value of pangeo-notebook-version?
  3. What is the appropriate license type? Copernicus looks to have their own spin on it (see the meta.yml file for a link to their license verbiage).
  4. I'm not sure what the best chunking strategy is here. I went with 2 items per chunk.

ktyle avatar Jul 25 '22 15:07 ktyle

:tada: New recipe runs created for the following recipes at sha 390834fce7bcff5308aec40a9871108bee3a67c5:

  • cmems-sla: https://pangeo-forge.org/dashboard/recipe-run/1003

pangeo-forge-bot avatar Jul 25 '22 15:07 pangeo-forge-bot

Hi @cisaacstern and @ktyle,

I'm looking into writing a CMEMS ASCAT recipe, and wanted to check the status of this SLA recipe (and also https://github.com/pangeo-forge/staged-recipes/pull/55)? I previously retrieved the data products using motuclient.py, but was going to look at porting this to the ftp approach you've both used. As with this current recipe, I'm unsure how to specify the user/password credentials for recipe execution, is there a convention for this?

Thanks, Derek

derekocallaghan avatar Aug 12 '22 14:08 derekocallaghan

Hi All. Sorry for letting this hang for so long. Since SciPy I have been on near constant travel.

The fact is that we don't have a robust system to securely manage credentials. In the meantime, I invite you to hard-code my own Copernicus credentials into the recipe.py file.

username: rabernathey password: oCg!25s%DN^M

You can pass these to your FilePattern via the fsspec_open_kwargs argument, e.g.

pattern = FilePattern(
    ...,
    fsspec_open_kwargs={"username": "rabernathey", "password": "oCg!25s%DN^M"}
)

rabernat avatar Aug 12 '22 15:08 rabernat

Thanks @rabernat

derekocallaghan avatar Aug 12 '22 15:08 derekocallaghan

cool, thanks @rabernat ... I will update the PR with the Copernicus creds you included.

ktyle avatar Aug 23 '22 14:08 ktyle