COSIMA
Intake conversion Compare_SST_SSS_TemperatureSalinity_to_WOA13
Following the discussion in issue #313, we propose converting the recipes to use Intake, given that the Cookbook is no longer supported and the ACCESS-NRI Intake catalog is now available.
A few months ago, @max-anu began working on this transition. This pull request contains the changes @max-anu made to the notebook specified in the title.
Check out this pull request on 
See visual diffs & provide feedback on Jupyter Notebooks.
Powered by ReviewNB
![review-notebook-app[bot] avatar](https://avatars.githubusercontent.com/in/15474?v=4 "Published by a pub.dev verified publisher"){kind=small}
The changes on this notebook are not complete - Cosima Cookbook is still being used. Moving to update.
@marc-white yes as @julia-neme said, pretty much all of these PRs from @max-anu require substantial modification, not just reviewing. If you'd like to go ahead and make these changes to remove references to the cookbook in this notebook, please do! Many of us have run out of steam for dealing with these broken PRs.
WOA13 data are also missing from intake, see https://github.com/ACCESS-NRI/access-nri-intake-catalog/issues/190
Feel free to close some of the PRs if data is missing and there is light at the end of the tunnel!
Yeah, the recipes that use WOA for now load it manually from ik11. We can keep doing that.. or wait until the catalog is updated? I think we are moving away from using hh5 @marc-white , but woa13 is in /g/data/ik11/observations/woa13
@navidcy I'm quite happy to keep the PRs open where there are major issues outstanding, I think it's probably the easiest way to keep track of where the updates to each recipe are up to (although we should probably mark any that aren't ready for actual review as DRAFT).
@julia-neme I think loading manually from ik11 is OK as an interim solution. I'm not sure if there's an authority/guideline on what should and should not end up in access-nri-intake (@rbeucher ? @dougiesquire ?), but as a general proposition, I'd strongly recommend that any data that external users are expected to touch should be available through one system.
definitely we'd like to have everything in the catalog! if you can add them there do it...!
I don't know about established authorities... feel free to be the authority! what you suggest is good!
@marc-white, creating the ct11 catalog https://github.com/ACCESS-NRI/ESMValTool-workflow/issues/201 will help to resolve that one as the WOA13 and WOA18 data are in /g/data/ct11/access-nri/replicas/esmvaltool/obsdata-v2/Tier2/WOA
Other relevant issues: https://github.com/ACCESS-NRI/access-nri-intake-catalog/issues/250
@rbeucher it seems that only the 1 degree WOA data are in the ESMValTool CT11 data collection. Does this sound right to you?
If so, we'll probably want to build a separate datastore for the WOA stuff, including the other resolutions & vertical level schemes. This is also an issue with #524.
Yes, I was actually looking into this while testing the catalog. The WOA data we have in ct11 is very limited... It's probably just there for an ESMValTool recipe test case. We need to extend the period at a minimum.
COSIMA has some regridded versions in /g/data/ik11/observations/woa13
We could add them to the catalog
Yeah, we found that it was worthwhile remapping WOA to model grids, because we only had a few model grids to worry about and this way we only had to do it once. I think that exercise may be worth repeating every time we get a new WOA or a new grid? But it isn't urgent - we can use ik11 for now ...
Hi all,
So I had a look at this and I have a few issues.
@AndyHoggANU @aekiss I can't find in the catalog
- 1deg_jra55v13_iaf_spinup1_B1
- 025deg_jra55v13_iaf_gmredi6
- 01deg_jra55v13_iaf
@charles-turner-1 WOA13 is in the catalog but extracting the relevant files is far from intuitive. It works for now but I think we should revisit it.
I would go with the same runs we are using here as a good example: https://github.com/COSIMA/cosima-recipes/blob/main/ACCESS-OM2-GMD-Paper-Figs/Fig4-DrakePassageTransport.ipynb
Probably best to just use the last cycle, i.e.:
1deg_jra55_iaf_omip2_cycle5
025deg_jra55_iaf_omip2_cycle5
01deg_jra55v140_iaf_cycle4
and to select the years to be the same as the WOA dataset.
Thanks @AndyHoggANU. I think the conversion is complete. Could someone please take a quick look?
I agree 100 percent. @charles-turner-1 is away for 2 weeks so let's keep that for when he is back
Ahh, sorry I know what the issue is here - just couldn't reproduce it when testing the datastore & I thought I was imagining it.
From the snippets here, I think the parser will need to be updated to include the directory as well as the file names in the .parse_filename method.
I'll see if I can bang it out on the flight to NZ, if not hopefully this comment should be helpful.