cosima-recipes icon indicating copy to clipboard operation
cosima-recipes copied to clipboard

Added an example for along-isobath averaging

Open taimoorsohail opened this issue 1 year ago • 12 comments

Hi, I am posting a new Contributed Example which averages properties along an isobath - useful for Antarctic margins analysis. Let me know if you have thoughts!

closes https://github.com/COSIMA/cosima-recipes/issues/397

taimoorsohail avatar Jul 08 '24 08:07 taimoorsohail

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

Seems that you added two files?

navidcy avatar Jul 08 '24 09:07 navidcy

Yeah sorry - I accidentally included a random change I made to the ContributedExamples/Cross-slope_section_potrho_cc.ipynb file - please delete if possible before pushing the new file :)

taimoorsohail avatar Jul 09 '24 04:07 taimoorsohail

@taimoorsohail is there a reason you wanted to make this a "Contributed Example"?

We decided to just have "Examples" and they should all be documented and in good shape; see #407.

navidcy avatar Jul 09 '24 04:07 navidcy

Ah sorry, I didn't know that we had moved on! Yes happy to add it to "Examples" if everyone is happy with the level of documentation and commenting.

taimoorsohail avatar Jul 09 '24 04:07 taimoorsohail

if everyone is happy with the level of documentation and commenting.

We'll let the review process determine that. :)

navidcy avatar Jul 09 '24 04:07 navidcy

Hey @taimoorsohail, out of interest, is it necessary to use st_edges_ocean for the bins? Would it be possible in theory to use a more even binning (say spaced every 50m in isobath depth) and then use ht/hu for computing the histogram?

adele-morrison avatar Jul 11 '24 10:07 adele-morrison

Currently the time average is done just before plotting, but the time information is not used. Perhaps time averaging and loading earlier could make it run faster?

adele-morrison avatar Jul 11 '24 20:07 adele-morrison

Also, if you edit your top post to add: closes https://github.com/COSIMA/cosima-recipes/issues/397, it will automatically close the issue when this is merged.

adele-morrison avatar Jul 11 '24 20:07 adele-morrison

Hey @taimoorsohail, out of interest, is it necessary to use st_edges_ocean for the bins? Would it be possible in theory to use a more even binning (say spaced every 50m in isobath depth) and then use ht/hu for computing the histogram?

Hey @adele-morrison! In theory, it should be OK. I had some issues masking ht/hu with st_edges_ocean for the bins, as ht and hu cover a larger range of depths than the bottom depth of the variables (see https://forum.access-hive.org.au/t/difference-between-bottom-bathymetry-variable-ht-and-actual-data-in-access-om2-01/2221). This meant I was getting more "jagged" averages which only really affected the "prettiness" of the visualisation. I didn't try reducing the bin spacing though.

Currently the time average is done just before plotting, but the time information is not used. Perhaps time averaging and loading earlier could make it run faster?

Won't time-averaging after binning account for the temporal variability within each bin? Whereas time-averaging before binning would get rid of that? I'm not sure...

taimoorsohail avatar Jul 12 '24 03:07 taimoorsohail

Hey @taimoorsohail how is this one going? This would be super useful code for a bunch of people, so would be great to get it included!

adele-morrison avatar Sep 01 '24 10:09 adele-morrison

Hi @adele-morrison thanks for the prod! I think I was confused how to handle varying vertical grids in this code. At the moment, I bin using a new bottom depth variable I create based on the last depth where T/S/rho data is available - this aligns with st_edges_ocean. In reality, the ht/hu variable contains the 'actual' bottom depth. It would be more accurate to use this value, but would involve stretching/shrinking the T/S/rho profiles, right? Anyway, your thoughts are welcome...

taimoorsohail avatar Sep 03 '24 05:09 taimoorsohail

I don't have strong opinions on the vertical grid question. But even leaving it as is and opening an issue for further discussion after the PR is merge is totally OK.

I do see a lot of plots though and a comment to reduce them. Could we simplify the notebook @taimoorsohail? I'm happy to have a look after -- let me know.

Just mentioning that this PR should also include an entry for you @taimoorsohail in the .zenodo.json at https://github.com/taimoorsohail/cosima-recipes/blob/main/.zenodo.json so that your name appears in all future releases in the Zenodo entry after this PR is merged.

navidcy avatar Jan 22 '25 20:01 navidcy

Thanks again both for prodding me on this. I have cleaned up the file and reduced the number of figures as per Adele's suggestion! Please feel free to review @navidcy. I will add an issue about the adaptive vertical grid for the next hackathon :)

taimoorsohail avatar Jan 24 '25 04:01 taimoorsohail

OK I have pushed a fix - does this work?

taimoorsohail avatar Jan 28 '25 06:01 taimoorsohail

OK I have pushed a fix - does this work?

Still I hit a wall opening the files... I pushed the file with the errors I get.

But for you it works??? Can you ensure that before you push you restart the kernel and run all cells sequentially (that's what I'm trying to do). Also could you tell me what resources you have on the notebook when you run it? [I'm using gadi_jupyter with ncpus=48, i.e. a whole node at the normal queue.]

navidcy avatar Jan 28 '25 19:01 navidcy

I've pushed the error I am getting when running the code. I think it has to do with accessing the variables using COSIMA cookbook - there may be a bug I don't know about? Any ideas @anton-seaice @dougiesquire or others at NRI?

taimoorsohail avatar Jan 29 '25 01:01 taimoorsohail

Yeah... I've been confused by this myself... any help is welcomed!

navidcy avatar Jan 29 '25 01:01 navidcy

Are we sure it's the Cookbook?

Have you tried with intake @taimoorsohail?

navidcy avatar Jan 29 '25 01:01 navidcy

That error normally means you need to set threads_per_worker = 1 as an argument in client = Client()

There is an long term bug in the netcdf libraries that causes this.

See https://github.com/COSIMA/cosima-recipes/issues/409

anton-seaice avatar Jan 29 '25 02:01 anton-seaice

Thanks! I'll try that!

navidcy avatar Jan 29 '25 03:01 navidcy

Thanks @taimoorsohail for the recipe! Welcome to the cookbook!

I will add an issue about the adaptive vertical grid for the next hackathon :)

Please remember to raise that issue mentioned above. Not necessarily "for the next hackathon" though ;) The issue can be taken up from anybody at anytime, even today!

navidcy avatar Jan 29 '25 03:01 navidcy

Just a small note: in 7defbf0 also changed a bunch of code comments to markdown text. I find that markdown text with formatting etc is much more user friendly to the eye than Python # comments within the code.

navidcy avatar Jan 29 '25 03:01 navidcy