Double gyre example

[navidcy]

This needs work but following suggestion, I submit a PR and we all work together... :)

Closes #678.

[codecov[bot]]

Codecov Report

:exclamation: No coverage uploaded for pull request base (main@fad81b0). Click here to learn what that means. Patch has no changes to coverable lines.

:exclamation: Current head 32a06cb differs from pull request most recent head 94fdef6. Consider uploading reports for the commit 94fdef6 to get more accurate results

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1085   +/-   ##
=======================================
  Coverage        ?   57.31%           
=======================================
  Files           ?      162           
  Lines           ?     3882           
  Branches        ?        0           
=======================================
  Hits            ?     2225           
  Misses          ?     1657           
  Partials        ?        0           

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

[ali-ramadhan]

This would make an awesome example!

By modifying the parameters it could also be used at https://github.com/CliMA/OceanModelComparison.jl to compare against the double gyre solution produced by ClimateMachine.jl.

[glwagner]

By modifying the parameters it could also be used at https://github.com/CliMA/OceanModelComparison.jl to compare against the double gyre solution produced by ClimateMachine.jl.

Indeed. I also want to see if I can get some eddies and do some time-averaging of the barotropic flow for pedagogical purposes.

[navidcy]

Any chance we use vertically stretched grid?

[glwagner]

Stretched grid implementation isn't complete...

[navidcy]

Hint, hint?

[glwagner]

We can't use a vertically stretched grid for this example because Oceananigans does not support vertically stretched grids.

[francispoulin]

Does this work without a vertically stretched grid?

[glwagner]

Yes, a double gyre simulation can be run on a regular grid.

[navidcy]

@francispoulin, yeah, why not? :) One might require more vertical grid points or what not, but I don't see any reason why it shouldn't.

[francispoulin]

Thanks. It seemed that was on the wish list but wasn't sure if anything bad was happening without the stretched grid.

I will hopefully run this example sometime soon and see the resutls myself.

[navidcy]

It's been on the wish list. The problem (if I recall correctly) is that we haven't quite yet found the parameters that would make this work and still be relatively "cheap" to add as an example in the Docs that would run automatically with Literate.jl :)

[glwagner]

Last we talked, we decided that the most appropriate way forward with this example would be to pick it up from a checkpoint, and add the checkpointed data to the repository. This would make it so the example could be run in a reasonable amount of time, and also provide an example of checkpointing.

[navidcy]

We should resurrect now with GPU on triply-Bounded topology + VerticallyStretchedGrid. :)

[ali-ramadhan]

Out of curiousity, is the plan for this PR to add an example as an eddying baroclinic double gyre on the GPU with IncompressibleModel while #1433 adds a viscous barotropic double gyre with HydrostaticFreeSurfaceModel as a validation experiment?

[glwagner]

There's no plan! We should make one.

[francispoulin]

Out of curiousity, is the plan for this PR to add an example as an eddying baroclinic double gyre on the GPU with IncompressibleModel while #1433 adds a viscous barotropic double gyre with HydrostaticFreeSurfaceModel as a validation experiment?

As a first step why not set up both IncompressibleModel and HydrostaticFreeSurfaceModel for a barotropic double gyre? Since there will be much in the way of nonhydrostatic effects, it would be a way to compare the two models in terms of the accuracy and the efficiency. Then it should be easy to change both to do a baroclinic double gyre.

[navidcy]

@glwagner, this should be feasible now with the but in the VerticallyStretchedRectilinearGrid and also with the HydrostaticFreeSurfaceModel with ImplicitFreeSurface().

I made some changes. Should we zoom and finish it off?

[glwagner]

@glwagner, this should be feasible now with the but in the VerticallyStretchedRectilinearGrid and also with the HydrostaticFreeSurfaceModel with ImplicitFreeSurface().

I made some changes. Should we zoom and finish it off?

This would be nice to have. Let's do it.

Should we run a short spin-up, or illustrate checkpointing with an equilibrated solution?

[navidcy]

Yes. But before doing that let's make sure it runs! It was blowing up when I was trying before. I may have done something wrong with forcing/bcs?

[glwagner]

Yes. But before doing that let's make sure it runs! It was blowing up when I was trying before. I may have done something wrong with forcing/bcs?

That's a good starting point...

[navidcy]

Some progress here...

https://user-images.githubusercontent.com/7112768/151718447-2a5fd619-3c4c-43c8-9aa5-29c97e0129b4.mp4

[navidcy]

[navidcy]

We need to think how we do this. We need a long run to produce a good figure. E.g., 100 years?

Should we run this somewhere and save output and then in this example show how we can load the output and plot?

Or should we run this somewhere for 95 years and save a checkpoint and then use this example to demonstrate how we can restart from a checkpoint and run for 5 years to produce animations/plots?

[navidcy]

If we use checkpointer then the PR will close #1136.

[francispoulin]

Great to see you working on this @navidcy !

Some questions/comments:

I think having an animation is a great idea. Any particular reason you are plotting u? Traditionally, people plot the streamfunction (pressure) and or the voriticty. What do you think of including the streamfunction in the animation?

From looking at the animation I was surprised that we don't see a double gyre. But from your next image we see there is a double gyre in the barotropic streamfunction. This makes me think that if we plot the total streamfunction (or maybe even the barotropic and baroclinic components) we should get images that are more like what I've seen in papers.

How long did this take you to run and what resolution did you use?

[glwagner]

I think we should run it at a reasonably high resolution to produce eddies, and then checkpoint the run and upload it to OceananigansArtifacts.jl. Then we can download the checkpoint in the example, restart from there and run maybe 100 iterations to make a (short) animation?

It might make sense to revisit some of the more expensive examples and reduce their cost too (until we find a way to run examples on tartarus, or some other way to make them less expensive).

[navidcy]

@francispoulin, I agree! I'd like to plot movie of barotropic streamfunction or free surface and the relative vorticity. Perhaps total speed also. (I just plotted u and v because it was the easiest... just wanted to see whether things blow up or have grid-scale noise and what not.)

[navidcy]

How long did this take you to run and what resolution did you use?

The animation above is a 10 year run starting from rest.

With a resolution of Nx = 160, Ny = 240, and Nz = 50 it took bit less than an hour on the GPU (or something like that).

[hdrake]

Would it make sense to initialize with the analytical solution to the (linear or weakly nonlinear) barotropic problem? Even if a simple Laplacian viscosity is a poor parameterization for eddy effects, it should dramatically accelerate the equilibration process, right?