OceanBioME.jl icon indicating copy to clipboard operation
OceanBioME.jl copied to clipboard

Regression tests to check models are still running without having to check example plots

Open jagoosw opened this issue 10 months ago • 1 comments

I've been trying to work out how to check we don't break models between commits because sometimes it becomes apparent that something has changed only when you look at the example plots and noticing they look wrong.

For example, when I recently changed the PAR API I only noticed that it wasn't working properly by looking at one of the box model example outputs.

I guess a solution to this is to improve unit testing, but that requires us to foresea the problems. Another idea I had was to run regression tests so if model outputs change we know, and can hunt down the cause.

I started writing this here: https://github.com/OceanBioME/OceanBioME.jl/tree/jsw/regression-tests/test/regression_tests but was wondering if anyone else thought it was worthwhile?

I'm dubious about calling them regression tests like Oceananigans does because we're not checking the models fit some physical/analytical result but just that they don't change each time.

jagoosw avatar Sep 07 '23 15:09 jagoosw