CalibrateEmulateSample.jl icon indicating copy to clipboard operation
CalibrateEmulateSample.jl copied to clipboard

[WIP] convergence plots for Emulator examples & a bugfix

Open odunbar opened this issue 1 year ago • 1 comments

Purpose

Adds convergence plots over many repeats of Emulator experiments Closes #278

To-do

  • improve the extraction of EKI diagnostics or other information from training (e.g. EKI itself?) currently get_optimizer only returns diagnostics
  • neaten plots (e.g. start them at iteration 0, consistent axis labels etc)

Content

  • adds repeats for the EDMF example
  • adds convergence plots (normalized by the initiali iteration to error=1) for Emulator/Ishigami, Emulator/L63, and EDMF_data/
  • adds a way of extracting the error of EKI
  • bugfixes the scalar RF index error

Example convergence plots

From the current example library - the plots look like this:

Ishigami (14 params)

ishigami_eki-conv_RF-scalar

L63 (41 params) - NB log-scale

l63_eki-conv_test

EDMF (634 params)

eki-conv_RF-vector-nosvd-nonsep


  • [ ] I have read and checked the items on the review checklist.

odunbar avatar Feb 07 '24 02:02 odunbar

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 88.35%. Comparing base (acf1051) to head (42ca861). Report is 17 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #279      +/-   ##
==========================================
+ Coverage   88.17%   88.35%   +0.18%     
==========================================
  Files           8        8              
  Lines        1184     1194      +10     
==========================================
+ Hits         1044     1055      +11     
+ Misses        140      139       -1     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov[bot] avatar Feb 07 '24 03:02 codecov[bot]

Merging to support code availability in the submission https://arxiv.org/abs/2407.00584

odunbar avatar Jul 13 '24 10:07 odunbar