User interface to `Scented` and `Unscented` Kalman inversions

[glwagner]

Constructors

The user interface to scented and unscented inversions defines two constructor-looking functions UnscentedKalmanInversion and UnscentedKalmanInversionPostprocess:

https://github.com/CliMA/OceanTurbulenceParameterEstimation.jl/blob/111e4c50fe33d46d9f7061e3256b9b16d70682c5/src/EnsembleKalmanInversions.jl#L204

https://github.com/CliMA/OceanTurbulenceParameterEstimation.jl/blob/111e4c50fe33d46d9f7061e3256b9b16d70682c5/src/EnsembleKalmanInversions.jl#L240

However, there are no objects / structs with names corresponding to those constructors.

Should we tweak the interface a little bit? It looks like there is a type called Unscented in EnsembleKalmanProcesses.jl:

https://github.com/CliMA/OceanTurbulenceParameterEstimation.jl/blob/111e4c50fe33d46d9f7061e3256b9b16d70682c5/src/EnsembleKalmanInversions.jl#L224

We could have users construct this object and pass to EnsembleKalmanInversion. It looks like the constructor for EnsembleKalmanProcess is slightly inconsistent for scented vs unscented inversion --- the scented flavor is

https://github.com/CliMA/OceanTurbulenceParameterEstimation.jl/blob/111e4c50fe33d46d9f7061e3256b9b16d70682c5/src/EnsembleKalmanInversions.jl#L155

We can still handle this.

Post-processing

If we want to keep the post-processing function, I think we should call it something like postprocess(eki::EnsembleKalmanInversion); then we can do the right thing whether eki represents a scented or unscented process.

However, another possibility is to give a kwarg to iterate!:

function iterate!(eki; iterations=1, postprocess=false)

when users select postprocess=true, we post-process results and store them inside the EKI object.

What do others think about these ideas? @adelinehillier @Zhengyu-Huang @navidcy

[adelinehillier]

Yes, good ideas! I think the existing UnscentedKalmanInversionPostprocess can be absorbed into IterationSummary. The arrays returned by UnscentedKalmanInversionPostprocess are

`mean :: Matrix{Float64}`: `N_iter` × `N_θ` mean matrix
`cov :: Vector{Matrix{Float64}}`: `N_iter` vector of `N_θ` × `N_θ` covariance matrix
`std :: Matrix{Float64}`: `N_iter` × `N_θ` standard deviation matrix
`err :: Vector{Float64}`: `N_iter` error array

We already store mean and std in IterationSummary, but cov and err would be good things to add. We should also consider whether we are storing the constrained or unconstrained versions of these variables and may want to store the inverse_parameter_transform-ed versions as UnscentedKalmanInversionPostprocess does.

Do you guys agree?

[glwagner]

Yes, good ideas! I think the existing UnscentedKalmanInversionPostprocess can be absorbed into IterationSummary. The arrays returned by UnscentedKalmanInversionPostprocess are

`mean :: Matrix{Float64}`: `N_iter` × `N_θ` mean matrix
`cov :: Vector{Matrix{Float64}}`: `N_iter` vector of `N_θ` × `N_θ` covariance matrix
`std :: Matrix{Float64}`: `N_iter` × `N_θ` standard deviation matrix
`err :: Vector{Float64}`: `N_iter` error array

We already store mean and std in IterationSummary, but cov and err would be good things to add. We should also consider whether we are storing the constrained or unconstrained versions of these variables and may want to store the inverse_parameter_transform-ed versions as UnscentedKalmanInversionPostprocess does.

Do you guys agree?

Ah, so the post-process stuff are properties of a single iteration (rather than many iterations)? In that case I agree adding to IterationSummary is cleanest here.

[glwagner]

I think the column model case has the greatest "symmetry" between the amount of data / cost of the ensemble_simulation versus the size / cost of computing particle statistics. And since even in that case the cost and size of particle statistics is completely negligible, it probably makes sense to compute everything we think will be useful later as a convenience.

[glwagner]

It looks like ensemble_cov is now a property of IterationSummary:

https://github.com/CliMA/OceanTurbulenceParameterEstimation.jl/blob/08f65160758a21e9b14252d7f4d2eaa2e38561d6/src/EnsembleKalmanInversions.jl#L271

Do we still need UnscentedKalmanInversionPostprocess?