Remove HyperParameter?

[devmotion]

Hej!

I like this package, and in principle I think it could be a nice building block for different GP packages such as https://github.com/PieterjanRobbe/GaussianRandomFields.jl or https://github.com/STOR-i/GaussianProcesses.jl. But I'm not sure whether the use of HyperParameters is actually needed; to me it seems it adds an additional layer of complexity that might not always be desired. My suggestion would be to follow the same approach as https://github.com/JuliaStats/Distributions.jl: restrict parameters <:Real of immutable kernel functions to certain intervals such as positive real line with the help of inner constructors. This would provide users with a very simple and completely flexible way of updating parameters by just creating new kernel functions with updated parameter values. For increased convenience one could even think about providing default transformations from https://github.com/tpapp/ContinuousTransformations.jl that would provide an easy way to map real values to required intervals and transforming likelihoods etc in a correct way.

[trthatcher]

Agreed, I would prefer simplicity as well.

For increased convenience one could even think about providing default transformations from https://github.com/tpapp/ContinuousTransformations.jl that would provide an easy way to map real values to required intervals and transforming likelihoods etc in a correct way.

What were you thinking for an interface here? Looking at the rational quadratic kernel and following Distributions.jl's lead:

"RationalQuadraticKernel(α,β) = (1 + α⋅‖x-y‖²)⁻ᵝ   α ∈ (0,∞), β ∈ (0,∞)"
struct RationalQuadraticKernel{T<:Real} <: MercerKernel{T}
    alpha::T
    beta::T
    function RationalQuadraticKernel{T}(α::T, β::T) where T
        @check_args(RationalQuadraticKernel, zero(α) < α)
        @check_args(RationalQuadraticKernel, zero(β) < β)
        new{T}(α, β)
    end
end

Were you thinking that something like the following functions (as an example) would be defined?

getparams(κ::RationalQuadraticKernel) = [t(κ.alpha); t(κ.beta)]
setparams(κ::RationalQuadraticKernel, p) = RationalQuadraticKernel(t_inv(p[1]), t_inv(p[2]))

where t and t_inv would be the transformation functions based on ContinuousTransformations.jl. Since a new struct would be created, the parameters would go through the checks embedded in the constructor.

This works for me (with some refinement). Thoughts?

[devmotion]

Yes, my rough idea was similar. However, my main idea was to work with untransformed values by default, i.e. to implement basically only the type definition you proposed and

RationalQuadraticKernel(alpha::T, beta::T) where T<:Real = 
    RationalQuadraticKernel{T}(alpha, beta)
RationalQuadraticKernel(alpha::Real, beta::Real) = 
    RationalQuadraticKernel(promote(alpha, beta)...)

params(x::RationalQuadraticKernel) = (x.alpha, x.beta)

Maybe it would even make sense to use NamedTuples?

I'm not sure whether it's actually helpful to define any other methods that allow the use of unconstrained real parameters. I assume that typically one can just define required transformations by using ContinuousTransformations.jl (or at some point its successor https://github.com/tpapp/TransformVariables.jl) as done in this example of DynamicHMC. The only thing that I thought could potentially be helpful is a set of either default transformations or parameter intervals for different kernels, such as e.g.

params_transformation(::RationalQuadraticKernel) = TransformationTuple(Exp(), Exp())

or

params_domain(::RationalQuadraticKernel) = (ℝ⁺, ℝ⁺)