Wrong Normalization

[ValentinKaisermayer]

The normalized variant of the RBF seems to be wrong.

function normalized_squared_exponential(v, vc, gamma::Number)
    r = squared_exponential(v, vc, gamma)
    r ./= (sum(r, dims=2) .+ 1e-8)  
end

Which yields the incorrect extrapolation behaviour:

The correct implementation would be

function normalized_squared_exponential(v, vc, gamma::Number)
    r = squared_exponential(v, vc, gamma)
    r ./= (sum(r, dims=2))  
end

Which yields:

However, for points that are really far outside of the support of the RBF this does not work either. I have a somewhat hacked solution:

function normalized_squared_exponential(v, vc, gamma::Number)
    r = squared_exponential(v, vc, gamma)
    s = vec(sum(r, dims=2))
    r ./= s
    r[isnan.(r)] .= 0.
    r[(s .<= eps()) .& (v .< vc[1]), 1] .= 1.
    r[(s .<= eps()) .& (v .> vc[end]), end] .= 1.
    r
end

This assumes that vc is ordered increasingly.

[ValentinKaisermayer]

The more I think about it, the more I come to the conclusion that simply returning NaN in the numerical unstable region would be useful. I think the user has to know that something went wrong.

[baggepinnen]

Would it work to just set the largest component to 1 whenever the sum is less than some value?

[ValentinKaisermayer]

I'm not sure. I think this would just extend the domain a bit.

[ValentinKaisermayer]

I think the correct approach would be to compute an augmented point that lies inside the computational stable area and return the function value for this point. This would also work in the 2D case. Since there it is not as easy as setting something to 1.