Loess.jl icon indicating copy to clipboard operation
Loess.jl copied to clipboard

Loess crashes with StackOverflowError when missing values are present

Open dlakelan opened this issue 2 years ago • 2 comments

julia> loess([1,2,3],[4,5,6])
Loess.LoessModel{Float64}([1.0; 2.0; 3.0;;], [4.0, 5.0, 6.0], [2.5714285714285725 1.6428571428571432 -0.21428571428571436; 1.741935483870967 2.0483870967741926 -0.20967741935483827; 2.5714285714285716 1.642857142857142 -0.2142857142857134; 0.23809523809523817 0.47619047619047633 0.9523809523809526], Dict([1.5] => 1, [3.0] => 2, [1.0] => 3, [2.0] => 4), Loess.KDTree{Float64}([1.0; 2.0; 3.0;;], [1, 2, 3], Loess.KDInternalNode{Float64}(1, 2.0, Loess.KDInternalNode{Float64}(1, 1.5, Loess.KDLeafNode(), Loess.KDLeafNode()), Loess.KDLeafNode()), Set([[1.5], [3.0], [1.0], [2.0]]), [1.0; 3.0;;]))

julia> loess([missing,1,2,3],[4,missing,5,6])
ERROR: StackOverflowError:
Stacktrace:
     [1] loess(xs::Vector{Union{Missing, Float64}}, ys::Vector{Union{Missing, Float64}})
       @ Loess ~/.julia/packages/Loess/dv7Ad/src/Loess.jl:119
     [2] loess(xs::Vector{Union{Missing, Float64}}, ys::Vector{Union{Missing, Float64}}; kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
       @ Loess ~/.julia/packages/Loess/dv7Ad/src/Loess.jl:121--- the last 2 lines are repeated 39990 more times ---

This appears to be due to trying to convert the values and then call loess recursively on the new type.

Should probably either throw an error, or have an "ignore missing" option which just drops the x,y values where either the x or the y or both are missing.

dlakelan avatar Oct 11 '22 21:10 dlakelan

Yes, have the same issue, cf. also https://discourse.julialang.org/t/loess-with-missing-values/90921

mreichMPI-BGC avatar Nov 28 '22 21:11 mreichMPI-BGC

Note that it seems to occur even if the vectors have no missing values, so long as their type is something like Vector{Union{Missing,Float64}} so stripping missings manually isn't enough, you also need to convert the result to Vector{Float64}

dlakelan avatar Jun 10 '24 21:06 dlakelan