Re: Bug in `warp` function for an array with NaN?

[mdhe1248]

Hi all, If NaN exists at the last row or column of an array, warp results in a wrong outcome: It seems to add more NaNs in the outcome array. Initially, I found this while playing with two sequential warping.

using Images, CoordinateTransformations

img = rand(10,10)
tfm = Translation(1,1)
imgw1 = warp(img, tfm, axes(img))

tfm = Translation(0,0)
imgw2 = warp(imgw1, tfm, axes(img))
@test isequal(imgw1, imgw2)

Then, it looks like that NaN somehow interferes the outcome of warp.

#### Tests
# NaN at the First row: Okay
A = rand(10,10)
A[:,1] .= NaN
tfm = Translation(0,0)
B = warp(A, tfm, axes(img))
@test isequal(A, B)

# First column: Okay
A = rand(10,10)
A[1,:] .= NaN
tfm = Translation(0,0)
B = warp(A, tfm, axes(img))
@test isequal(A, B)

# Last row: Not okay
A = rand(10,10)
A[:, end] .= NaN
tfm = Translation(0,0)
B = warp(A, tfm, axes(img))
@test isequal(A, B)

# Last column: Not okay
A = rand(10,10)
A[end, :] .= NaN
tfm = Translation(0,0)
B = warp(A, tfm, axes(img))
@test isequal(A, B)

# Last row and column: Not okay
A = rand(10,10)
A[end, :] .= A[:, end] .= NaN
tfm = Translation(0,0)
B = warp(A, tfm, axes(img))
@test isequal(A, B)

# Single NaN at the last row: Not okay
A = rand(10,10)
A[end, 1] = NaN
tfm = Translation(0,0)
B = warp(A, tfm, axes(img))
@test isequal(A, B)

# Single NaN at the last column: Not okay
A = rand(10,10)
A[1,end] = NaN
tfm = Translation(0,0)
B = warp(A, tfm, axes(img))
@test isequal(A, B)

Does anyone have thoughts? It might be an issue related to OffsetArray, but I am not sure. Also, please advise me if this weird outcome is from my mistake.

[mdhe1248]

Apparently, there was a similar issue: https://github.com/JuliaImages/ImageTransformations.jl/issues/45 Maybe the issue is related to interpolations. Any thoughts or solutions?

[zygmuntszpak]

Hi, Thanks for reporting this issue. You are right that this is indeed related to interpolations (see https://github.com/JuliaMath/Interpolations.jl/issues/192).

A similar issue was reported in scipy, and Cody-G pointed out:

I think the reason for this behavior is the same as discussed in an issue I opened a while ago in the Julia interpolations package: JuliaMath/Interpolations.jl#192

TL;DR it's a performance optimization. It avoids the branching logic that would be required to keep the NaN value out of the computation at grid points.

So the bottom line seems to be that you cannot warp an image that contains NaN without it potentially propagating more NaN values.

Could you describe your particular use case? Does your original input image contain NaN values, or are you trying to apply a sequence of warps and the very first warp introduces NaN values?

If your original input doesn't contain NaN values, and you want to apply a sequence of warps, then you could keep your original image around, compose the sequence of warps, and apply it to your original image.