The value range of guidemap is incorrect.

[onpix]

I check the PyTorch doc and found that in function grid_sample:

In the case of 5D inputs, grid[n, d, h, w] specifies the x, y, z pixel locations for interpolating output[n, :, d, h, w].

and

grid specifies the sampling pixel locations normalized by the input spatial dimensions. Therefore, it should have most values in the range of [-1, 1]. For example, values x = -1, y = -1 is the left-top pixel of input, and values x = 1, y = 1 is the right-bottom pixel of input.

, which means that guidemap_guide[m, d, h, w] should be in range [-1, 1]. In code, you normalize hg and wg to [-1, 1] but keep guidemap its original value in [0, 1]. Does it seem that guidemap should also be normalized to [-1, 1]?

[puneetmatharu]

I too noticed this and simply rescaled the values of guidemap_guide to the expected range (i.e [-1, 1]).

[creotiv]

This better to be tested. cause it's hard to say how grid_sample behave

[creotiv]

ive added bilateral_slice from original repo compiled for jit. But still has some problems with optimization for some reason. So i think grid_sample was working correctly

[QiuJueqin]

After some comparison with my customized tri-linear interpolation, which consists of multiple 2D bilinear interpolation, I'm now pretty sure that the second argument to F.grid_sample (grid) should be something like

torch.cat([wg, hg, guidemap], dim=3).unsqueeze(1)

instead of

torch.cat([hg, wg, guidemap], dim=3).unsqueeze(1)

Furthermore, elements in grid along all axes should be in [-1, 1] range, not [0, 1], which means in the guidance net, the activation should be torch.tanh, instead of torch.sigmoid.

The result of my customized slicing oprator is very similar to the F.grid_sample with inputs formatted mentioned above. The abs error is smaller than 1E-5:

all close with atol=1E-6:  False
all close with atol=1E-5:  True