tensorflow
[feature request] How to implement torch.nn.functional.grid_sample padding mode in Tensorflow?
https://stackoverflow.com/questions/52888146/what-is-the-equivalent-of-torch-nn-functional-grid-sample-in-tensorflow-numpy
How to implement padding mode?
System information
- TensorFlow version (use command below): 1.10.0
Describe the problem
How to implement torch.nn.functional.grid_sample padding mode in Tensorflow?
Source code / logs
https://github.com/tensorflow/models/blob/master/research/transformer/spatial_transformer.py
Thank you for your post. We noticed you have not filled out the following field in the issue template. Could you update them if they are relevant in your case, or leave them as N/A? Thanks. What is the top-level directory of the model you are using Have I written custom code OS Platform and Distribution TensorFlow installed from Bazel version CUDA/cuDNN version GPU model and memory Exact command to reproduce
https://stackoverflow.com/questions/52888146/what-is-the-equivalent-of-torch-nn-functional-grid-sample-in-tensorflow-numpy
How to implement padding mode?
System information
- TensorFlow version (use command below): 1.10.0
- What is the top-level directory of the model you are using: research/transformer/spatial_transformer.py
- Have I written custom code: N/A
- OS Platform and Distribution: MacOS10.14.5
- TensorFlow installed from: source
- Bazel version: N/A
- CUDA/cuDNN version: N/A
- GPU model and memory: N/A
- Exact command to reproduce: N/A
Describe the problem
How to implement torch.nn.functional.grid_sample padding mode in Tensorflow?
Source code / logs
https://github.com/tensorflow/models/blob/master/research/transformer/spatial_transformer.py
Found a solution. Works for me:) @OrkhanHI @Eurus-Holmes @Feynman1999
def bilinear_sampler(img, coords):
H = tf.shape(img)[1]
W = tf.shape(img)[2]
max_y = tf.cast(H - 1, 'int32')
max_x = tf.cast(W - 1, 'int32')
# ----------------- Changes below -------------------------
# -> padding_mode = 'border'
# "#o" means original, "#t" means modified
# zero = tf.zeros([], dtype='int32') #o
zero = tf.zeros([1], dtype=tf.int32) #t
eps = tf.constant([0.5], 'float32') #t
# rescale x and y to [0, W-1/H-1]
x, y = coords[:, ..., 0], coords[:, ..., 1]
x = tf.cast(x, 'float32')
y = tf.cast(y, 'float32')
x = 0.5 * ((x + 1.0) * tf.cast(max_x - 1, 'float32'))
y = 0.5 * ((y + 1.0) * tf.cast(max_y - 1, 'float32'))
x = tf.clip_by_value(x, eps, tf.cast(max_x, tf.float32) - eps) #t
y = tf.clip_by_value(y, eps, tf.cast(max_y, tf.float32) - eps) #t
# -------------- Changes above --------------------
# grab 4 nearest corner points for each (x_i, y_i)
x0 = tf.cast(tf.floor(x), 'int32')
x1 = x0 + 1
y0 = tf.cast(tf.floor(y), 'int32')
y1 = y0 + 1
# clip to range [0, H-1/W-1] to not violate img boundaries
x0 = tf.clip_by_value(x0, zero, max_x)
x1 = tf.clip_by_value(x1, zero, max_x)
y0 = tf.clip_by_value(y0, zero, max_y)
y1 = tf.clip_by_value(y1, zero, max_y)
# get pixel value at corner coords
Ia = get_pixel_value(img, x0, y0)
Ib = get_pixel_value(img, x0, y1)
Ic = get_pixel_value(img, x1, y0)
Id = get_pixel_value(img, x1, y1)
# recast as float for delta calculation
x0 = tf.cast(x0, 'float32')
x1 = tf.cast(x1, 'float32')
y0 = tf.cast(y0, 'float32')
y1 = tf.cast(y1, 'float32')
# calculate deltas
wa = (x1 - x) * (y1 - y)
wb = (x1 - x) * (y - y0)
wc = (x - x0) * (y1 - y)
wd = (x - x0) * (y - y0)
# add dimension for addition
wa = tf.expand_dims(wa, axis=3)
wb = tf.expand_dims(wb, axis=3)
wc = tf.expand_dims(wc, axis=3)
wd = tf.expand_dims(wd, axis=3)
# compute output
out = tf.add_n([wa * Ia, wb * Ib, wc * Ic, wd * Id])
return out
Any update on this feature request (made in 2019)?
Please transform this ticket to https://github.com/tensorflow/tensorflow cc. @cantonios
Take a look at this, seems to be working reasonably well
https://github.com/AlexanderLutsenko/nobuco/blob/aa4745e6abb1124d90f7d3ace6d282f923f08a40/nobuco/node_converters/grid_sampling.py#L38
@dexter2406 would you mind sharing what your get_pixel_value looks like?
It's the only missing piece there
