predicting next frame

[zisismp4]

Hello and thanks for sharing your great work! Is there an easy way to predict the next frame using the flow? [ function(frame0, flow) ==> predicted_frame1 ]

[salvadog]

use cv2.remap

[zisismp4]

how should I use cv2.remap? predicted_frame_by_img1 = cv2.remap( img1, flo_2chan[:, :, 1], flo_2chan[:, :, 0], interpolation=? ) is this the correct way? And what is the appropriate interpolation?

[salvadog]

I simply warp img2 back to img1 using the code below, cracks may appear due to missing information

    h = flow.shape[0]
    w = flow.shape[1]
    flow[:,:,0] += np.arange(w)
    flow[:,:,1] += np.arange(h)[:,np.newaxis]
    warped_img2 = cv2.remap(img2, flow, None, cv2.INTER_LINEAR)

[rastaman7]

If I just use the code above, I get this error.

Traceback (most recent call last): File "j_warp_default.py", line 79, in demo(args) File "j_warp_default.py", line 62, in demo flow_up[:,:,0] += np.arange(w) File "/opt/conda/lib/python3.7/site-packages/torch/tensor.py", line 480, in array return self.numpy() TypeError: can't convert cuda:0 device type tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first.

I’ve been trying different things, but they are not working. Could anyone share how to warp images using the flow(flow_up)?