perspective transform image using colmap camera poses

[pknmax]

Hello @ahojnnes Is it possible to orient/transform image (in image coordinate system) according to reference image whose colmap poses: R, t, projection matrix, FoVx, FoVy, image_width, image_height is known?

I want to see how this image will look like from that colmap camera pose?

[tsattler]

I don't understand the question. What do you want to achieve?

[pknmax]

@tsattler Suppose I have colmap camera poses, is it possible and how to obtain a new view of input image I (planar object) from a different viewpoint/camera pose using those poses?

Colmap camera poses has following data:

        extr = cam_extrinsics[key]
        intr = cam_intrinsics[extr.camera_id]
        height = intr.height
        width = intr.width

        uid = intr.id
        R = np.array(qvec2rotmat(extr.qvec))
        T = np.array(extr.tvec)

        if intr.model=="SIMPLE_PINHOLE":
            focal_length_x = intr.params[0]
            FovY = focal2fov(focal_length_x, height)
            FovX = focal2fov(focal_length_x, width)
            fx = fy = intr.params[0]
            cx = intr.params[1]
            cy = intr.params[2]

        elif intr.model=="PINHOLE":
            focal_length_x = intr.params[0]
            focal_length_y = intr.params[1]
            FovY = focal2fov(focal_length_y, height)
            FovX = focal2fov(focal_length_x, width)
            fx = intr.params[0]
            fy = intr.params[1]
            cx = intr.params[2]
            cy = intr.params[3]

class DummyCamera:

    def __init__(self, uid, R, T, FoVx, FoVy, K, image_width, image_height):
        self.uid = uid
        self.R = R
        self.T = T
        self.FoVx = FoVx
        self.FoVy = FoVy
        self.K = K
        self.image_width = image_width
        self.image_height = image_height
        self.projection_matrix = getProjectionMatrix(znear=0.01, zfar=100.0, fovX=FoVx, fovY=FoVy).transpose(0,1).cuda()
        self.world_view_transform = torch.tensor(getWorld2View2(R, T, np.array([0,0,0]), 1.0)).transpose(0, 1).cuda()
        self.full_proj_transform = (self.world_view_transform.unsqueeze(0).bmm(self.projection_matrix.unsqueeze(0))).squeeze(0)
        self.camera_center = self.world_view_transform.inverse()[3, :3]

Colmap camera poses are computed on different flat object, size of images used in this computation is different from size of image I

[pknmax]

@tsattler any suggestions please ?

[tsattler]

You can compute a homography from the camera parameters and the known position of the plane.

[pknmax]

@tsattler thanks. do you know any example code/resource for this?

[tsattler]

Colmap should have code for homography estimation and decomposition and you can read up on the concept in the literature.

[pknmax]

I am still struggling to do this, any help will be appreciated... https://stackoverflow.com/q/77811600