Camera pose change

[DCS4]

Hi, using the camera external parameter under the opencv coordinate system

camera internal parameters: And by way of reversing the y-axis and z-axis transformations of the camera external parameters and generating the projection matrix according to the formula, I finally found that only the black background could be rendered. Maybe there is something wrong with the transformation of camera internal and external parameters, how can I convert camera internal and external parameters to openGL's modelview matrix and projection matrix?

Here is the python code to transform the camera pose

def camera_extrinsic_to_opengl_mv(W2C, device):

    flip_yz = np.eye(4)
    flip_yz[1, 1] = -1
    flip_yz[2, 2] = -1

    #W2C:  World coordinate system to camera coordinate system
    W2C = np.matmul(W2C, flip_yz)
    return torch.from_numpy(W2C).to(device).float()


def camera_intrinsic_to_opengl_projection(intrinsic, width, height, device, 
                                            near=1.0, far=100.0,flip_y=False):
    fx = intrinsic[0, 0]
    fy = intrinsic[1, 1]
    cx = intrinsic[0, 2]
    cy = intrinsic[1, 2]

     # Compute left, right, bottom, top planes
    left = -cx * near / fx
    right = (width - cx) * near / fx
    bottom = -cy * near / fy
    top = (height - cy) * near / fy

    # Compute perspective projection matrix
    mat = torch.zeros((4, 4), device=device)
    mat[0, 0] = 2 * near / (right - left)
    mat[0, 2] = (right + left) / (right - left)
    mat[1, 1] = 2 * near / (top - bottom) * (-1 if flip_y else 1)
    mat[1, 2] = (top + bottom) / (top - bottom)
    mat[2, 2] = -(far + near) / (far - near)
    mat[2, 3] = -2 * far * near / (far - near)
    mat[3, 2] = -1

    return mat.float()

[wpalfi]

Projection matrices are tricky:-) This works for me:

def camera_intrinsic_to_opengl_projection(intrinsic,w,h,n,f,flip_y):
    fx = intrinsic[0,0]
    fy = intrinsic[1,1]
    cx = intrinsic[0,2]
    cy = intrinsic[1,2]
    
    proj = np.array([
        [2.*fx/w,   0,    1-2*cx/w,           0],
        [0,    2*fy/h,   -1+2*cy/h,           0],
        [0,         0,(-f-n)/(f-n),-2*f*n/(f-n)],
        [0,         0,          -1,           0]
        ])
        
    if flip_y:
        proj[1,:] *= -1

    return proj