(Understanding) How to transform between coordinate frames to image plane?

[corcasta]

Hi, there's one little step that it's bugging me (i'm trying to understand) in the provided examples when transforming from different coord frames to image plane.

According to the given example:

Y = P_rect_xx * R_rect_00 * (R|T)_velo_to_cam * (R|T)_imu_to_velo * X

The matrices are:

• P_rect_xx (3x4): rectfied cam 0 coordinates -> image plane
• R_rect_00 (4x4): cam 0 coordinates -> rectified cam 0 coord.
• (R|T)_velo_to_cam (4x4): velodyne coordinates -> cam 0 coordinates
• (R|T)_imu_to_velo (4x4): imu coordinates -> velodyne coordinates

Why do i have to explicitly transform from "cam 0 coordinates" to "rectified cam 0 coord"?, isn't supposed to be that transformation implicitly done within the P_rect_xx (projection matrix)? I thought P_rect was created considering the rectified rotation and translation like in the image below.

Wouldn't be enough to provide just "cam 0 coordinates" to P_rect?

Thanks

[corcasta]

I manage to understand that the definition used as P_rect is the following:

thus requiring the transformation from cam 0 coords to rectified cam 0 coord.