Registring point clouds from multiple rendered images

[yash-goel]

Hi, thanks for this library! This is not an issue rather some understanding.

I am trying to register multiple point cloud together each of which are generated from the color and depth image rendered for a particular mesh through the following steps:

1. With a mesh set with I pose, I set an intrinsic camera with pose R|t represented by the homogenous matrix T.

2. Then I render the RGB and depth image (for which I believe the view matrix or extrinsics will be inv(T)).

3. I get the point cloud for each instance in camera frame by de-projecting the 2D points using the intrinsics and depth. Then I finally register the point cloud by multiplying it by T.

When I put together all the point clouds, I don't get the desired result. Am I missing something somewhere?

Thanks for the help.