About multi-camera alignment

[szhang963]

trafficstars

Hi, I am very interested in your promising work. I have some questions about multi-camera alignment.

1. Why did you use Metashape to refine the camera pose? Is the result better than COLMAP for multi-camera pose estimating?
2. Did you only refine multi-camera extrinsic (cami2ego)? Then, Did you use the pose of ego car from the vehicle sensor to translate the camera coord to the world？

Thanks in advance.

[yifanlu0227]

Hi!

1. We follow READ: Large-Scale Neural Scene Rendering for Autonomous Driving to use MetaShape for camera poses. We didn't use COLMAP to estimate the position before, so we are not sure which of them is more effective. However, considering that Metashape is a commercial software, it may provide a more accurate pose. In any case, using the original pose from the waymo dataset will lead to poor rendering results, and re-calibrating is important for NeRF trainining.

2. Yes, as written in the paper, the coordinate system was eventually transformed back to the world coordinate system.

[yifanlu0227]

There are many existing ways to use COLMAP for estimating camera poses. We also consider introducing relevant scripts to provide another open source and free choice.

[szhang963]

Thanks for your active reply.

[szhang963]

Hi, I have a question about metashape. Have you fixed each camera's intrinsic to estimate camera poses?

I followed the script to reconstruct Waymo multi-camera scenes, but the intrinsic of the camera sensor was modified and only one sensor existed. Could you provide some workflow or settings for multi-camera reconstruction, such as setting multi-camera intrinsic from the Waymo dataset as default parameters for metashape?

Thank you for your reply.

[yifanlu0227]

We use 3 front-view cameras (FRONT, FRONT-LEFT, FRONT-RIGHT) in our ChatSim.

In Metashape, we assume they share the same intrinsics. We adopted the estimated camera intrinsics instead of using the original waymo intrinsics.

The code will be released in the next day or two, and you can follow our calibration and conversion process of multi-camera alignment.

[szhang963]

Thanks. I am eagerly looking forward to your code release.