3dgs背景图有点不清晰

[fcdml]

[yifanlu0227]

This is possible. 3DGS has different reconstruction performance in different scenes, and some scene can be particularly difficult. You can adjust some hyperparameters to optimize the results, but note that ChatSim just use the vanilla 3DGS, without any advanced techniques tailored for autonomous driving data (e.g. modeling the sky separately).

It is recommended to check the latest literatures and borrow their methods to achieve a better reconstruction.

[fcdml]

我用原版的3dgs跑了一下 比原来好些 想问下chatsim单独做了相机参数的转换 将waymo的坐标转成colmap 跟这个有关系么

[yifanlu0227]

确实也不排除这个scene里colmap标出来的位姿比原来的效果要差的可能。可以尝试在config里位姿直接用cams_meta_waymo.npy，这是从waymo原始数据集里提取出来的位姿

data
`-- waymo_multi_view
    |-- ...
    `-- segment-1172406780360799916_1660_000_1680_000_with_camera_labels
        |-- 3d_boxes.npy                # 3d bounding boxes of the first frame
        |-- images                      # a clip of waymo images used in chatsim (typically 40 frames)
        |-- images_all                  # full waymo images (typically 198 frames)
        |-- map.pkl                     # map data of this scene
        |-- point_cloud                 # point cloud file of the first frame
        |-- cams_meta.npy               # Camera ext&int calibrated by metashape and transformed to waymo coordinate system.
        |-- cams_meta_metashape.npy     # Camera ext&int calibrated by metashape (intermediate file, relative scale, not required by simulation inference)
        |-- cams_meta_colmap.npy        # Camera ext&int calibrated by colmap (intermediate file, relative scale, not required by simulation inference)
        |-- cams_meta_waymo.npy         # Camera ext&int from original waymo dataset (intermediate file, not required by simulation inference)
        |-- shutters                    # normalized exposure time (mean=0 std=1)
        |-- tracking_info.pkl           # tracking data
        |-- vehi2veh0.npy               # transformation matrix from i-th frame's vehicle coordinate to the first frame's vehicle 
        |-- camera.xml                  # calibration file from Metashape (intermediate file, not required by simulation inference)
        `-- colmap/sparse_undistorted/[images/cams_meta.npy/points3D_waymo.ply]   # calibration files from COLMAP (intermediate file, only required when using 3dgs rendering)
        

[fcdml]

我好像知道问题在哪了 如果自己安装colmap 提取点云等信息 跑出来的效果就是第一张图 如果用你矫正后的colmap数据就没问题 生成的类似用于之前的第二张图 效果贴近3dgs生成的 可能自己安装colmap还不够 还需要调colmap 想问下 你大概是怎么调的

[Mr-ind1fferent]

我好像知道问题在哪了 如果自己安装colmap 提取点云等信息 跑出来的效果就是第一张图 如果用你矫正后的colmap数据就没问题 生成的类似用于之前的第二张图 效果贴近3dgs生成的 可能自己安装colmap还不够 还需要调colmap 想问下 你大概是怎么调的

请问一下，如何使用waymo得到colmap格式的数据集呢