4DGaussians
4DGaussians copied to clipboard
Published
20 hours ago •
hustvl
hustvl
Problem with custom stereo dataset
Hi,
I am trying to train a 4D Gaussian model using my own set of stereo video pairs using dynerf type dataset directory structure. I have two video files (.mp4) from my stereo camera setup and have generated a poses_bounds.npy file with LLFF format because I was unable to generate camera poses for a stereo camera setup using COLMAP. Now I run into a problem when trying to generate the point clouds from my input data. It seems like it is unable to do triangulation as the image sees 0 points:
['r_000', 'png']
['r_001', 'png']
[[[ 1.77708380e-02 -9.99836743e-01 -3.26325977e-03 0.00000000e+00]
[ 9.83394206e-01 1.80679709e-02 -1.80580720e-01 0.00000000e+00]
[ 1.80610195e-01 -2.32830644e-10 9.83554780e-01 0.00000000e+00]]
[[ 1.77708380e-02 -9.99836743e-01 -3.26325977e-03 1.19952627e-01]
[ 9.83394206e-01 1.80679709e-02 -1.80580720e-01 0.00000000e+00]
[ 1.80610195e-01 -2.32830644e-10 9.83554780e-01 0.00000000e+00]]]
I20240610 15:52:57.562021 139730277408768 misc.cc:198]
==============================================================================
Feature extraction
==============================================================================
I20240610 15:53:00.702751 139728917594112 feature_extraction.cc:254] Processed file [1/2]
I20240610 15:53:00.702776 139728917594112 feature_extraction.cc:257] Name: r_000.png
I20240610 15:53:00.702779 139728917594112 feature_extraction.cc:283] Dimensions: 1352 x 1014
I20240610 15:53:00.702782 139728917594112 feature_extraction.cc:286] Camera: #1 - SIMPLE_RADIAL
I20240610 15:53:00.702784 139728917594112 feature_extraction.cc:289] Focal Length: 1622.40px
I20240610 15:53:00.702790 139728917594112 feature_extraction.cc:302] Features: 7461
I20240610 15:53:00.853336 139728917594112 feature_extraction.cc:254] Processed file [2/2]
I20240610 15:53:00.853363 139728917594112 feature_extraction.cc:257] Name: r_001.png
I20240610 15:53:00.853368 139728917594112 feature_extraction.cc:283] Dimensions: 1352 x 1014
I20240610 15:53:00.853372 139728917594112 feature_extraction.cc:286] Camera: #2 - SIMPLE_RADIAL
I20240610 15:53:00.853376 139728917594112 feature_extraction.cc:289] Focal Length: 1622.40px
I20240610 15:53:00.853386 139728917594112 feature_extraction.cc:302] Features: 7820
I20240610 15:53:00.857477 139730277408768 timer.cc:91] Elapsed time: 0.055 [minutes]
/home/zomhussa/4dgs/4DGaussians/database.py:11: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
return array.tostring()
/home/zomhussa/4dgs/4DGaussians/database.py:17: DeprecationWarning: The binary mode of fromstring is deprecated, as it behaves surprisingly on unicode inputs. Use frombuffer instead
return np.fromstring(blob, dtype=dtype).reshape(*shape)
I20240610 15:53:01.037186 139837249699840 misc.cc:198]
==============================================================================
Exhaustive feature matching
==============================================================================
I20240610 15:53:01.085564 139837249699840 feature_matching.cc:231] Matching block [1/1, 1/1]
I20240610 15:53:01.134742 139837249699840 feature_matching.cc:46] in 0.049s
I20240610 15:53:01.134897 139837249699840 timer.cc:91] Elapsed time: 0.002 [minutes]
I20240610 15:53:01.226635 140239616794624 misc.cc:198]
==============================================================================
Loading model
==============================================================================
I20240610 15:53:01.226825 140239616794624 misc.cc:198]
==============================================================================
Loading database
==============================================================================
I20240610 15:53:01.230371 140239616794624 database_cache.cc:54] Loading cameras...
I20240610 15:53:01.230443 140239616794624 database_cache.cc:64] 2 in 0.000s
I20240610 15:53:01.230483 140239616794624 database_cache.cc:72] Loading matches...
I20240610 15:53:01.230548 140239616794624 database_cache.cc:78] 1 in 0.000s
I20240610 15:53:01.230588 140239616794624 database_cache.cc:94] Loading images...
I20240610 15:53:01.231338 140239616794624 database_cache.cc:143] 2 in 0.001s (connected 2)
I20240610 15:53:01.231460 140239616794624 database_cache.cc:154] Building correspondence graph...
I20240610 15:53:01.231700 140239616794624 database_cache.cc:190] in 0.000s (ignored 0)
I20240610 15:53:01.231756 140239616794624 timer.cc:91] Elapsed time: 0.000 [minutes]
I20240610 15:53:01.233173 140239616794624 misc.cc:198]
==============================================================================
Triangulating image #1 (0)
==============================================================================
I20240610 15:53:01.233289 140239616794624 sfm.cc:473] => Image sees 0 / 974 points
I20240610 15:53:01.233537 140239616794624 sfm.cc:478] => Triangulated 0 points
I20240610 15:53:01.233571 140239616794624 misc.cc:198]
==============================================================================
Triangulating image #2 (1)
==============================================================================
I20240610 15:53:01.233663 140239616794624 sfm.cc:473] => Image sees 0 / 974 points
I20240610 15:53:01.233927 140239616794624 sfm.cc:478] => Triangulated 0 points
I20240610 15:53:01.233962 140239616794624 misc.cc:198]
==============================================================================
Retriangulation
==============================================================================
I20240610 15:53:01.234068 140239616794624 incremental_mapper.cc:175] => Completed observations: 0
I20240610 15:53:01.234098 140239616794624 incremental_mapper.cc:178] => Merged observations: 0
I20240610 15:53:01.234150 140239616794624 misc.cc:198]
==============================================================================
Bundle adjustment
==============================================================================
F20240610 15:53:01.234272 140239616794624 sfm.cc:514] Check failed: bundle_adjuster.Solve(reconstruction.get())
@ 0x7f8c1c18578a google::LogMessage::Fail()
@ 0x7f8c1c186f45 google::LogMessageFatal::~LogMessageFatal()
@ 0x556f5e693291 colmap::RunPointTriangulatorImpl(std::shared_ptr<colmap::Reconstruction> const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&,�?
@ 0x556f5e693786 colmap::RunPointTriangulator(int, char**)
@ 0x556f5e673685 main
@ 0x7f8c1a7f6083 __libc_start_main
@ 0x556f5e679fa4 (unknown)
colmap.sh: line 20: 143810 Aborted (core dumped) colmap point_triangulator --database_path $workdir/colmap/database.db --image_path $workdir/colmap/images --input_path $workdir/colmap/sparse_custom --output_path $workdir/colmap/sparse/0 --clear_points 1
I20240610 15:53:01.355672 140201049677824 misc.cc:198]
==============================================================================
Reading reconstruction
==============================================================================
F20240610 15:53:01.355728 140201049677824 reconstruction.cc:683] cameras, images, points3D files do not exist at /home/zomhussa/4dgs/SpacetimeGaussians/data/ringstable/colmap/sparse/0
@ 0x7f832150d78a google::LogMessage::Fail()
@ 0x7f832150ef45 google::LogMessageFatal::~LogMessageFatal()
@ 0x55ff55b88d7c colmap::Reconstruction::Read(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)
@ 0x55ff55a8462f colmap::RunImageUndistorter(int, char**)
@ 0x55ff55a44685 main
@ 0x7f831fb7e083 __libc_start_main
@ 0x55ff55a4afa4 (unknown)
colmap.sh: line 23: 143836 Aborted (core dumped) colmap image_undistorter --image_path $workdir/colmap/images --input_path $workdir/colmap/sparse/0 --output_path $workdir/colmap/dense/workspace
I20240610 15:53:01.478018 140714622971904 patch_match.cc:231] Reading workspace...
F20240610 15:53:01.478185 140714622971904 reconstruction.cc:683] cameras, images, points3D files do not exist at /home/zomhussa/4dgs/SpacetimeGaussians/data/ringstable/colmap/dense/workspace/sparse
@ 0x7ffb5830378a google::LogMessage::Fail()
@ 0x7ffb58304f45 google::LogMessageFatal::~LogMessageFatal()
@ 0x561a79090d7c colmap::Reconstruction::Read(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)
@ 0x561a793c1461 colmap::mvs::Model::ReadFromCOLMAP(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_tr�?
@ 0x561a793c3af3 colmap::mvs::Model::Read(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)
@ 0x561a793c78b2 colmap::mvs::Workspace::Workspace(colmap::mvs::Workspace::Options const&)
@ 0x561a793ccaec colmap::mvs::CachedWorkspace::CachedWorkspace(colmap::mvs::Workspace::Options const&)
@ 0x561a790d6e74 colmap::mvs::PatchMatchController::ReadWorkspace()
@ 0x561a790e11ed colmap::mvs::PatchMatchController::Run()
@ 0x561a790cd614 colmap::Thread::RunFunc()
@ 0x7ffb56d7fe95 execute_native_thread_routine
@ 0x7ffb582a9609 start_thread
@ 0x7ffb56a6f353 clone
colmap.sh: line 24: 143861 Aborted (core dumped) colmap patch_match_stereo --workspace_path $workdir/colmap/dense/workspace
I20240610 15:53:01.602423 139881016057856 misc.cc:205]
StereoFusion::Options
---------------------
I20240610 15:53:01.602461 139881016057856 fusion.cc:87] mask_path:
I20240610 15:53:01.602471 139881016057856 fusion.cc:88] max_image_size: -1
I20240610 15:53:01.602490 139881016057856 fusion.cc:89] min_num_pixels: 5
I20240610 15:53:01.602496 139881016057856 fusion.cc:90] max_num_pixels: 10000
I20240610 15:53:01.602502 139881016057856 fusion.cc:91] max_traversal_depth: 100
I20240610 15:53:01.602508 139881016057856 fusion.cc:92] max_reproj_error: 2
I20240610 15:53:01.602520 139881016057856 fusion.cc:93] max_depth_error: 0.01
I20240610 15:53:01.602526 139881016057856 fusion.cc:94] max_normal_error: 10
I20240610 15:53:01.602533 139881016057856 fusion.cc:95] check_num_images: 50
I20240610 15:53:01.602541 139881016057856 fusion.cc:96] use_cache: 0
I20240610 15:53:01.602547 139881016057856 fusion.cc:97] cache_size: 32
I20240610 15:53:01.602553 139881016057856 fusion.cc:100] bbox_min: -3.40282e+38 -3.40282e+38 -3.40282e+38
I20240610 15:53:01.602564 139881016057856 fusion.cc:101] bbox_max: 3.40282e+38 3.40282e+38 3.40282e+38
I20240610 15:53:01.602575 139881016057856 fusion.cc:148] Reading workspace...
F20240610 15:53:01.602611 139881016057856 misc.cc:282] Check failed: file.is_open() /home/zomhussa/4dgs/SpacetimeGaussians/data/ringstable/colmap/dense/workspace/stereo/fusion.cfg
@ 0x7f395158278a google::LogMessage::Fail()
@ 0x7f3951583f45 google::LogMessageFatal::~LogMessageFatal()
@ 0x55f748fda7ed colmap::ReadTextFileLines(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)
@ 0x55f74928fa28 colmap::mvs::StereoFusion::Run()
@ 0x55f748fe3614 colmap::Thread::RunFunc()
@ 0x7f394fffee95 execute_native_thread_routine
@ 0x7f3951528609 start_thread
@ 0x7f394fcee353 clone
colmap.sh: line 25: 143887 Aborted (core dumped) colmap stereo_fusion --workspace_path $workdir/colmap/dense/workspace --output_path $workdir/colmap/dense/workspace/fused.ply
Did someone experience this issue when trying to work with stereo data ? If someone has any idea about what might be going wrong then please help me with it. Thanks
