Open3D
Open3D copied to clipboard
Published
20 hours ago •
isl-org
isl-org
Regression in RGBDOdometry
trafficstars
Checklist
- [X] I have searched for similar issues.
- [X] For Python issues, I have tested with the latest development wheel.
- [X] I have checked the release documentation and the latest documentation (for
masterbranch).
Describe the issue
RGBD images that previously had no trouble tracking odometry in O3D started to not work on GPU when I recently tested the tip of the master branch. I bisected it down to this commit: 47e320ea5b9ef2a8e529894d9833193efde572ea. Looks like a pretty large change so I haven't quite narrowed it down more than that.
The tip of master also works if I revert 47e320ea5b9ef2a8e529894d9833193efde572ea and c355b0dcbc8a96c9c979dc6a167b4aca6b06d62a to be able to compile without errors.
The CPU version of code seems to work. The GPU version exceptions out due to a singular matrix.
Steps to reproduce the bug
I've added a unit test here: https://github.com/jdavidberger/Open3D/commit/f3d38cdd941402c34fee55cbc37a1b6e810cfc1b
Assets for that test are here: https://drive.google.com/file/d/1fMzl2BG40x8hKhqYi05XpWu3oKPoYuSv/view?usp=sharing
Apologies for the hard coded links; wasn't sure the most expedient way to include large assets in open3d for testing. If there is interest in adding these in for regression testing, I can fix that up with some direction on where to dump the NPY files.
The error seems somewhat transient. 95% of the time I get the exception; but ~5% of the time it seems to work with likely valid transforms.
Error message
C++ exception with description "[Open3D Error] (void open3d::t::pipelines::kernel::DecodeAndSolve6x6(const open3d::core::Tensor&, open3d::core::Tensor&, float&, int&)) /home/justin/source/oss/Open3D/cpp/open3d/t/pipelines/kernel/TransformationConverter.cpp:145: Singular 6x6 linear system detected, tracking failed.
Backtrace:
(gdb) bt
#0 open3d::t::pipelines::kernel::DecodeAndSolve6x6 (A_reduction=..., delta=..., inlier_residual=@0x7fffffffc0d8: 2.60017881e+29, inlier_count=@0x7fffffffc0dc: 21845) at /home/justin/source/oss/Open3D/cpp/open3d/t/pipelines/kernel/TransformationConverter.cpp:145
#1 0x0000555557b9f86c in open3d::t::pipelines::kernel::odometry::ComputeOdometryResultHybridCUDA (source_depth=..., target_depth=..., source_intensity=..., target_intensity=..., target_depth_dx=..., target_depth_dy=..., target_intensity_dx=..., target_intensity_dy=..., source_vertex_map=..., intrinsics=..., init_source_to_target=..., delta=..., inlier_residual=@0x7fffffffc0d8: 2.60017881e+29, inlier_count=@0x7fffffffc0dc: 21845, depth_outlier_trunc=0.100000001, depth_huber_delta=0.0500000007, intensity_huber_delta=0.100000001) at /home/justin/source/oss/Open3D/cpp/open3d/t/pipelines/kernel/RGBDOdometryCUDA.cu:366
#2 0x0000555557b7e5aa in open3d::t::pipelines::kernel::odometry::ComputeOdometryResultHybrid (source_depth=..., target_depth=..., source_intensity=..., target_intensity=..., target_depth_dx=..., target_depth_dy=..., target_intensity_dx=..., target_intensity_dy=..., source_vertex_map=..., intrinsics=..., init_source_to_target=..., delta=..., inlier_residual=@0x7fffffffc0d8: 2.60017881e+29, inlier_count=@0x7fffffffc0dc: 21845, depth_outlier_trunc=0.100000001, depth_huber_delta=0.0500000007, intensity_huber_delta=0.100000001) at /home/justin/source/oss/Open3D/cpp/open3d/t/pipelines/kernel/RGBDOdometry.cpp:200
#3 0x0000555557b2b117 in open3d::t::pipelines::odometry::ComputeOdometryResultHybrid (source_depth=..., target_depth=..., source_intensity=..., target_intensity=..., target_depth_dx=..., target_depth_dy=..., target_intensity_dx=..., target_intensity_dy=..., source_vertex_map=..., intrinsics=..., init_source_to_target=..., depth_outlier_trunc=0.100000001, depth_huber_delta=0.0500000007, intensity_huber_delta=0.100000001) at /home/justin/source/oss/Open3D/cpp/open3d/t/pipelines/odometry/RGBDOdometry.cpp:490
#4 0x0000555557b2a2ee in open3d::t::pipelines::odometry::RGBDOdometryMultiScaleHybrid (source=..., target=..., intrinsics=..., trans=..., depth_scale=1000, depth_max=3, criteria=std::vector of length 3, capacity 3 = {...}, params=...) at /home/justin/source/oss/Open3D/cpp/open3d/t/pipelines/odometry/RGBDOdometry.cpp:387
#5 0x0000555557b27268 in open3d::t::pipelines::odometry::RGBDOdometryMultiScale (source=..., target=..., intrinsics=..., init_source_to_target=..., depth_scale=1000, depth_max=3, criteria=std::vector of length 3, capacity 3 = {...}, method=open3d::t::pipelines::odometry::Method::Hybrid, params=...) at /home/justin/source/oss/Open3D/cpp/open3d/t/pipelines/odometry/RGBDOdometry.cpp:120
#6 0x00005555562036d9 in open3d::tests::OdometryPermuteDevices_TestFailureCase_Test::TestBody (this=0x555570a92460) at /home/justin/source/oss/Open3D/cpp/tests/t/pipelines/odometry/RGBDOdometry.cpp:360
#7 0x0000555557e797a6 in testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void> (object=0x555570a92460, method=&virtual testing::Test::TestBody(), location=0x55555da1a3bb "the test body") at /home/justin/source/oss/Open3D/cmake-build-debug/googletest/src/ext_googletest/googletest/src/gtest.cc:2607
#8 0x0000555557e72d2d in testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void> (object=0x555570a92460, method=&virtual testing::Test::TestBody(), location=0x55555da1a3bb "the test body") at /home/justin/source/oss/Open3D/cmake-build-debug/googletest/src/ext_googletest/googletest/src/gtest.cc:2643
#9 0x0000555557e4b12c in testing::Test::Run (this=0x555570a92460) at /home/justin/source/oss/Open3D/cmake-build-debug/googletest/src/ext_googletest/googletest/src/gtest.cc:2682
#10 0x0000555557e4bb05 in testing::TestInfo::Run (this=0x5555705189b0) at /home/justin/source/oss/Open3D/cmake-build-debug/googletest/src/ext_googletest/googletest/src/gtest.cc:2861
#11 0x0000555557e4c3bb in testing::TestSuite::Run (this=0x555570517050) at /home/justin/source/oss/Open3D/cmake-build-debug/googletest/src/ext_googletest/googletest/src/gtest.cc:3015
#12 0x0000555557e5ad23 in testing::internal::UnitTestImpl::RunAllTests (this=0x55557026bd50) at /home/justin/source/oss/Open3D/cmake-build-debug/googletest/src/ext_googletest/googletest/src/gtest.cc:5855
#13 0x0000555557e7ab36 in testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool> (object=0x55557026bd50, method=(bool (testing::internal::UnitTestImpl::*)(testing::internal::UnitTestImpl * const)) 0x555557e5a970 <testing::internal::UnitTestImpl::RunAllTests()>, location=0x55555da1aea8 "auxiliary test code (environments or event listeners)") at /home/justin/source/oss/Open3D/cmake-build-debug/googletest/src/ext_googletest/googletest/src/gtest.cc:2607
#14 0x0000555557e73d89 in testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool> (object=0x55557026bd50, method=(bool (testing::internal::UnitTestImpl::*)(testing::internal::UnitTestImpl * const)) 0x555557e5a970 <testing::internal::UnitTestImpl::RunAllTests()>, location=0x55555da1aea8 "auxiliary test code (environments or event listeners)") at /home/justin/source/oss/Open3D/cmake-build-debug/googletest/src/ext_googletest/googletest/src/gtest.cc:2643
#15 0x0000555557e59549 in testing::UnitTest::Run (this=0x5555701e5ac0 <testing::UnitTest::GetInstance()::instance>) at /home/justin/source/oss/Open3D/cmake-build-debug/googletest/src/ext_googletest/googletest/src/gtest.cc:5438
#16 0x000055555628e7b1 in RUN_ALL_TESTS () at /home/justin/source/oss/Open3D/cmake-build-debug/googletest/src/ext_googletest/googletest/include/gtest/gtest.h:2490
#17 0x000055555628e1b8 in main (argc=1, argv=0x7fffffffe5e8) at /home/justin/source/oss/Open3D/cpp/tests/Main.cpp:70
#18 0x00007ffff74e9083 in __libc_start_main (main=0x55555628e11b <main(int, char**)>, argc=3, argv=0x7fffffffe5e8, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffe5d8) at ../csu/libc-start.c:308
#19 0x0000555555a851de in _start ()
Expected behavior
Expected behavior is to not throw an exception and to exit with a transform more or less like the one in the test case as ground truth.
Open3D, Python and System information
- Operating system: Ubuntu 20.04
- Python version: Python 3.8
- System architecture: x86
- Is this a remote workstation?: no
- How did you install Open3D?: build from source
- Compiler version (if built from source): gcc 9.4
Additional information
No response
Thanks for reporting the bug! Could be some initialization issues in the CUB reduction that I wasn't aware of. I'm working on it.
