Open3D
Open3D copied to clipboard
Floating-point equality check in RANSAC result comparison may never trigger
Checklist
- [x] I have searched for similar issues.
- [x] For Python issues, I have tested with the latest development wheel.
- [x] I have checked the release documentation and the latest documentation (for
mainbranch).
Describe the issue
In PointClooud::SegmentPlane, the RANSAC result comparison includes a direct equality check on fitness_, which is a double:
if (this_result.fitness_ > result.fitness_ ||
(this_result.fitness_ == result.fitness_ &&
this_result.inlier_rmse_ < result.inlier_rmse_)) {
Since fitness_ is defined as a double, exact equality is highly unlikely due to precision limitations. This makes the tie-breaking condition based on inlier_rmse_ effectively unreachable in most cases.
Suggestion:
const double epsilon = 1e-8;
if (this_result.fitness_ > result.fitness_ ||
(std::abs(this_result.fitness_ - result.fitness_) < epsilon &&
this_result.inlier_rmse_ < result.inlier_rmse_)) {
This would allow more robust tie-breaking when fitness_ values are nearly equal.