IDS icon indicating copy to clipboard operation
IDS copied to clipboard

Tolerance test issues in 1.0

Open andyward opened this issue 1 year ago • 3 comments

A couple of issues discovered in the tolerance test cases when implementing the latest 1.0 logic:

1. Test values outside the precision of a double fail_comparison_tolerance_for_floating_point_negative_high_number_lower_bound.ids is testing that.

-1000001.0000010001 == -1000000 which is expected to Fail but actually Passes. This test is the failing counterpart to -1000001.000001 == -1000000 which does pass as expected

It actually passes since the least significant digit of -1000001.0000010001 is beyond the precision of a IEEE74 double so is actually equivalent with the passing value.

I believe the values chosen come from this PR https://github.com/buildingSMART/IDS/pull/304#issuecomment-2129482354

Recommendation: adjust the IfcReal test value to -1000001.0000011

2. Exclusive range test cases on the exact tolerance boundary. E.g. pass_comparison_tolerance_for_floating_point_range_greater_than_zero_exclusive.ifc is expecting 0.000001 > 0 to pass, but it fails since the tolerance for 0 is exactly 1e-6,

pass_comparison_tolerance_for_floating_point_range_lower_than_zero_exclusive.ifc has the same issue

Recommendation: adjust the test values to 0.0000011 and -0.0000011 respectively

Note: both the above files have extraneous characters that may cause issues for some folks image

andyward avatar Jun 06 '24 16:06 andyward

It actually passes since the least significant digit of -1000001.0000010001 is beyond the precision of a IEEE74 double so is actually equivalent with the passing value.

Good point. The values in #304 tests come from: https://github.com/buildingSMART/IDS/blob/0d50fd8f2dbd5b388f6fafb67da255cc3ce2b4ca/Documentation/tolerance.md I overrode the value with those proposed by @andyward in the test. Should we also amend or explain this in the tolerance table? @CBenghi @giuseppeverduciALMA

Doesn't it also apply to the other rows? (I didn't create dedicated tests for those values)

v Lower bound
-10000.0 -10000.0100010000
-100000.0 -100000.1000010000
  1. Exclusive range test cases on the exact tolerance boundary.

Again, good point; the fix is on its way.

Note: both the above files have extraneous characters that may cause issues for some folks

Not sure how it got there, but I'm removing them. Thanks.

atomczak avatar Jun 10 '24 08:06 atomczak

Doesn't it also apply to the other rows? (I didn't create dedicated tests for those values)

I'm not sure we need to test all the values - just some boundaries seems enough, which you've done.

In #304 the other tests are OK since the integer component is smaller than abs(1000000).

value == 100000
	fail 99999.8999989
	pass 99999.899999
	pass 100000.100001	OK
	fail 100000.1000011	OK
...
value == 0.0000001
	fail -0.00000090000011	OK
	pass -0.0000009000001	OK
	pass 0.0000011000001	OK
	fail 0.00000110000011	OK
...
value == -1000000.0
	fail -1000001.00000100001	**NOT OK due to truncations**
	pass -1000001.0000010000
	pass -999998.999999
	fail -999998.9999989

andyward avatar Jun 10 '24 08:06 andyward

yes, however, I was talking about the table in docs, where we should probably mention this double precision aspect. Because the tolerance.md didn't exist on the development branch, I edited the one from @giuseppeverduciALMA and pushed it. cc: @CBenghi

atomczak avatar Jun 10 '24 10:06 atomczak

I think this is ready for closing, @andyward?

atomczak avatar Aug 21 '24 19:08 atomczak

Yes, closing as fixed/superceded now

andyward avatar Aug 21 '24 19:08 andyward