The default float epsilon is too small

[dumganhar]

creator:

https://github.com/cocos/cocos-engine/blob/v3.6.0/native/cocos/math/Utils.h#L11

// in cocos/math/Utils.h, we define the default value
constexpr auto EPSILON = 0.000001; // 5 zeros after point

cpp code was ported from ts: https://github.com/cocos/cocos-engine/blob/v3.6.0/cocos/core/math/utils.ts#L33

But js uses double to represent float number, but cpp uses float instead. The precision is lower than which in ts.

godot:

https://github.com/godotengine/godot/blob/fe52458154c64fb1b741df4f7bd10106395f7cbd/core/math/math_defs.h#L34

#define CMP_EPSILON 0.00001 // // 4 zeros after point

I think godot's implementation is better.

[dumganhar]

Currently, the unit test here https://github.com/cocos/cocos-engine/pull/12169 will fail. If change it to 4 zeros after point, it will succeed.

[minggo]

May be ts can also change to 0.00001 too. Babylon.js just use 0.001.

[dumganhar]

Math library is using 0.0001 for precision now: https://github.com/cocos/cocos-engine/pull/12610 Should not change the default precision in math::isEquals function since there're many places use it, for example in GFX module. Change the milestone to 3.7

[dumganhar]

@hana-alice suggested not to modify the default precision of math::isEquals since GFX code depends on the it. If we change it, we need to do lots of tests.

Since all math classes have a approxEquals method with default precision of #define CC_FLOAT_CMP_PRECISION (0.00001F), I think this issue could be closed.