i2cdevlib icon indicating copy to clipboard operation
i2cdevlib copied to clipboard

getMagnitude() returning nan for VectorInt16

Open yolomilk opened this issue 5 years ago • 1 comments

the magnitude in helper_3dmath is currently calculated as following:

sqrt(x*x + y*y + z*z); this works well for Quaternion and VectorFloat but is causing issues for VectorInt16 I think it should be changed to

sqrt((float)x*(float)x + (float) y*(float)y + (float)z*(float)z);

or have a warning that the numbers have to be smaller than 256 so that it works. This is also effecting the funktions normalize and getNormalized, because getMagnitude is used there.

yolomilk avatar Oct 15 '19 18:10 yolomilk

You are right. I tried your version and it works just fine.

MR-Nejati avatar Jul 10 '21 07:07 MR-Nejati