delaunator-cpp icon indicating copy to clipboard operation
delaunator-cpp copied to clipboard

Unsigned integer overflow detected by AddressSanitizer

Open delfrrr opened this issue 7 years ago • 1 comments

when compiling with -fsanitize=address,undefined,integer

$ make test
/home/travis/build/delfrrr/delaunator-cpp/include/delaunator.hpp:412:34: runtime error: unsigned integer overflow: 18446744073709551615 + 2 cannot be represented in type 'unsigned long'

delfrrr avatar Sep 09 '18 07:09 delfrrr

I assume this is from the fact that I selected the std::numeric_limits<std::size_t>::max() to be our magic value for indexes that are not yet assigned. Therefore, at some point we are calling legalize() and it is selecting a half_edge that has this index value preset. There likely needs to be a check to prevent this from occurring here.

flippmoke avatar Sep 10 '18 13:09 flippmoke