enoki icon indicating copy to clipboard operation
enoki copied to clipboard

UB issues with bool_array_t initialization.

Open mp3guy opened this issue 5 years ago • 1 comments

I get an UndefinedBehaviourSanitizer hit from Google's sanitizer (https://github.com/google/sanitizers) when initializing a dynamic array of a struct containing bool values to a number of slices not a multiple of the packet size.

Taking the example from here https://enoki.readthedocs.io/en/master/dynamic.html?highlight=bool_array_t#custom-dynamic-data-structures, if you do something like

using FloatP = Packet<float, 4>;
using FloatX = DynamicArray<FloatP>;
using GPSCoord2fX = GPSCoord2<FloatX>;
GPSCoord2fX coord;
set_slices(coord, 1001);

UBSAN will fire saying:

enoki/array_fallbacks.h:495:16: runtime error: load of value 190, which is not a valid value for type 'const bool'

I dug a little into this and traced it down to the clean_trailing_() function in dynamic.h, specifically this line;

store(addr, load<Packet>(addr) & mask);

Something weird is happening with the types here that it doesn't like. I think load<Packet>(addr) causes a read of uninitialized bool values, which are then put into the & expression at array_fallbacks.h:495. A workaround is changing the Bool type in the struct to:

using Bool = enoki::replace_scalar_t<Value, uint8_t>;

This avoids the UB and functions as you'd expect.

mp3guy avatar Jan 24 '20 15:01 mp3guy

A similar thing happens in template<typename T, typename U> ENOKI_INLINE T memcpy_cast(const U &val) in array_generic.h. It seems adding a special case for bool helps here, too. (Though I haven't tested the correctness of my band-aid-fix).

gergol avatar Mar 17 '20 11:03 gergol