libpostal
libpostal copied to clipboard
Enabling SSE creates memory write access violations
There are two main issues:
The remez9_0_log2_sse function assumes that the buffer it is handed has a size of multiples of 4 doubles. This isn't assured anywhere and make check will cause an access violation in the crf_context test because it allocates a buffer of 9.
The posix_memalign call "almost" handles this by aligning to 16 (2 doubles) as the allocated buffer will always be multiples of the alignment. Changing the alignment from 16 to 32 resolves this problem.
There is no such thing as a realloc for aligned memory, but vector.h tries to implement one. It is undefined whether realloc on a posix_memalign allocation even works... though from searching google it sounds like it does. But the problem is realloc doesn't take into account that the size needs to be a multiple of the alignment. So when the unit tests asks for 72, it gets 72 and the call to remez9_0_log2_sse gives an access violation.
The safe thing here would be to not use realloc. But then you have the issue that the _aligned_realloc function doesn't know the existing size of the buffer in order to do the copy. So you have to align the size to realloc yourself and hope the C library doesn't corrupt the heap.
There is something else going on too that I haven't figured out but my recommendation at the moment is to simply disable SSE by default.
Fixed by #632