Future Feature: Import crc32 SIMD code from Linux kernel

[marcusmueller]

arch/x86/ contains several pieces of GPL (or compatible, haven't checked) code that does very DSPy stuff.

Of prime interest are the crc32 implementations, both for smaller chunks of data, which uses the crc32 SSE4.2 instruction, as well as a version that uses the PCLMULQDQ (which, inside the kernel is used only for pretty large chunks of data, as using that requires stacking the FP registers).

Bonus: Stuff is well-tested.

[jdemel]

So far we don't have any CRC code in VOLK. There are libraries like CRCpp that promise very fast results. Would copying over CRC code from the Linux kernel do a better job? I suggest we do a performance evaluation first. Then decide if we want CRC code in VOLK.

[michaelld]

I'd love to see CRC speedups in Volk! If someone has time, please make it so in a PR! Until then, let's leave this issue open to remind us that this would be a cool addition in the future sometime.

[jdemel]

I assume this enhancement becomes obsolete with our move to LGPL.

[marcusmueller]

No, as most of the code in these assembler / optimized implementation files is not actually GPL, but MIT, BSD and others (e.g. arch/x86/crypto/crc32c-pcl-intel-asm_64.S)

[Mixaill]

DPDK contains CRC implementations using SSE4.2, AVX512 and NEON under BSD 3-clause license.

https://github.com/DPDK/dpdk/blob/master/lib/net/net_crc_sse.c