Target AVX2 in releases?

[richardstartin]

I noticed this in the makefile

ifndef CXXFLAGS_BITSHUFFLE
  ifeq ($(OS_NAME)-$(OS_ARCH),Linux-x86_64)
	# SSE2 is supported in all the x86_64 platforms and AVX2 is only supported
        # in the small part of them. gcc in linux/x86_64 typically enables SSE2 by default though,
	# we explicitly set flags below to make this precondition clearer.
	CXXFLAGS_BITSHUFFLE:=-U__AVX2__ -msse2
  else
	# Undefined macros to generate a platform-independent binary
	CXXFLAGS_BITSHUFFLE:=-U__AVX2__ -U__SSE2__
  endif
endif

This was written in 2016 but AVX2 is quite mainstream now, is it worth revisiting this? Obviously a user can build their own, but not all users will have the required skills to do this in order to get more from their commodity class servers.

[xerial]

@maropu What do you think?

[maropu]

@richardstartin Thanks for your attention! Yea, as you said, I also think most processors have AVX2, e.g., we can use AVX2 on all the x64 AWS ec2 instances, right?

I found a list of processors with AVX2; https://en.wikipedia.org/wiki/Advanced_Vector_Extensions#CPUs_with_AVX2 It seems Intel processors based on Nehalem, Sandy Bridge, and Ivy Bridge don't have AVX2. I'm not 100% sure that no user uses these processors now? I remember we turn off AVX2 because the portability is the highest priority among the others, that is, run everywhere on x64 platforms! The decision depends on how we should respect minor users using these processors.

Anyway, I think we have some options for that;

• Waits some more time to support AVX2, one or two more years? Not sure.
• Turns on AVX2 by default and then document it explicitly
• Bundles two binaries for x64 (with/without AVX2) and then load the correct binary by checking the support on runtime (cpuid)
• Other ideas?

Also, it might be worth checking performance gain when turning on AVX2. If we have less gain, we might not need the support soon.

[re-thc]

Having multiple binaries and deciding at startup might be a good idea. If space is a concern, maven classifiers can always be used to filter out binaries included in the jar, so it's not an issue.