A path to making OpenBLAS optarch aware.

[terjekv]

• A first and very raw implementation, mapping optarch's march directly to targets known to OpenBLAS.
• A better mapping is required. Skylake-avx512 vs SKYLAKEX as an obvious example.

[terjekv]

@boegel and @tgamblin, I'm trying to return to resolving the lack of optarch support for OpenBLAS, but in a slightly more generic fashion. @boegel asked if leaning on https://github.com/archspec/archspec would be a good idea, and that most certainly seems like a very good idea indeed. However, there are a few thoughts on how to proceed:

1. My current thinking is that the goal should be to map an architecture specification, be it GCCs nomenclature, OpenBLAS' own, or something completely different, to a label from archspec. From there we can thus "translate" from one naming scheme to another at will, using archspec as our Rosetta Stone,

2. The question then becomes one of implementation. Should these distinct nomenclatures be integrated as "plugins" or something similar to archspec, or should these individual libraries be "consumers" of archspec, remaining completely distinct from the central project? Separation removes clutter, but it require more "bits" to be installed in the form of distinct libraries, and this will be something the user will have to maintain (or optionally EasyBuild?)

3. Versioning of a given local nomenclature to archspec mappings. Archspec is already familiar with how to make certain supported compilers optimise for a given architecture, and what versions of the compilers are required. But, we would need an even broader mapping for applications. The OpenBLAS TargetList both adds and renames targets. This implies that a correct mapping will have to be release-dependent.

4. As a side note to this work... If we get this working, someone needs to decide the extent of EasyBuild's optarch support. Long term, support for cross-compiling will probably become a question that will require a definitive answer.

[tgamblin]

Did you look at the Spack openblas package? The mapping from archspec to openblas is here:

• https://github.com/spack/spack/blob/develop/var/spack/repos/builtin/packages/openblas/package.py#L141

It's close enough that we just search for a target based on the archspec name, with a few special cases.

Given that OpenBLAS is one package, I'm not inclined to add its names to archspec -- I think the translation belongs in the package file. If the goal is to avoid duplicating that work for lots of package managers, I'd say we should try to upstream archspec support to openblas -- it's already pretty close so they might go for something like this.

[terjekv]

Hi @tgamblin and thank you for your input! I poked at at the Spack implementation and it looks very smooth. I feel there needs to be a bit of version checking on the TargetList, but that's another problem all together.

As you say, cluttering the main archspec JSON is a bad idea, which is why I thought of secondary mappings or "plugins" to provide extended support. There are also cases beyond OpenBLAS that will eventually need addressing. However, in the short term, using archspec and the Spack implementation as a reference would solve the immediate issue, and then we can take it from there. Hm.

Thanks again for your input, very helpful!

(And if you can get OpenBLAS to be a little bit more conformant, that'd be amazing.)

[boegel]

@terjekv I think you'll agree with me that it's unlikely that this PR will get merged.

It was at least partially motivated by a need in EESSI, and we've started taking a different approach there in https://github.com/EESSI/software-layer/pull/260 by implementing a custom hook for OpenBLAS which injects DYNAMIC_ARCH=1 when using --optarch=GENERIC.

That could/should probably also be implemented in the OpenBLAS easyblock, but in a different way than is being done here - this PR does way more than just injecting DYNAMIC_ARCH=1.

[terjekv]

I'd wager it could be merged if I rewrote it to use DYNAMIC_ARCH in a sane way when seeing optarch=GENERIC, but hooks work fine for now. So fine I've been using it for long while myself.

Close and let a new, maybe working, PR arise if one ha the energy. :)