simde icon indicating copy to clipboard operation
simde copied to clipboard

Possibility to generate custom amalgamated simde header for a given program?

Open mr-c opened this issue 4 years ago • 4 comments

Example, https://github.com/mengyao/Complete-Striped-Smith-Waterman-Library/blob/master/src/ssw.c is is the only file in a two** file C library that is designed to be dropped in to another source code tree, not compiled into a library.

Therefore adding SIMDe using this repo, simde-no-tests, or even the amalgamated SSE2 header feels too heavy.

It would be great if a slimmed down single header could be derived for just the functions and types used in a given source file. If small enough it could be appended/incorporated into an existing header file, thus keeping to the "copy these two files into your source tree and be done" principle given in the example above.

Such a tool/script should leave in markers in the generated code (as comments) that enable regeneration or updating with later SIMDe releases.

** four files, if one is using C++

mr-c avatar Jun 27 '20 09:06 mr-c

I like the idea, I just can't think of a realistic way to do it.

Functions often call other functions, so we would need some sort of way to calculate the dependencies. Throw in all the macros which make those dependencies vary by compiler, flags, target, etc. and you basically have to parse not only C, but the preprocessor stuff as well.

The most reasonable thing I can think of would be adding some sort of structured comments to the code to tell a tool where to cut files up and what each segment depends on. That would be a pain to develop, but writing all those comments up to date would require a lot more work and be a huge maintenance burden.

If you can think of a better idea I'd definitely be interested; this would be a nice feature.

As for that project, perhaps a better way to go would be to use a preprocessor macro to switch between SIMDe and the native headers? Something like:

#if defined(SSW_USE_SIMDE)
#  define SIMDE_ENABLE_NATIVE_ALIASES
#include <simde/x86/sse2.h>
#else
#  include <emmintrin.h>
#endif

Then you just pass -DSSW_USE_SIMDE and as long as your include path contains the simde directory you should be good.

Depending on how much you care about portability, you could also use __has_include:

#if defined(__has_include)
#  if __has_include(<simde/x86/sse2.h>)
#    include <simde/x86/sse2.h>
#  else
#    include <emmintrin.h>
#  endif
#else
#  include <emmintrin.h>
#endif

__has_include is in C++17 so most new compilers should support it. IIRC GCC and clang both support it as an extension even in non-C++17 (and even non-C++) modes; clang has supported it for a long time, but it's a pretty recent addition to GCC.

If it were a header-only library you could just do something like

#if !defined(_MM_SHUFFLE2)
#include <emmintrin.h>
#endif

And the file including it can just include SIMDe (with SIMDE_ENABLE_NATIVE_ALIASES) prior to including the header. Unfortunately that's not really something you can do with a *.c file; it's possible, but ugly and hackish.

nemequ avatar Jun 28 '20 16:06 nemequ

مثال ، https://github.com/mengyao/Complete-Striped-Smith-Waterman-Library/blob/master/src/ssw.c هو الملف الوحيد في مكتبة ** ملف C التي تم تصميمها ليتم إسقاطها في شجرة شفرة مصدر أخرى ، وليس تجميعها في مكتبة.

لذلك ، فإن إضافة SIMDe باستخدام هذا الريبو ، simde-no-testsأو حتى رأس SSE2 المدمج يبدو ثقيلًا للغاية.

سيكون من الرائع أن يتم اشتقاق رأس واحد مخفف للوظائف والأنواع المستخدمة في ملف مصدر معين. إذا كان حجمه صغيرًا بدرجة كافية ، فيمكن إلحاقه / دمجه في ملف رأس موجود ، وبالتالي الحفاظ على مبدأ "نسخ هذين الملفين إلى شجرة المصدر الخاصة بك وتنفيذها" الوارد في المثال أعلاه.

يجب أن تترك هذه الأداة / البرنامج النصي في علامات في الكود الذي تم إنشاؤه (كتعليقات) التي تتيح التجديد أو التحديث مع إصدارات SIMDe اللاحقة.

** أربعة ملفات ، إذا كان أحدهم يستخدم C ++

أنا هنا للعون و العمل معاً

AniesAliasGethub1 avatar Sep 07 '20 11:09 AniesAliasGethub1

@nemequ Is there a way to leverage gcc or clang to strip out the unneeded parts?

Perhaps it is time to approach GCC / clang about including SIMDe or some derivative as an opt-in implementation of <emmintrin.h> and so forth.

@AniesAliasGethub1 thank you for your message and your offer of help. Do you have a proposal of how to accomplish this? Would you like to explore using gcc or clang to produce a reduced version of SIMD for a particular codebase, like ssw.c ?

mr-c avatar Nov 25 '20 08:11 mr-c

I'm not really familiar with LLVM's internals, so I can't say for sure, but it would probably be possible to write something which parsed the code, eliminated dead code, then printed the code back out. I don't think it's feasible, though:

  • Obviously this would mangle the code, so the output would be ugly. Probably not a big deal, especially if you run it through clang-format.
  • It would have to happen after preprocessing, so the output wouldn't be portable to other architectures or compilers.
  • I'm not sure it would be possible to separate SIMDe's code out from the caller so you only dump SIMDe. You may be restricted to basically dumping the entire compilation unit.

The most feasible thing I can think of would be to use LLVM to generate a call graph to generate a list of required functions, then have some sort of special SIMDe-specific parser which could extract those functions; you wouldn't want, or need, a full C/C++ parser for this since you want to include all the preprocessor stuff. However, that will still break down in some situations since the call graph will reflect what LLVM is doing, which may not be what other compilers will do. For example, LLVM uses a_.f32 + b_.f32 to add a couple of 256-bit vectors, but MSVC would have fall back on 2 × simde_mm_add_ps.

nemequ avatar Dec 18 '20 23:12 nemequ