Windows Support

[RoyiAvital]

It seems that Windows Support is implied yet I couldn't find any explicit documentation about it in the web site (For instance, Installation section in https://blasfeo.syscop.de).

Are there any official instructions to create a Static and Shared libraries of BLASFEO under Windows?

[giaf]

First of all, please notice that personally my experience with windows is limited, so I can not provide as much giudance as with linux.

That said, in windows you can either:

• use make (which supports both BLASFEO and BLAS APIs) and MinGW-w64 compiler (which supports any target, provided that OS in Makefile.rule is set to WINDOWS to correctly build the assembly code for the windows function calling convention), both of which you can install with e.g. the Cygwin environment;
• use cmake (that at the moment supports the BLASFEO API but not the BLAS API) and visual studio compiler (that at the moment only support the generic target);
• it should be possible to do also something with WSL (windows subsystem for linux) in windows 10, but I don't have direct experience with that, so I don't know what is the workflow there.

If you try out any of them, I would appreciate to get feedback on it :)

[giaf]

If some other user like e.g. @jkoendev has some experience to share, it would be great :)

[jkoendev]

i compiled acados on windows using cmake and mingw-w64 which also builds blasfeo, so it is probably similar.

I documented the procedure here: https://github.com/acados/acados/issues/411

[RoyiAvital]

@giaf , Where can I find how to build CMake + MSVC with the GENERIC target? What prevents using other targets?

[giaf]

Personally I don't have first-hand experience on windows systems using MSVC, but in that case building the GENERIC target of BLASFEO using cmake shouldn't be any different than building any other C project.

About the use of other targets than GENERIC, the issue there is that the assembly .S files are coded in the AT&T syntax, while MSVC requires the Intel syntax (while GENERIC is entirely written in C without assembly). One work around is to use mingw-w64 to assemble the .S files into .o files, and then modify the CMakeLists.txt to have as targets the .o files instead of the .S files. In this way, MSVC will simply compile the remaining .c files and make a library out of all the .o files (regardless of whether they originate form .S or .c files). This works because the BLASFEO .S files are entirely self-contained and don't contain any call to e.g. system functions.

[RoyiAvital]

What about Clang? We have Clang in Windows.

By the way, on Windows when I set SHARED_LIBS=ON it doesn generate the DLL but I don't see the .lib file to attach. At least usually for project to use DLL you need the lib for the compilation phase.

Remark: I'm under the impression the lib files generated in the non shared libs mode are those for static linking.

[TureganoJose]

I managed to build it in Windows with CMake, creating a solution for VS then you can build either, static or dynamic libraries. There was a bit of fiddling with the c code. Apparently void* is handled in a different way by gcc.

[RoyiAvital]

@TureganoJose , could you share your code and configuration? I'd assume it is something @giaf will be happy about - offering Windows compatibility.

Thank You.

[TureganoJose]

I'm fine with sharing as long as @giaf is ok with it. I'm using for my own purposes.

[giaf]

@TureganoJose sure I would be happy to include the changes (code and configuration description) in the repo to improve Windows compatibility. Thanks for the work on that!

Can you make a pull request or share a git diff or something similar with the changes you did? Then I can go though them and merge with the repo.

[giaf]

The code has now been fixed to build library and examples using cmake on windows with visual studio compiler, for the GENERIC target in BLASFEO.

@TureganoJose posted some pictures to show the necessary setup steps, they can be found in here https://github.com/giaf/blasfeo/pull/119

[RoyiAvital]

@giaf , This is great!

Is there a hope to have non generic as well? Since we have GCC (MinGW) and Clang-CL on Windows.

[giaf]

Hi @RoyiAvital Yes with MinGW-w64 you should be able to compile any target, while I can't say with Clang-CL because I don't have any experience with that (Clang can compile any target on linux, so there is hope). OTOH visual studio is only limited to GENERIC target because it doesn't accept the assembly dialect used in the other BLASFEO targets.

If you want to experiment and give some feedback, it would be great :) I don't have access to a windows machine ATM.

[RoyiAvital]

So no need for MSYS or Cygwin, just GCC on MinGW64 and it will work?

Which dialect of assembly do you use? Which is used by VS?

[giaf]

In theory you just need MinGW-w64 and cmake. Then if you have MSYS or Cygwin you can also use the make build system, git and whatever you want like if you were in linux, if this is your usual workflow. I tested some time ago, but again I can't do it now.

For x86 and x86_64 targets, I use AT&T syntax, while visual studio only accepts Intel syntax AFAIK. (If the use of visual studio is necessary e.g. for inclusion in another toolchain, and if there are both MinGW-w64 and visual studio available on the system, an option is to use MinGW-w64 to assemble the .S files into windows object files, and then use visual studio to compile the remaining .c files and include all object files into a library. This would give the possibility to use any BLASFEO target in a visual studio project.)

[RoyiAvital]

This is what I thought. Maybe using GIT LFS we can add pre compiled object files of the assembly for Windows users?

I can try generating them and build something to automate their creation (Given it is easy to do using MinGW64). Should I generate the S files in the Archive (For example avx/archive/) sub folders or only the parent folder (avx/)?

[giaf]

The issue is that I edit also the assembly files on a regular basis while developing new features. So it would be great to have the object files automatically precompiled somewhere on the cloud, for every commit. Or alternatively to release them only once in a while attached to some specially tagged commit. @tmmsartor do you have any suggestion here?

Whatever is in any archive folder should be disregarded, as it is not compiled in the libraries. As a guideline, only the .o files generated from .S by the make build system are needed, so in theory it is only a matter of running make static_library from the main BLASFEO folder and then collect the needed files. (All .S files are completely self-contained with e.g. no system calls, this makes it possible to compile them with MinGW-w64 and include them with visual studio.)

[RoyiAvital]

OK, I'm trying to compile the project on Windows using MinGW64.

I get an error no matter what target I chose:

Generic Target

cmake -G"MinGW Makefiles" -DTARGET=GENERIC ..
-- The C compiler identification is GNU 7.3.0
-- The ASM compiler identification is GNU
-- Found assembler: D:/Applications/Programming/MinGW/bin/gcc.exe
-- Check for working C compiler: D:/Applications/Programming/MinGW/bin/gcc.exe
-- Check for working C compiler: D:/Applications/Programming/MinGW/bin/gcc.exe - works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Using linear algebra: HIGH_PERFORMANCE
-- Using external BLAS: 0
-- Testing target GENERIC: compilation [failed]
Compile output:
Change Dir: C:/Users/User/BLASFeo/Build/compilerTest/GENERIC/CMakeFiles/CMakeTmp

Run Build Command(s):D:/Applications/Programming/MinGW/bin/mingw32-make.exe cmTC_08227/fast && D:/Applications/Programming/MinGW/bin/mingw32-make.exe  -f CMakeFiles\cmTC_08227.dir\build.make CMakeFiles/cmTC_08227.dir/build
mingw32-make.exe[1]: Entering directory 'C:/Users/User/BLASFeo/Build/compilerTest/GENERIC/CMakeFiles/CMakeTmp'
Building C object CMakeFiles/cmTC_08227.dir/isa_test.c.obj
D:\Applications\Programming\MinGW\bin\gcc.exe   -DLA_HIGH_PERFORMANCE -DK_MAX_STACK=300 -DUSE_C99_MATH -DEXT_DEP -DOS_WINDOWS -MT -DEXTERNAL_BLAS_NONE -O2 -fPIC     -o CMakeFiles\cmTC_08227.dir\isa_test.c.obj   -c "C:\Users\User\BLASFeo\cmake\isa_tests\isa_test.c"
cc1.exe: error: to generate dependencies you must specify either -M or -MM
mingw32-make.exe[1]: *** [CMakeFiles\cmTC_08227.dir\build.make:85: CMakeFiles/cmTC_08227.dir/isa_test.c.obj] Error 1
mingw32-make.exe[1]: Leaving directory 'C:/Users/User/BLASFeo/Build/compilerTest/GENERIC/CMakeFiles/CMakeTmp'
mingw32-make.exe: *** [Makefile:140: cmTC_08227/fast] Error 2


CMake Error at cmake/TestSingleTarget.cmake:25 (message):
  Unable to compile for target GENERIC
Call Stack (most recent call first):
  CMakeLists.txt:308 (TestSingleTarget)


-- Configuring incomplete, errors occurred!
See also "C:/Users/User/BLASFeo/Build/CMakeFiles/CMakeOutput.log".

Automatic Target

cmake -G"MinGW Makefiles" -DTARGET=X64_AUTOMATIC ..
-- The C compiler identification is GNU 7.3.0
-- The ASM compiler identification is GNU
-- Found assembler: D:/Applications/Programming/MinGW/bin/gcc.exe
-- Check for working C compiler: D:/Applications/Programming/MinGW/bin/gcc.exe
-- Check for working C compiler: D:/Applications/Programming/MinGW/bin/gcc.exe - works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Using linear algebra: HIGH_PERFORMANCE
-- Using external BLAS: 0
-- Testing target X64_INTEL_HASWELL: compilation [failed]
-- Testing target X64_AMD_BULLDOZER: compilation [failed]
-- Testing target X64_INTEL_SANDY_BRIDGE: compilation [failed]
-- Testing target X64_INTEL_CORE: compilation [failed]
-- Testing target GENERIC: compilation [failed]
CMake Error at cmake/X64AutomaticTargetSelection.cmake:31 (message):
  Unable to identify a target to use.  Please select one manually.
Call Stack (most recent call first):
  CMakeLists.txt:304 (X64AutomaticTargetSelection)

Intel Core Target

cmake -G"MinGW Makefiles" -DTARGET=X64_INTEL_CORE ..
-- The C compiler identification is GNU 7.3.0
-- The ASM compiler identification is GNU
-- Found assembler: D:/Applications/Programming/MinGW/bin/gcc.exe
-- Check for working C compiler: D:/Applications/Programming/MinGW/bin/gcc.exe
-- Check for working C compiler: D:/Applications/Programming/MinGW/bin/gcc.exe - works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Using linear algebra: HIGH_PERFORMANCE
-- Using external BLAS: 0
-- Testing target X64_INTEL_CORE: compilation [failed]
Compile output:
Change Dir: C:/Users/User/BLASFeo/Build/compilerTest/X64_INTEL_CORE/CMakeFiles/CMakeTmp

Run Build Command(s):D:/Applications/Programming/MinGW/bin/mingw32-make.exe cmTC_f5e5d/fast && D:/Applications/Programming/MinGW/bin/mingw32-make.exe  -f CMakeFiles\cmTC_f5e5d.dir\build.make CMakeFiles/cmTC_f5e5d.dir/build
mingw32-make.exe[1]: Entering directory 'C:/Users/User/BLASFeo/Build/compilerTest/X64_INTEL_CORE/CMakeFiles/CMakeTmp'
Building C object CMakeFiles/cmTC_f5e5d.dir/isa_test.c.obj
D:\Applications\Programming\MinGW\bin\gcc.exe   -DLA_HIGH_PERFORMANCE -DK_MAX_STACK=300 -DUSE_C99_MATH -DEXT_DEP -DOS_WINDOWS -MT -DEXTERNAL_BLAS_NONE -O2 -fPIC -m64 -msse3 -DTEST_SSE3   -o CMakeFiles\cmTC_f5e5d.dir\isa_test.c.obj   -c "C:\Users\User\BLASFeo\cmake\isa_tests\isa_test.c"
cc1.exe: error: to generate dependencies you must specify either -M or -MM
mingw32-make.exe[1]: *** [CMakeFiles\cmTC_f5e5d.dir\build.make:85: CMakeFiles/cmTC_f5e5d.dir/isa_test.c.obj] Error 1
mingw32-make.exe[1]: Leaving directory 'C:/Users/User/BLASFeo/Build/compilerTest/X64_INTEL_CORE/CMakeFiles/CMakeTmp'
mingw32-make.exe: *** [Makefile:140: cmTC_f5e5d/fast] Error 2


CMake Error at cmake/TestSingleTarget.cmake:25 (message):
  Unable to compile for target X64_INTEL_CORE
Call Stack (most recent call first):
  CMakeLists.txt:308 (TestSingleTarget)


-- Configuring incomplete, errors occurred!
See also "C:/Users/User/BLASFeo/Build/CMakeFiles/CMakeOutput.log".

This is the log file if you're interested: CMakeOutput.log

[giaf]

Hi @RoyiAvital By looking at the CFLAGS above, I notice the flag -MT which is not one I set, nor I can reproduce when I run the command on my linux PC. And it seems to be related to the error message somehow https://gcc.gnu.org/onlinedocs/gcc/Preprocessor-Options.html I guess this flag comes from cmake for the target compiler, or something like that.

In this regard, I also notice that the compiler is mingw32. Now I don't recall exactly, but I seem to remember that there was some issue with that (I think related to 32-bitness, but anyway still issues), while the compiler MinGW-w64 (which is a different project) used to work. This is the best hint I can give you now to find the issue.

[RoyiAvital]

@giaf , You're the one setting the -MT flag:

if(${CMAKE_HOST_SYSTEM_NAME} MATCHES "Windows")
	set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -DOS_WINDOWS -MT")
	set(CMAKE_ASM_FLAGS "${CMAKE_ASM_FLAGS} -DOS_WINDOWS")
endif()

Probably to use the static run time.

The compile is MinGW64. I use this distribution (Works perfectly on other projects) - https://nuwen.net/mingw.html.

By the way, even when I remove it I get:

Building ASM object CMakeFiles/cmTC_8b366.dir/TEST_SSE3.S.obj
D:\Applications\Programming\MinGW\bin\gcc.exe   -DTEST_SSE3   -o CMakeFiles\cmTC_8b366.dir\TEST_SSE3.S.obj -c "C:\Users\User\BLASFeo\cmake\isa_tests\TEST_SSE3.S"
Linking C executable cmTC_8b366.exe
D:\Applications\Programming\CMake\bin\cmake.exe -E cmake_link_script CMakeFiles\cmTC_8b366.dir\link.txt --verbose=1
D:\Applications\Programming\CMake\bin\cmake.exe -E rm -f CMakeFiles\cmTC_8b366.dir/objects.a
D:\Applications\Programming\MinGW\bin\ar.exe cr CMakeFiles\cmTC_8b366.dir/objects.a @CMakeFiles\cmTC_8b366.dir\objects1.rsp
D:\Applications\Programming\MinGW\bin\gcc.exe  -DLA_HIGH_PERFORMANCE -DK_MAX_STACK=300 -DUSE_C99_MATH -DEXT_DEP -DEXTERNAL_BLAS_NONE -O2 -fPIC -m64 -msse3 -DTEST_SSE3    -Wl,--whole-archive CMakeFiles\cmTC_8b366.dir/objects.a -Wl,--no-whole-archive  -o cmTC_8b366.exe -Wl,--out-implib,libcmTC_8b366.dll.a -Wl,--major-image-version,0,--minor-image-version,0 @CMakeFiles\cmTC_8b366.dir\linklibs.rsp
CMakeFiles\cmTC_8b366.dir/objects.a(isa_test.c.obj):isa_test.c:(.text.startup+0xa): undefined reference to `test_sse3'
collect2.exe: error: ld returned 1 exit status
mingw32-make.exe[1]: *** [CMakeFiles\cmTC_8b366.dir\build.make:115: cmTC_8b366.exe] Error 1

[RoyiAvital]

OK, I solved it by changing:

if(${CMAKE_HOST_SYSTEM_NAME} MATCHES "Windows")
	set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -DOS_WINDOWS -MT")
	set(CMAKE_ASM_FLAGS "${CMAKE_ASM_FLAGS} -DOS_WINDOWS")
endif()

Into:

if(${CMAKE_HOST_SYSTEM_NAME} MATCHES "Windows")
	set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -DOS_WINDOWS")
	set(CMAKE_ASM_FLAGS "${CMAKE_ASM_FLAGS} -DOS_WINDOWS")
endif()

By the way, no need for the -MT flag. New versions of CMAKE will ignore it anyway on MSVC.

The question now is what difference does it make to have DOS_WINDOWS compared to DOS_LINUX or DOS_MAC? Does it have any effect on the code itself?

By the way, GCC on Windows has issues with AVX code. See https://github.com/msys2/MSYS2-packages/issues/1209#issuecomment-379547589.

[giaf]

I don't have the flag -MT in the CMakeLists.txt in the current version of BLASFEO https://github.com/giaf/blasfeo/blob/master/CMakeLists.txt#L273 and I couldn't find it with blame on older versions too.

[RoyiAvital]

I'm sorry, so it might be me who has done it before and forgot.

Sorry for that.

[giaf]

What is the difference between the two codes in the fix besides the -MT ?

Yes -DOS_WINDOWS vs LINUX or MAC also affects the code, as the function calling convention is different for the three. So for assembly functions, the correct function calling convention has to be used by hand, see e.g. https://github.com/giaf/blasfeo/blob/master/cmake/isa_tests/TEST_SSE3.S This applies to all .S files. Additionally, some system functions are different in the different systems, so the correct one has to be picked up, e.g. https://github.com/giaf/blasfeo/blob/master/auxiliary/blasfeo_stdlib.c

[RoyiAvital]

I think I just ran cmake -G"MinGW Makefiles" -DBUILD_SHARED_LIBS=OFF -DTARGET=X64_INTEL_CORE -DBLASFEO_TESTING=OFF -DBLASFEO_BENCHMARKS=OFF -DBLASFEO_EXAMPLES=OFF .. and it worked.

I chose X64_INTEL_CORE since I'm not sure if AVX2 will work on Windows with GCC as I linked above.

I think the best policy is to make Clang-CL support on Windows.

[giaf]

In BLASFEO memory alignment is handled explicitly, so I think there shouldn't be issues in that regards. But can give it a try.

Actually I just fixed some remaining alignments which may have created issues with visual studio compiler for non-GENERIC targets. https://github.com/giaf/blasfeo/commit/29329ec67e8e96b83ea867079c6a65b3dea23e21

[RoyiAvital]

Could you make a new release? Currently I'm on 0.1.1.

[giaf]

Yes once all of this is fixed I can make a release, also because I'm planning some change which shouldn't get into it.

[RoyiAvital]

I am working on a Pull Request to fix things. Please wait for me. I'd be honored to contribute here.

[giaf]

Ok then, thanks a lot :)