Windows compilation error: Missing rc.exe

[davidbtucker]

Hi. I'm trying to follow https://github.com/fluent/fluent-bit-docs/blob/master/installation/windows.md#compile-from-source, but I think there's a setup step missing.

When I follow the instructions on a fresh Windows Server 2019 VM, I get an error indicating that it cannot find the rc command:

C:\Users\dbtucker\tmp\fluent-bit\build>cmake .. -G "NMake Makefiles" -- The C compiler identification is MSVC 19.28.29334.0 -- The CXX compiler identification is MSVC 19.28.29334.0 -- Detecting C compiler ABI info -- Detecting C compiler ABI info - failed -- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/BuildTools/VC/Tools/MSVC/14.28.29333/bin/Hostx86/x86/cl.exe -- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/BuildTools/VC/Tools/MSVC/14.28.29333/bin/Hostx86/x86/cl.exe - broken CMake Error at C:/Program Files (x86)/Microsoft Visual Studio/2019/BuildTools/Common7/IDE/CommonExtensions/Microsoft/CMake/CMake/share/cmake-3.18/Modules/CMakeTestCCompiler.cmake:66 (message): The C compiler

"C:/Program Files (x86)/Microsoft Visual Studio/2019/BuildTools/VC/Tools/MSVC/14.28.29333/bin/Hostx86/x86/cl.exe"

is not able to compile a simple test program.

It fails with the following output:

Change Dir: C:/Users/dbtucker/tmp/fluent-bit/build/CMakeFiles/CMakeTmp

Run Build Command(s):nmake /nologo cmTC_acd7d\fast &&       "C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.28.29333\bin\HostX86\x86\nmake.exe"  -f CMakeFiles\cmTC_acd7d.dir\build.make /nologo -L                  CMakeFiles\cmTC_acd7d.dir\build
Building C object CMakeFiles/cmTC_acd7d.dir/testCCompiler.c.obj
    C:\PROGRA~2\MICROS~1\2019\BUILDT~1\VC\Tools\MSVC\1428~1.293\bin\Hostx86\x86\cl.exe @C:\Users\dbtucker\AppData\Local\Temp\2\nmC4EB.tmp
testCCompiler.c
Linking C executable cmTC_acd7d.exe
    "C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin\cmake.exe" -E vs_link_exe --intdir=CMakeFiles\cmTC_acd7d.dir --rc=rc --mt=CMAKE_MT-NOTFOUND --manifests -- C:\PROGRA~2\MICROS~1\2019\BUILDT~1\VC\Tools\MSVC\1428~1.293\bin\Hostx86\x86\link.exe /nologo @CMakeFiles\cmTC_acd7d.dir\objects1.rsp @C:\Users\dbtucker\AppData\Local\Temp\2\nmC5D6.tmp
RC Pass 1: command "rc /fo CMakeFiles\cmTC_acd7d.dir/manifest.res CMakeFiles\cmTC_acd7d.dir/manifest.rc" failed (exit code 0) with the following output:
The system cannot find the file specifiedNMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin\cmake.exe"' : return code '0xffffffff'
Stop.
NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.28.29333\bin\HostX86\x86\nmake.exe"' : return code '0x2'
Stop.

CMake will not be able to correctly generate this project. Call Stack (most recent call first): CMakeLists.txt:2 (project)

-- Configuring incomplete, errors occurred! See also "C:/Users/dbtucker/tmp/fluent-bit/build/CMakeFiles/CMakeOutput.log". See also "C:/Users/dbtucker/tmp/fluent-bit/build/CMakeFiles/CMakeError.log".

[edsiper]

@fujimotos ^

[fujimotos]

Hi. I'm trying to follow https://github.com/fluent/fluent-bit-docs/blob/master/installation/windows.md#compile-from-source, but I think there's a setup step missing. When I follow the instructions on a fresh Windows Server 2019 VM, I get an error indicating that it cannot find the rc command:

Hm. rc.exe is normally included in the default installation of MSVC. Please check your installation as follows:

1. Launch vs.exe (vs_buildtools.exe)
2. Check the module list. See if Windows SDK (10.0...) is selected.
3. If not, select that item and proceed installation.

I suspect that you did not install SDK and it causes the failure.

[davidbtucker]

Thanks Fujimoto. Yes, I followed those steps and this module is already selected: Windows 10 SDK (10.0.18362.0).

Maybe I need a different version of the SDK?

On Fri, Dec 4, 2020 at 7:56 PM Fujimoto Seiji [email protected] wrote:

Hi. I'm trying to follow https://github.com/fluent/fluent-bit-docs/blob/master/installation/windows.md#compile-from-source, but I think there's a setup step missing. When I follow the instructions on a fresh Windows Server 2019 VM, I get an error indicating that it cannot find the rc command:

Hm. rc.exe is normally included in the default installation of MSVC. Please check your installation as follows:

1. Launch vs.exe (vs_buildtools.exe)
2. Check the module list. See if Windows SDK (10.0...) is selected.
3. If not, select that item and proceed installation.

I suspect that you did not install SDK and it causes the failure.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/fluent/fluent-bit-docs/issues/427#issuecomment-739095560, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJLB333BKPPWCCENVHWGEITSTGALVANCNFSM4UNSVMWQ .

[fujimotos]

@davidbtucker Evidently this is a common pitfall of MSVC.

• https://stackoverflow.com/questions/14372706/visual-studio-cant-build-due-to-rc-exe
• https://stackoverflow.com/questions/43847542/rc-exe-no-longer-found-in-vs-2015-command-prompt

It appears that the VC installer can fail to place rc.exe to PATH.

On these threads, it was reported that the issue could be resolved by choosing and (re-)installing the correct Windows kit on the wizard.

FWIW the following is the packages installed on our clean dev environment:

... and I found rc.exe in the following path:

PS. Get-Command rc.exe

CommandType  Name     Version    Source
-----------  ----     -------    ------
Application  rc.exe   10.0.18... C:\Program Files (x86)\Windows Kits\10\bin\10.0.18362.0\x64\rc.exe

The above threads also suggest moving rc.exe to PATH manually, but it feels like a last resort.

[davidbtucker]

Interesting. I tried the uninstall & reinstall, but still no luck.

The file "C:\Program Files (x86)\Windows Kits\10\bin\10.0.18362.0\x64\rc.exe" does exist, but I guess it's not in the path.

FYI, this is the path var on my VM:

PS C:\Windows\system32> echo $Env:Path %PATH%;C:\WinFlexBison;C:\Program Files\Git\cmd;C:\Users\dbtucker\AppData\Local\Microsoft\WindowsApps;C:\WinFlexBison;C:\Program Files\Git\cmd;C:\Users\dbtucker\AppData\Local\Microsoft\WindowsApps

Does that look right? Thanks.

On Mon, Dec 14, 2020 at 4:23 AM Fujimoto Seiji [email protected] wrote:

@davidbtucker https://github.com/davidbtucker Evidently this is a common pitfall of MSVC.

https://stackoverflow.com/questions/14372706/visual-studio-cant-build-due-to-rc-exe

https://stackoverflow.com/questions/43847542/rc-exe-no-longer-found-in-vs-2015-command-prompt

It appears that the VC installer can fail to place rc.exe to PATH.

On these threads, it was reported that the issue could be resolved by choosing and (re-)installing the correct Windows kit on the wizard.

FWIW the following is the packages installed on our clean dev environment:

[image: vs2019] https://user-images.githubusercontent.com/8974561/102063186-0b629480-3e39-11eb-8b66-b35b31861188.png

... and I found rc.exe in the following path:

PS. Get-Command rc.exe

CommandType Name Version Source----------- ---- ------- ------ Application rc.exe 10.0.18... C:\Program Files (x86)\Windows Kits\10\bin\10.0.18362.0\x64\rc.exe

The above threads also suggest moving rc.exe to PATH manually, but it feels like a last resort.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/fluent/fluent-bit-docs/issues/427#issuecomment-744303819, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJLB334OQUQREH426IDUKC3SUXKQTANCNFSM4UNSVMWQ .

[fujimotos]

Does that look right? Thanks.

@davidbtucker Your PATH seems not correct as I see no VS paths there.

Try loading appropriate paths using vcvarsall.bat. You'll find that script in the following directory:

% call "C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Auxiliary\Build\vcvarsall.bat"

and see if C:\Program Files (x86)\Windows Kits\10\bin\10.0.18362.0\x64 is in your path.

I think the easiest way to confirm it is to invoke rc.exe:

# Execute rc.exe to see if it can be invoked
% rc.exe
# or use PowerShell's Get-Command 
% powershell
PS> Get-Command rc.exe

[davidbtucker]

Aha, the path "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\Build\vcvarsall.bat" does not exist. Under "C:\Program Files (x86)\Microsoft Visual Studio\2019" I only have a "BuildTools" subdirectory.

I'll try to investigate why it's missing.

On Wed, Dec 16, 2020 at 10:18 AM Fujimoto Seiji [email protected] wrote:

Does that look right? Thanks.

@davidbtucker https://github.com/davidbtucker Your PATH seems not correct as I see no VS paths there.

Try loading appropriate paths using vcvarsall.bat. You'll find that script in the following directory:

% call "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\Build\vcvarsall.bat"

and see if C:\Program Files (x86)\Windows Kits\10\bin\10.0.18362.0\x64 is in your path.

I think the easiest way to confirm it is to invoke rc.exe:

Execute rc.exe to see if it can be invoked

% rc.exe# or use PowerShell's Get-Command % powershell PS> Get-Command rc.exe

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/fluent/fluent-bit-docs/issues/427#issuecomment-746458613, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJLB333VD4I5774Q2UTYKLTSVDFT7ANCNFSM4UNSVMWQ .

[sophieyfang]

I followed the instructin and i saw rc.exe under the path mentioned in this comment:https://github.com/fluent/fluent-bit-docs/issues/427#issuecomment-744303819. However, i didn't build .exe successfully. There is another error popped up:

I use Windows Server 2016 Datacenter.

co.lib(libco.c.obj) : warning LNK4078: multiple '.text' sections found with different attributes (C0300040)
   Creating library ..\library\fluent-bit.lib and object ..\library\fluent-bit.exp
chunkio-static.lib(cio_os.c.obj) : error LNK2001: unresolved external symbol _SHCreateDirectoryExA
..\bin\fluent-bit.dll : fatal error LNK1120: 1 unresolved externals
NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin\cmake.exe"' : return code '0xffffffff'
Stop.
NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.28.29333\bin\HostX86\x86\nmake.exe"' : return code '0x2'
Stop.
NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.28.29333\bin\HostX86\x86\nmake.exe"' : return code '0x2'
Stop.

[sophieyfang]

The function SHCreateDirectoryExA with linking error is from https://docs.microsoft.com/en-us/windows/win32/api/shlobj_core/nf-shlobj_core-shcreatedirectoryexa which is used in fluent-bit: https://github.com/fluent/fluent-bit/blob/master/lib/monkey/mk_core/deps/libevent/evdns.c#L84

I wonder if the function is only available in certain versions of Windows as mentioned in the function description. I used Windows Server 2016 Datacenter.

[fujimotos]

I followed the instructin and i saw rc.exe under the path mentioned in this comment:#427 (comment). However, i didn't build .exe successfully. There is another error popped up:

I use Windows Server 2016 Datacenter.

@sophieyfang I took some time today compiling Fluent Bit on a clean-installed Windows Server to investigate your issue.

Here is the environment I used for my testing:

C:\fluent-bit\build> SYSTEMINFO
...
OS Name:                   Microsoft Windows Server 2016 Datacenter
...
C:\fluent-bit\build> cmake .. -G"NMake Makefiles"
...
Building for: Visual Studio 16 2019
Selecting Windows SDK version 10.0.18362.0 to target Windows 10.0.14393.
...

... and I actually could confirm that Fluent Bit compiles successfully (with the latest master HEAD ad66a2a):

C:\fluent-bit\build> nmake
...
[100%] Linking C executable ..\bin\fluent-bit.exe
   Creating library ..\library\fluent-bit.lib and object ..\library\fluent-bit.exp
Generating code
Finished generating code

I wonder if the function is only available in certain versions of Windows as mentioned in the function description. I used Windows Server 2016 Datacenter.

I actually can confirm that SHCreateDirectoryExA() is in Shell32 on Windows Server 2016. So it should compile without failure on your machine as well.

C:\Windows\system> dumpbin /exports shell32.dll
...
337   9A 002A3790 SHCreateDirectoryExA
...

[fujimotos]

@sophieyfang Can you apply the patch edsiper/chunkio/pull/60 to your lib/chunkio/src/CmakeLists.txt and rebuild the source code again?

(It's a single line patch, so you should be able to apply it trivially)

Try the following procedure to guarantee a clean recompilation:

# Add "Shell32.lib" to line 24
% notepad lib\chunkio\src\CMakeLists.txt
% rd /s /q build
% mkdir build
% cd build
% cmake -G"NMake Makefiles" ..
% nmake

Although it compiles fine without that change on our dev machine (and also on AppVeyor), I suspect that adding an explicit dependency might resolve your issue.

[sophieyfang]

Thanks Fujimoto!!

I have no trouble running cmake .. -G "NMake Makefiles". That command finishes successfully. The trouble appears when i run cmake --build . or nmake. I tried adding the Shell32.lib to line24 in that chunkio\src\CMakesLists.txt file, then retried commands, however still getting the exact same error.

[fujimotos]

@sophieyfang After some more digging, I found that I can reproduce your issue on 32-bit build (on the other hand, 64-bit build works fine).

I posted https://github.com/edsiper/chunkio/pull/62 that fixes the build issue.

# apply chunkio/pull/62 to 
% notepad lib\chunkio\src\cio_os.c
% nmake

You should be able to confirm that that patch resolves your issue by following the above steps.

[edsiper]

FYI: edsiper/chunkio#62 merged and ported to Fluent Bit git master

[sophieyfang]

Yes, that works out!! Thanks a lot Fujimoto :) and thanks Eduardo for the fast turnaround

Although my machine is 64bit:

But i cloned the fluent-bit folder and also executed the build commands under C:\Program Files (x86) directory. Maybe that's reason?🤷‍♀️

[fujimotos]

But i cloned the fluent-bit folder and also executed the build commands under C:\Program Files (x86) directory. Maybe that's reason?woman_shrugging

It depends on which Developer Command line tool you launched. According to Microsoft's official documentation:

• Developer Command Prompt - Sets the environment to use 32-bit, x86-native tools to build 32-bit, x86-native code.
• x86 Native Tools Command Prompt - Sets the environment to use 32-bit, x86-native tools to build 32-bit, x86-native code.
• x64 Native Tools Command Prompt - Sets the environment to use 64-bit, x64-native tools to build 64-bit, x64-native code.
• x86_x64 Cross Tools Command Prompt - Sets the environment to use 32-bit, x86-native tools to build 64-bit, x64-native code.
• x64_x86 Cross Tools Command Prompt - Sets the environment to use 64-bit, x64-native tools to build 32-bit, x86-native code.

https://docs.microsoft.com/en-us/cpp/build/building-on-the-command-line?view=msvc-160#developer_command_prompt_shortcuts

The essential point here is that the normal developer command prompt will target the build to 32-bit.

To compile the source in 64-bit mode, you need to choose "x64 Native Tools Command Prompt" in the start menu,

[sophieyfang]

This is a very valuable summary! Thanks for sharing.

[ezewena]

I have a same Error, and i don't understand any of your says.