Provide way to pass env-vars from bat-file to cmake process (for nmake)

[kutschkem]

This is a...

• [X] Bug Report
• [X] Enhancement Request

Brief Description

For my nmake to work, I had to make sure to call vcvars32.bat before cmake. However I found no way to do this other than writing a wrapper batch script around cmake.

(If Bug Report) Brief Summary

What I tried:

1. Prepending the call to the build command in Preferences -> C/C++ Build -> Builder settings. This did not work as the command is not used for the cmake call.
2. Using it as pre-build step in Preferences -> C/C++ Build -> Settings -> Build steps. This did not lead to the batch file being called before cmake.
3. Prepend the call to Preferences -> C/C++ Build -> CMake -> OS Host override -> Windows -> CMake Executable. Didn't work because the entry is parsed as one command instead of allowing arbitrary shell script statements.

In the end I wrote a batch script that called vcvars32 and then passed the arguments to cmake, and used that as cmake executable.

What is the expected behavior?

One of the methods above should have allowed to call the script before cmake is executed to populate environment variables - or another method should exist that allows this.

What behavior are you observing?

cmake seems to be the first thing to be called in the build and no way exists to define a "pre-cmake" step.

Provide the steps to reproduce the issue, if applicable:

(If Enhancement Request) Describe it.

Add a "pre-cmake" step.

Useful Information

cmake4eclipse verssion: 1.10.3 Which OS do you use: Windows 7 Cmake version: 3.10.0

[15knots]

Calling vcvars32.bat from Eclipse won't help, a sub-process cannot set environment variables for its parent process (Eclipse in this case). You will have to stay with your existing cmake-wrapper or write a similar wrapper which start Eclipse.

[15knots]

One possible solution could be to call cmake in a sub-shell, and run vcvars32.bat in the same shell before. But that might fail when cmake is called from the makefile once a CMakeLists.txt was changed.

[kutschkem]

Shouldn't 3. work if the commands were not passed to the shell in one string but split?

[15knots]

Splitting the string at whitespaces? Then nobody will be able to add a path to cmake with white spaces, e.g c:\program files\cmake\...

[kutschkem]

Using a tokenizer should work, but that would mean "just" putting in a path with whitespaces wouldn't work anymore, they would have to be surrounded by quotation marks. (java.io.StreamTokenizer should work, I think)

[15knots]

The path-to-cmake should just hold what its label suggests, adding some token parsing would just confuse useres.

But I could think of an extra option 'run with environment script' which could allow to run a single batch file in the same shell prior to cmake.

[15knots]

What is your experience with your wrapper script that calls vcvars32 prior to cmake? Does it work when cmake is called from the makefile? You will need to edit a CMakeLists.txt and run a build to provoke that.

If that does not cause any issues, I could implement the solution proposed in https://github.com/15knots/cmake4eclipse/issues/91#issuecomment-429532799. .

[kutschkem]

I understand the question to be:

1. generate NMake Makefile with the wrapper script
2. build using nmake, not by calling cmake --build with the script
3. check that 2. is succesful

I will test that as soon as possible and let you know the results. My hope would be that all necessary information from the vcvars finds its way in the Makefile, but that needs to be tested for sure.

[15knots]

No, even simpler: With your wrapper script in place,

• run the build once to let the wrapper script generate the makefiles (but you did that already)
• edit one of your cmakelists.txt and run the build again.

The second build should trigger some makefile rules that invoke cmake from nmake in order to re-generate the makefiles (at least, this is the case with unix makefiles). The cmake process in this case is run without your wrapper script and does not have the environment vars that come from vcvars32. If cmake runs without errors here, we're fine.

[kutschkem]

OK that is simpler, I will try that.

[kutschkem]

No it does not work. I tested this by explicitly setting an environment variable in the wrapper script and reading it in the CMakeLists.txt. The first run picks up the variable, the second one does not.

[15knots]

Of course it will not work if you explicitly require a new env-var. But that is a different use case. Does it work if you, for example, add an add_exxecutable()?

[kutschkem]

Next test, I did:

1. build with wrapper script in place
2. change CMakeLists.txt to trigger cmake to run
3. build again - works as expected
4. change cmake executable to use the system one - builds as in 3.
5. remove CMakeCache.txt to trigger cmake to run again - build is unsuccessful

The issue, I think, is that what is not working is the compiler test - it tries to link against libs from the Windows SDK when trying to compile its "simple program", and only afterwards sets the necessary variables to do what the build takes. I think my environment variable test shows that the build can't be depending on the environment variables, somewhere cmake is setting the include paths.

Another test I did was setting the INCLUDE and LIB environment variables to the necessary directories for the Windows SDK - the build without the wrapper script worked fine. So somewhere CMake is keeping the IncludeDirectories and Libpath directories, the rest of the environment doesn't seem to matter. I do not know where it is that this information is kept.

[kutschkem]

By the way, when I change the cmake command, it does not force a cmake rerun, I think that is probably a bug.

[kutschkem]

I think I fixed my problem, but it is debatable how nice it is. I call the script and read out the environment variables it sets.

For INCLUDE:

execute_process(COMMAND $ENV{VS140COMNTOOLS}/VsDevCmd.bat && set include OUTPUT_VARIABLE WINDOWSSDK_INCLUDE_DIRS)
string(REPLACE "\\" "/" WINDOWSSDK_INCLUDE_DIRS "${WINDOWSSDK_INCLUDE_DIRS}")
string(REPLACE "INCLUDE=" "" WINDOWSSDK_INCLUDE_DIRS "${WINDOWSSDK_INCLUDE_DIRS}")
message(STATUS "Found the following VS140 include directories: ${WINDOWSSDK_INCLUDE_DIRS}")

And then the same for LIB. I put this all in a find_package script and then use it like any other library. Not really nice, but better than a wrapper script in my opinion.

[15knots]

Reply to https://github.com/15knots/cmake4eclipse/issues/91#issuecomment-435373963

Next test, I did: 1. build with wrapper script in place 2. change CMakeLists.txt to trigger cmake to run 3. build again - works as expected

Fine! This is the prerequisite of the fix of this issue..

4. change cmake executable to use the system one - builds as in 3.
5. remove CMakeCache.txt to trigger cmake to run again - build is unsuccessful

Step 5 is the same as the initial case with freshly checked out sources and no build directory. This case is going to be addressed by my proposed fix: Running cmake in a sub-shell, and run vcvars32.bat in the same shell before.

BTW, cmake stores the path to the compiler and linker in CMakeCache.txt and the header and library search path in the generated makefiles.

[15knots]

Reply to https://github.com/15knots/cmake4eclipse/issues/91#issuecomment-435443044:

With the proposed fix, you won't need that kind of CMakeLists.txt massaging.

[15knots]

Could you please try staged version 1.12.3 from https://drive.google.com/open?id=1PHVnRK_ovsExdKlhnjVL3aCAfkWlL_XI ? It is a zipped p2 update site.

[kutschkem]

It does not work, and I think it is because of the command line you generate:

cmd.exe /c "C:\\LegacyApp\\VisualStudio2015\\Common7\\Tools\\VsDevCmd.bat" && cmake -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON -DCMAKE_RULE_MESSAGES:BOOL=OFF -DCMAKE_EXPORT_COMPILE_COMMANDS:BOOL=ON -G "\"NMake Makefiles\"" "\"-DCMAKE_MODULE_PATH:PATH=C:\\Program Files\\eCAL\\cmake\"" -DCMAKE_BUILD_TYPE:STRING=Release -DOFFLINE:STRING=ON -DCMAKE_C_COMPILER_FORCED:STRING=1 -DCMAKE_CXX_COMPILER_FORCED:STRING=1 "C:\\Users\\<user name>\\runtime-xcit-rcp.product\\test_yaaf\\projects\\build\\autogen"

I think this creates a CMD subprocess and then a cmake subprocess, when what you wanted to achieve was the cmake subprocess INSIDE the cmd subprocess. What I think it should look like (enclosing double quotation marks):

cmd.exe /c ""C:\\LegacyApp\\VisualStudio2015\\Common7\\Tools\\VsDevCmd.bat" && cmake -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON -DCMAKE_RULE_MESSAGES:BOOL=OFF -DCMAKE_EXPORT_COMPILE_COMMANDS:BOOL=ON -G "\"NMake Makefiles\"" "\"-DCMAKE_MODULE_PATH:PATH=C:\\Program Files\\eCAL\\cmake\"" -DCMAKE_BUILD_TYPE:STRING=Release -DOFFLINE:STRING=ON -DCMAKE_C_COMPILER_FORCED:STRING=1 -DCMAKE_CXX_COMPILER_FORCED:STRING=1 "C:\\Users\\<user name>\\runtime-xcit-rcp.product\\test_yaaf\\projects\\build\\autogen""

See https://stackoverflow.com/a/12892791/1319284

[15knots]

Staged a new version that put everything after the /C in a single argument. https://drive.google.com/open?id=1PHVnRK_ovsExdKlhnjVL3aCAfkWlL_XI

Please test!

[kutschkem]

cmd.exe /c "C:\\LegacyApp\\VisualStudio2015\\Common7\\Tools\\VsDevCmd.bat && cmake -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON -DCMAKE_RULE_MESSAGES:BOOL=OFF -DCMAKE_EXPORT_COMPILE_COMMANDS:BOOL=ON -G \"NMake Makefiles\" \"-DCMAKE_MODULE_PATH:PATH=C:\\Program Files\\eCAL\\cmake\" -DCMAKE_BUILD_TYPE:STRING=Release -DOFFLINE:STRING=ON -DCMAKE_C_COMPILER_FORCED:STRING=1 -DCMAKE_CXX_COMPILER_FORCED:STRING=1 C:\\Users\\uidj1662\\runtime-xcit-rcp.product\\test_yaaf\\projects\\build\\autogen && exit %%ERRORLEVEL%%" 
CMake Error: Could not create named generator "NMake

Generators
  Visual Studio 15 2017 [arch] = Generates Visual Studio 2017 project files.
                                 Optional [arch] can be "Win64" or "ARM".
  Visual Studio 14 2015 [arch] = Generates Visual Studio 2015 project files.
                                 Optional [arch] can be "Win64" or "ARM".
  Visual Studio 12 2013 [arch] = Generates Visual Studio 2013 project files.
                                 Optional [arch] can be "Win64" or "ARM".
  Visual Studio 11 2012 [arch] = Generates Visual Studio 2012 project files.
                                 Optional [arch] can be "Win64" or "ARM".
  Visual Studio 10 2010 [arch] = Generates Visual Studio 2010 project files.
                                 Optional [arch] can be "Win64" or "IA64".
  Visual Studio 9 2008 [arch]  = Generates Visual Studio 2008 project files.
                                 Optional [arch] can be "Win64" or "IA64".
  Visual Studio 8 2005 [arch]  = Deprecated.  Generates Visual Studio 2005
                                 project files.  Optional [arch] can be
                                 "Win64".
  Borland Makefiles            = Generates Borland makefiles.
  NMake Makefiles              = Generates NMake makefiles.
  NMake Makefiles JOM          = Generates JOM makefiles.
  Green Hills MULTI            = Generates Green Hills MULTI files
                                 (experimental, work-in-progress).
  MSYS Makefiles               = Generates MSYS makefiles.
  MinGW Makefiles              = Generates a make file for use with
                                 mingw32-make.
  Unix Makefiles               = Generates standard UNIX makefiles.
  Ninja                        = Generates build.ninja files.
  Watcom WMake                 = Generates Watcom WMake makefiles.
  CodeBlocks - MinGW Makefiles = Generates CodeBlocks project files.
  CodeBlocks - NMake Makefiles = Generates CodeBlocks project files.
  CodeBlocks - NMake Makefiles JOM
                               = Generates CodeBlocks project files.
  CodeBlocks - Ninja           = Generates CodeBlocks project files.
  CodeBlocks - Unix Makefiles  = Generates CodeBlocks project files.
  CodeLite - MinGW Makefiles   = Generates CodeLite project files.
  CodeLite - NMake Makefiles   = Generates CodeLite project files.
  CodeLite - Ninja             = Generates CodeLite project files.
  CodeLite - Unix Makefiles    = Generates CodeLite project files.
  Sublime Text 2 - MinGW Makefiles
                               = Generates Sublime Text 2 project files.
  Sublime Text 2 - NMake Makefiles
                               = Generates Sublime Text 2 project files.
  Sublime Text 2 - Ninja       = Generates Sublime Text 2 project files.
  Sublime Text 2 - Unix Makefiles
                               = Generates Sublime Text 2 project files.
  Kate - MinGW Makefiles       = Generates Kate project files.
  Kate - NMake Makefiles       = Generates Kate project files.
  Kate - Ninja                 = Generates Kate project files.
  Kate - Unix Makefiles        = Generates Kate project files.
  Eclipse CDT4 - NMake Makefiles
                               = Generates Eclipse CDT 4.0 project files.
  Eclipse CDT4 - MinGW Makefiles
                               = Generates Eclipse CDT 4.0 project files.
  Eclipse CDT4 - Ninja         = Generates Eclipse CDT 4.0 project files.
  Eclipse CDT4 - Unix Makefiles= Generates Eclipse CDT 4.0 project files.

Buildfile generation error occurred..
cmd.exe exited with status 1. See CDT global build console for details.
Build stopped..

I think you don't need to escape the quotes inside, it looks for the first and last quotes automatically, I think. See https://ss64.com/nt/cmd.html

/S Strip " quote characters from command. If command starts with a quote, the first and last quote chars in command will be removed, whether /s is specified or not.

Spaces in Program Path + parameters with spaces: CMD /k ""c:\batch files\demo.cmd" "Parameter 1 with space" "Parameter2 with space""

[15knots]

My plugin does not do the quote-escaping, it just passes the arguments cmd, /c and envars.bat && cmake "-G NMake Makefiles" ... to an instance of ICommandLauncher. All that quoting happens somewhere in the windows-implementation of Javas Runtime.exec(String[] args) or in Spawner.exec0( String[] cmdarray, String[] envp, String dir, int[] chan). The latter is implemented as windows-specific native C-code. With all that unknown quoting and cmd.exes unknown/confusing un-quoting, someone with access to a windows machine should help here.

Anyway, I'll give it a last blind try. Staged a new version. https://drive.google.com/open?id=1PHVnRK_ovsExdKlhnjVL3aCAfkWlL_XI

[kutschkem]

OK, I'll test and if it doesn't work I can see if I can make time to work on it. Which branch is it in?

[15knots]

It is not checked in currently. If you want to do work on it, start using Runtime.exec(String[] cmdline) and figure the out the cmdline-array to call cmd.exe and run two commands (envars.bat and cmake) which may have spaces in theire command name and spaces in their arguments. Then hope the Spawner does the same quoting as Runtime.exec() :-)

[kutschkem]

OK will do. I assume I can somehow trick it with single quotes.

[kutschkem]

This works for me:

package test_cmake4eclipse;

import java.io.IOException;
import java.lang.ProcessBuilder.Redirect;

public class Main {

  public static void main(String[] args) throws IOException, InterruptedException {
    String[] cmdarray = new String[] {"cmd.exe", "/c", "C:\\LegacyApp\\VisualStudio2015\\Common7\\Tools\\VsDevCmd.bat && cmake -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON -DCMAKE_RULE_MESSAGES:BOOL=OFF -DCMAKE_EXPORT_COMPILE_COMMANDS:BOOL=ON -G \"NMake Makefiles\" \"-DCMAKE_MODULE_PATH:PATH=C:\\Program Files\\eCAL\\cmake\" -DCMAKE_BUILD_TYPE:STRING=Release -DOFFLINE:STRING=ON -DCMAKE_C_COMPILER_FORCED:STRING=1 -DCMAKE_CXX_COMPILER_FORCED:STRING=1 C:\\Users\\uidj1662\\runtime-xcit-rcp.product\\test_yaaf\\projects\\build\\autogen && exit %%ERRORLEVEL%%"};
    ProcessBuilder bldr = new ProcessBuilder(cmdarray);
    bldr.redirectOutput(Redirect.INHERIT);
    bldr.redirectError(Redirect.INHERIT);
    Process pr = bldr.start();
    pr.waitFor();

  }

}

[kutschkem]

Tested the last staged version, didn't work, with similar error message as before.

[kutschkem]

The same array as above also works with Runtime.exec

[15knots]

But that is exactly what one of the previous versions did: Enclose any argument with spaces in double quotes.

[kutschkem]

I'll see if I can find that version and debug what went wrong.