[Meta] Controlling the environment with presets

[MathiasMagnus]

Brief Issue Summary

This issue is borderline a discsussion and intends to collect feedback around finding a solution for a number of issues around the inability to properly control or infer the environment to drive toolchains. An honest effort to enumerate related issues:

• #2912
• #3436
• #2036
• #1897
• #1885
• #2286

Relevant upstream issues:

• presets: sourcing environment scripts #21619

This linked issues, especially the upstream one has a fair amount of context. The takeaway in my read about all of it is:

• CMake (and Kitware) does not wish to concern itself with setting up an environment for configuring using makefile generators
• Users clearly want CMake to be concerned with that
  • It is a constant source of friction
  • IDE tooling can only mitigate the issue, but cannot solve it. CMake Tools can't help us in CI scenarios
• Presets already has a built-in mechanism to control the environment
  • The configuration env can propagate to build and tests in a well defined manner

The solution space is vast.

• Do what CMake Tools tries to do now and trigger extra logic for "strategy": "external" mixed with inspecting CMAKE_[C|CXX]_COMPILER
  • this won't help with shell/script/CI use-cases.
  • even in the IDE, this breaks with less popular toolchains.
    • nvc++, amdclang++, etc. I happen to be a GPGPU dev, so this is my everyday use-case.
    • I constantly keep getting Neither dumpbin, llvm-objdump nor objdump could be found. Can not take care of dll dependencies. warnings on every target when using Vcpkg because CMake Tools cares not about the env in my preset.
• One of the issue authors advocates for supporting the "cmakeGenerateCommand" preset vendor extension
  • neither will this help with shell/script/CI use-cases.
• Have CMake persist env changes during configuration-time to build/test/package-time and enact env reqs in CMAKE_TOOLCHAIN_FILEs or CMAKE_PROJECT_TOP_LEVEL_INCLUDES.
  • This requires CMake changes
• Have tools respect environments setup through presets and have toolchains declare their env requirements in a preset-friendly way.
  • This is what I've been trying to advocate for in the upstream issue here.
• ??? (something else?)

To elaborate a bit further on each, IDEs and other tools (GitHub/GitLab CI, homebrew scripts) trying to take on the burden of bootstrapping the environment is duplication of effort, yet another source of friction (CI matrix entries have to match exactly what GitHub actions / vcavrsall / etc. accept as arguments).

Telling users to launch their IDE from an appropriate shell is no solution either, switching betweenNVCC, NVC++, ROCmCC becomes tedious very soon, or we'll need to have 3 shells and 3 IDEs opening the same folder, which will drive other tooling crazy.

Proposed solution

If CMake can find all the MSBuild installations on a given system, why can't CMake (or even just toolchain vendors collectively) canonicalize some location on disk where toolchains/users can install their preset fragments which can then be included? This would likely need preset JSON schema extensions and behavioral additions. Proper MSBuild installations are detectable by CMake, at which point the user selects a toolchain using a custom string such as "v143,host=x64". Users of makefile driven toolchains (once CMake included all the JSONs from a canonical location) could also select one using either some string or something more structured. It could be something as simple as "llvm-latest", "llvm-16", or "visual-studio-17-2022-msvc-v143", "amd-rocm-latest" with each vendor coming up with a convenient string format, but could also be something more structured extending the available "condition"s with things like "versionGreaterEqual", etc.

I don't want to go too muc into detail, as it's not that important. What is currently missing is consensus on wanting to address the issue and doing so in a coordinated fashion (from Kitware's and tool vendor's POV).

My current workaround is running a custom script on every toolchain update that saves it in a non-standard location and exposes the environment as a hidden preset which other presets inherit, for eg. "developer-command-prompt-clang-rocm". This works fine locally, while CMake Tools outright disregards this, but in CI if I need to run a custom script, I'm back to square 1, at which point I may as well have used some action like Enable Developer Command Prompt.

CMake Tools Diagnostics

No response

Debug Log

No response

Additional Information

No response

[MathiasMagnus]

@gcampbell-msft @bobbrow Is this something the tool authors see as a problem or something that should be acted upon, discussed? Feel free to turn it into a discussion. The only reason I made it an issue is because it's actionable and definitely has some resolution.

Just after one week, yet another related issue came in:

• #3473

[herring-swe]

@MathiasMagnus I noted the exact same issue and made a very similar approach with CMakePresets.json including a generated cmake-file containing the environment of the user. I cannot grasp why sourcing environment from scripts are not allowed in CMakePresets.json. My big blaming finger is on KitWare for sure. As a fallback it would be nice if vscode could help setup such environment. Sourcing scripts is a requirement since it's impossible to include support for all compilers. Also more work needs to be put into merging preset environments (PATH-like variables must be merged). See my approach: https://github.com/herring-swe/cmake-presets

[MathiasMagnus]

@herring-swe If you ask me sourcing environment scripts isn't the best solution to this problem, for multiple reasons:

• vcvarsall.bat would add a considerable overhead to all CMake/CTest/CPack invocations.
• If you start authoring environment scripts for your custom toolchains, debugging those or their interplay with configuration will soon become a nightmare. The stateless nature of presets is a huge selling point.
  • the conditions part does resemble handwriting an AST, but that's the price we pay for manually authoring presets, and given its scope, it's still maintainable.

Preset inclusion seems like a more natural fit with some extra details sorted out, such as the merging of PATH-like env vars. While we're at merging, CMAKE_PREFIX_PATH is also something which multiple hidden presets would want to contribute to. Much like cacheVariables may be an object with key-value pairs or have properly typed entries in them, environment variables could also be as such and use a shared mechanism for presets merging their values, unlike $penv{NAME} which only allow one level of inheritance from the parent process.

{
  "configurePresets": [
    {
      "name": "some-repo",
      "hidden": true,
      "cacheVariables": {
        "CMAKE_PREFIX_PATH": {
          "type": "PATH",
          "value": "${pvalue}${cmakeListSeparator}C:\\Something",
          "merge": true
        }
      }
    },
    {
      "name": "some-tool",
      "hidden": true,
      "environment": {
        "ToolVar": "debug",
        "PATH": {
          "value": "${pvalue}${pathListSep}C:\\Tool",
          "merge": true,
          "inheritParent": true
        }
      }
    }
  ]
}

{pvalue} would be the parent value, which is the previous value in the inheritance chain and inheritParent is the zero-element in the fold over the values, so the 0th ${pvalue}. The existing $penv{NAME} doesn't mix well. It could also be pvalue{NAME}, but that would allow not repeating the variable name verbatim which can cause arcane errors/invalidities.

Once again, the actual design is unimportant and should be offloaded to a discussion of its own. The missing piece is consensus around wanting to solve this issue.

[herring-swe]

@MathiasMagnus

@herring-swe If you ask me sourcing environment scripts isn't the best solution to this problem, for multiple reasons:

• vcvarsall.bat would add a considerable overhead to all CMake/CTest/CPack invocations.

For more advanced scripts possibly. I think a fair tradeoff for simplicity. You can still have static environments if you prefer.

• If you start authoring environment scripts for your custom toolchains, debugging those or their interplay with configuration will soon become a nightmare. The stateless nature of presets is a huge selling point.

But no 3rd party presets are provided and we cannot count on vscode-cmake-tools to support the exact tools we want. So we are stuck sourcing scripts which most developers should know how to either call or create for batch tool setup. Or make tools to generate presets. My goal is to make it easy for the rest in my team and the latter is the best approach for us.

To go back to vscode-cmake-tools and it's CMake integration. Looking at the options with either Kits or Presets (or bleh, use a console...). Presets are for sure the cleanest and best approach for collaboration. I'm coming from QT-Creator where I didn't really have a problem setting up custom compilers (which were initialized using source scripts). For vscode there was a lot of fiddling and investigating to find a possible solution and most work I've done is to circumvent problems with the presets. For instance, presets are parsed differently than CMake. One that is validated by vscode-cmake-tools is not validated by cmake and the other way around (until restart of vscode). For instance cmake forbids inheriting presets defined in CMakePresets.json from another included cmake-file, in vscode-cmake-tools this was allowed and worked fine! And then fiddling with include order to actually make preset merging work as intended although no conflicts in variable names existed. In the end, I have something that works but I wouldn't trust merging preset environments.

Regarding PATH-like variables. A cleaner approach would be to tell which variables should be considered with append or prepend. For example:

{
  {
    "name": "some-tool",
    "hidden": true,
    "environmentPrependPath": {
      "PATH": "mypath${pathListSep}mypath2"
      "INC": "myinc"
    }
  }
}

Similarly this could be implemented for cacheVariables. Both cases deal with instances of knowing the separator. This avoids also ${pathListSep}$penv{PATH} which may evaluated to a dangling separator ':' which may be troublesome for whatever tool not dealing with it. I would also expect it to prepend the variables regardless of how the inheritance chain looks like... But again. This would fall upon KitWare.