scons MSVC Version 14.1 not found with 14.2 installed

Describe the bug I have VS2019 installed, with the VC++ 14.2, 14.1, and 14.0 build tools selected in the installer. When MSVC_VERSION="14.1" is specified, SCons cannot find the 14.1 build tools. When MSVC_VERSION="14.2" or 14.0 are specified, SCons works as expected.

Required information

Link to SCons Users thread discussing your issue: https://discord.com/channels/571796279483564041/571796280146133047/713183631547302008
Version of SCons: 3.1.2
Version of Python: 3.7.9
Which python distribution if applicable (python.org, cygwin, anaconda, macports, brew,etc): Anaconda
How you installed SCons: Conda
What Platform are you on? (Linux/Windows and which version): Windows 10
How to reproduce your issue? Please include a small self contained reproducer. Likely a SConstruct should do for most issues.

env = Environment(MSVC_VERSION="14.1")
env.Program("hello", "hello.cpp")

How you invoke scons (The command line you're using "scons --flags some_arguments"): scons

Further information:

With MSVC_VERSION="14.2", SCons generates this command for vswhere.exe: vswhere.exe -products * -version "[16.0, 17.0)" -property installationPath and the output when run on the command line is: C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise
With MSVC_VERSION="14.1", SCons generates this command: vswhere.exe -products * -version "[15.0, 16.0)" -property installationPath and there is no output when run on the command line
The 14.16 (current version as of this writing) build tools are located in subfolders of C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.16.27023
The output of vswhere --products * -format json is:

[   {     "instanceId": "7876f29c",     "installDate": "2020-02-04T16:37:05Z",     "installationName": "VisualStudio/16.6.0+30114.105",     "installationPath": "C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Enterprise",     "installationVersion": "16.6.30114.105",     "productId": "Microsoft.VisualStudio.Product.Enterprise",     "productPath": "C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Enterprise\\Common7\\IDE\\devenv.exe",     "state": 4294967295,     "isComplete": true,     "isLaunchable": true,     "isPrerelease": false,     "isRebootRequired": false,     "displayName": "Visual Studio Enterprise 2019",     "description": "Scalable, end-to-end solution for teams of any size",     "channelId": "VisualStudio.16.Release",     "channelUri": "https://aka.ms/vs/16/release/channel",     "enginePath": "C:\\Program Files (x86)\\Microsoft Visual Studio\\Installer\\resources\\app\\ServiceHub\\Services\\Microsoft.VisualStudio.Setup.Service",     "releaseNotes": "https://go.microsoft.com/fwlink/?LinkId=660893#16.6.0",     "thirdPartyNotices": "https://go.microsoft.com/fwlink/?LinkId=660909",     "updateDate": "2020-05-20T17:41:11.1443849Z",     "catalog": {       "buildBranch": "d16.6",       "buildVersion": "16.6.30114.105",       "id": "VisualStudio/16.6.0+30114.105",       "localBuild": "build-lab",       "manifestName": "VisualStudio",       "manifestType": "installer",       "productDisplayVersion": "16.6.0",       "productLine": "Dev16",       "productLineVersion": "2019",       "productMilestone": "RTW",       "productMilestoneIsPreRelease": "False",       "productName": "Visual Studio",       "productPatchVersion": "0",       "productPreReleaseMilestoneSuffix": "7.0",       "productSemanticVersion": "16.6.0+30114.105",       "requiredEngineVersion": "2.6.2108.39667"     },     "properties": {       "campaignId": "",       "channelManifestId": "VisualStudio.16.Release/16.6.0+30114.105",       "nickname": "",       "setupEngineFilePath": "C:\\Program Files (x86)\\Microsoft Visual Studio\\Installer\\vs_installershell.exe"     }   } ]

May 22 '20 01:05 bryanwweber

Just as a "note to ourselves" - this failure isn't a surprise, as things are currently constituted. vswhere only gives us the top level information - the Visual Studio version (15.x for 2017, 16.x for 2019) - as you might expect from the name (short for visual studio where, I assume). It doesn't give us the Visual C++ compilers available under either of those. In fact, we have a mapping table that goes the opposite direction:

_VCVER_TO_VSWHERE_VER = {
    '14.2': '[16.0, 17.0)',
    '14.1': '[15.0, 16.0)',
}

So when asking for 14.1, we use the table to ask vswhere for Visual Studio 2017 aka 15.x. We're clearly going to miss 14.1 if it only lives under the 2019 product.

May 24 '20 15:05 mwichmann

Just going to dump some research data here. On a setup with Build Tools 19 installed, we start with the 14.26 compiler. Used the setup tool to add 14.25 and 14.16 compilers to this. The list of cl.exe files now found (this is not saying anything different from the original note):

Microsoft Visual Studio/2019/BuildTools/VC/Tools/MSVC/14.16.27023/bin/HostX64/x64/cl.exe
Microsoft Visual Studio/2019/BuildTools/VC/Tools/MSVC/14.16.27023/bin/HostX64/x86/cl.exe
Microsoft Visual Studio/2019/BuildTools/VC/Tools/MSVC/14.16.27023/bin/HostX86/x64/cl.exe
Microsoft Visual Studio/2019/BuildTools/VC/Tools/MSVC/14.16.27023/bin/HostX86/x86/cl.exe
Microsoft Visual Studio/2019/BuildTools/VC/Tools/MSVC/14.25.28610/bin/HostX64/x64/cl.exe
Microsoft Visual Studio/2019/BuildTools/VC/Tools/MSVC/14.25.28610/bin/HostX64/x86/cl.exe
Microsoft Visual Studio/2019/BuildTools/VC/Tools/MSVC/14.25.28610/bin/HostX86/x64/cl.exe
Microsoft Visual Studio/2019/BuildTools/VC/Tools/MSVC/14.25.28610/bin/HostX86/x86/cl.exe
Microsoft Visual Studio/2019/BuildTools/VC/Tools/MSVC/14.26.28801/bin/Hostx64/x64/cl.exe
Microsoft Visual Studio/2019/BuildTools/VC/Tools/MSVC/14.26.28801/bin/Hostx64/x86/cl.exe
Microsoft Visual Studio/2019/BuildTools/VC/Tools/MSVC/14.26.28801/bin/Hostx86/x64/cl.exe
Microsoft Visual Studio/2019/BuildTools/VC/Tools/MSVC/14.26.28801/bin/Hostx86/x86/cl.exe

May 29 '20 19:05 mwichmann

Okay, I have a simple enough bit of code which detects the msvc versions installed under a given product, but the structure of vc.py is such that that information isn't widely usable - the place that makes use of the information that there's a cl.exe found doesn't share that information, it just answers True/False. I'll leave it here in case anyone else feels like running with this...

def _findall_vctools_files(startpath):
    """Extract versions of msvc available for a given product

    Given a path to a 2017-or-later install (where multi-version was enabled),
    look for Microsoft.VCToolsVersion*.txt files and extract the version
    string from them (we could also parse the .props file but this seems easier)

    Args:
        startpath: starting path

    Returns:
        list: the versions found, sorted in descending numerical order.

    """
    import glob

    vcversions = []
    for vfile in glob.glob(r'%s/**/Microsoft.VCToolsVersion*txt' % startpath, recursive=True):
        with open(vfile, 'r') as f:
            vcversions.append(f.readlines()[0].strip())
    vcversions = sorted(set(vcversions), reverse=True)  # trim any dupes
    debug('_findall_vctools_files(): located %s' % str(vcversions))
    return vcversions

May 31 '20 15:05 mwichmann

Let me know if I can help provide any information or test anything! Thanks 😃

Jun 01 '20 20:06 bryanwweber

Might be a bit... trying to think about how to set this up.

Jun 02 '20 12:06 mwichmann

Just another note to ourselves. Informational only.

This appears to be a known problem that falls through the cracks between the VC++ development team and vswhere.

There was an issue filed for vswhere concerning this very problem that appears to have been opened and closed on March 30, 2020. The same issue was reported to the VC++ team much earlier in October 2019.

Of interest is the distinction between a "product" and a "feature":

vswhere is for finding VS and other products. Features within those products are up to the feature teams to support. For VS2019 the VC team made a version-compatible way of finding it for future toolsets they should maintain. That doesn't apply to older versions.

Please work with the VC++ team on how to properly locate older versions. See our wiki as well since I wrote some examples based on their recommendations.

Historical reference:

March 30, 2020: https://github.com/microsoft/vswhere/issues/217
April 2, 2020: https://developercommunity.visualstudio.com/content/problem/975002/cannot-use-vswhere-to-setup-a-vs-2017-that-was-ins.html
October13, 2019: https://developercommunity.visualstudio.com/content/problem/775038/visual-studio-2017-build-tools-not-installed.html

Jun 09 '20 09:06 jcbrill

After stepping through the logic in vc.py, it looks theoretically possible that if you have a batch file which sets up the environment correctly for the compiler you want to use, you can pass the path to it in env['MSVC_USE_SCRIPT'] and have SCons use that, bypassing a bunch of the detection logic (this is in the manpage). It might be necessary to craft that batch file yourself, i.e. it's not certain that a 14.1 under VS2019 actually provides the right one. Won't know until it's tried, it was apparently intended that way but it's also possible that a bunch of subsequent wiring and duct taping might cause it to no longer work.

Jun 09 '20 16:06 mwichmann

From the vswhere issue listed above. Sugguestion that the following would work

"C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\VC\Auxiliary\Build\vcvarsall.bat" x64 -vcvars_ver=14.16

Jun 09 '20 16:06 bdbaddog

FYI, the MSVS batch file code for processing the vcvars_ver argument is:

@REM Support the VS 2015 Visual C++ Toolset
if "%__VCVARS_VERSION%" == "14.0" (
    goto :vcvars140_version
)

@REM If VCVARS_VERSION was not specified, then default initialize the environment
if "%__VCVARS_VERSION%" == "" (
    goto :check_platform
)

:check_vcvars_ver_exists
@REM If we've reached this point, we've detected an override of the toolset version.

@REM Check if full version was provided and the target directory exists. If so, we can proceed to environment setup.
if EXIST "%VSINSTALLDIR%\VC\Tools\MSVC\%__VCVARS_VERSION%" (
    goto :check_platform
)

@REM Check if a partial version was provided (e.g. MAJOR.MINOR only).  In this case,
@REM select the first directory we find that matches that prefix.
set __VCVARS_VER_TMP=
setlocal enableDelayedExpansion
for /F %%a IN ('dir "%VSINSTALLDIR%\VC\Tools\MSVC\" /b /ad-h /o-n') DO (
    set __VCVARS_DIR=%%a
    set __VCVARS_DIR_REP=!__VCVARS_DIR:%__VCVARS_VERSION%=_vcvars_found!
    if "!__VCVARS_DIR!" NEQ "!__VCVARS_DIR_REP!" (
        set "__VCVARS_VER_TMP=!__VCVARS_DIR!"
        goto :check_vcvars_ver_exists_end
    )
)
:check_vcvars_ver_exists_end

If a full version number (e.g., "14.NN.NNNNN") is not passed via vcvars_ver, an attempt is made to make a partial match of the user's version number and the folder names under VC\Tools\MSVC.

The vc\tools\msvc directory contents are sorted by reverse name order. If the user's vcvars_ver argument matches ANY part of the folder name then it is considered to found.

In Bill's example above, using "-vcvars_ver=14.1" should work just as well and will match the "largest" version of 14.1X installed (i.e., "14.1X.nnnnn").

Due to the code above, caution should be exercised when passing a version number as a mistake may very well match: "-vcvars_ver=26" will match "14.26.28801".

Jun 09 '20 17:06 jcbrill

I have not had time to test the following yet.

A quick-and-dirty hack would be to add an environment setting (e.g., "MSVC_VCVARS_VER") that will be passed to the vcvarsall batch file via "-vcvars_ver" argument.

In the msvc_find_valid_batch_script method in vc.py , an argument list is already being constructed. It might be possible to add the "-vcvars_ver" argument similar to the following:

    # Get just version numbers
    maj, min = msvc_version_to_maj_min(version)
    # VS2015+
    if maj >= 14:
        if env.get('MSVC_UWP_APP') == '1':
            # Initialize environment variables with store/universal paths
            arg += ' store'
    # VS2017+
    if maj >= 14.1:
        vcvars_ver = env.get('MSVC_VCVARS_VER')
        if vcvars_ver:
            vcvars_ver = vcvars_ver.strip()
            if vcvars_ver:
                arg += ' -vcvars_ver=' + vcvars_ver

Usage would be something along the lines of:

Environment(MSVC_VERSION="14.2", MSVC_VCVARS_VER="14.1")

If there is a problem with user's version string, the batch file execution should fail.

While this is quick-and-dirty and may work, there may be deeper issues involving the cache.

In the long run, a second variable may be necessary: MSVC_VERSION should be the installed version (e.g., 14.2, 14.1, etc) and the second variable should be a proper prefix of the msvc tool folder name. This would allow any tool version to be used and mirror the behavior of the MS batch files.

A user that has both 2017 and 2019 installed, may desire to use 2017 rather than the 2019 build tools for 2017. In this case specifying a MSVC_VERSION of "14.1" would mean the 2017 installation and not the 2019 installation even if the 2017 tools are installed.

Jun 09 '20 18:06 jcbrill

Proof-of-concept code for supporting specific msvc toolset versions is shown below and attached. Testing has been limited.

An environment variable "MSVC_SPECIFIC_VERSION" is used to choose a specific toolset for a given "MSVC_VERSION".

The motivating example from above would be specified as follows for MSVC 2019:

env = Environment(MSVC_VERSION="14.2", MSVC_SPECIFIC_VERSION="14.1")
env.Program("hello", "hello.cpp")

A patch of the modified version of vc.py from the scons master is attached in scons-master-msvcver-patch.zip.

The modified version of vc.py from the scons master is attached in scons-master-msvcver-souce.zip along with a test folder containing a minimal SConstruct file used for testing.

The implementation:

Prior to the existing code determining the default version to use for 2017 and later, the msvc installation folders are processed and an ordered list of toolset versions for each (host, target) pair is saved in a dictionary.
When the "MSVC_SPECIFIC_VERSION" variable is defined for 2017 and later, the toolset version dictionary is searched for a match. The specific version must be a proper prefix of the toolset version (i.e., "startswith"). If a match is found, the formatted "-vcvars_ver=XX.XX.XXXXX" for the full version is returned and added to the batch file arguments. There is a special shortcut for "14.0" (2015).
If the "MSVC_SPECIFIC_VERSION" variable is not defined or is empty, None is returned, and no argument is added for the specific version.
The most likely place for failure is the host folder name mapping.
While the changes were made against the master, it could probably be patched into the current production version without much effort.

New code:

# Specific toolset version support for 14.1 (VS2017) and later
# Store toolset version list by (host,target) for each _VCVER version > 14
_VCVER_SPECIFIC_VERSIONS = {}

# Map MSVC host/target directory names to (host,target) specification names
_HOST_DIRNAME_TO_HOST_TARGETNAME_GREATER_THAN_14 = {}

# Mapping of directory names to specification names done once during initialization
__SETUP_HOST_DIRNAME_TO_HOST_TARGETNAME_GREATER_THAN_14 = False

def _find_all_cl_in_vc_dir(env, vc_dir, msvc_version):
    """
    Find the locations of all cl.exe installed by Visual Studio/VCExpress.

    For versions 2017 and later, store individual toolset versions for each
    (host, target) pair discovered.  Save the vcvars_ver argument as well.
    """

    global __SETUP_HOST_DIRNAME_TO_HOST_TARGETNAME_GREATER_THAN_14

    ver_num = float(get_msvc_version_numeric(msvc_version))

    if ver_num > 14:

        if not __SETUP_HOST_DIRNAME_TO_HOST_TARGETNAME_GREATER_THAN_14:
            # map msvc host/target directory names to (host,target) specification names
            for vc_target_pair, vc_dir_pair inHOST_TARGET_TO_CL_DIR_GREATER_THAN_14.items():
                # case insensitive store and lookup
                _HOST_DIRNAME_TO_HOST_TARGETNAME_GREATER_THAN_14[vc_dir_pair[0].lower()] = vc_target_pair[0]
                _HOST_DIRNAME_TO_HOST_TARGETNAME_GREATER_THAN_14[vc_dir_pair[1].lower()] = vc_target_pair[1]
            __SETUP_HOST_DIRNAME_TO_HOST_TARGETNAME_GREATER_THAN_14 = True

        vc_tool_d = {}

        # expected folder structure: VC_DIR\Tools\MSVC\XX.XX.XXXXX\bin\HOST\TARGET\cl.exe

        vc_msvc_path = os.path.join(vc_dir, 'Tools', 'MSVC')
        for vc_specific_version in os.listdir(vc_msvc_path):

            vc_tool_path = os.path.join(vc_msvc_path, vc_specific_version)
            if not os.path.isdir(vc_tool_path): continue

            # folder structure: VC_DIR\Tools\MSVC\XX.XX.XXXXX
            debug('vc_specific_version: ' + vc_specific_version)

            vc_bin_path = os.path.join(vc_tool_path, 'bin')
            if not os.path.exists(vc_bin_path): continue

            for vc_host_dir in os.listdir(vc_bin_path):

                vc_host_path = os.path.join(vc_bin_path, vc_host_dir)
                if not os.path.isdir(vc_host_path): continue

                # folder structure: VC_DIR\Tools\MSVC\XX.XX.XXXXX\bin\HOST
                debug('vc_host_dir: ' + vc_host_dir)

                for vc_target_dir in os.listdir(vc_host_path):

                    vc_target_path = os.path.join(vc_host_path, vc_target_dir)
                    if not os.path.isdir(vc_target_path): continue

                    # folder structure: VC_DIR\Tools\MSVC\XX.XX.XXXXX\bin\HOST\TARGET
                    debug('vc_target_dir: ' + vc_target_dir)

                    cl_tool_path = os.path.join(vc_target_path, _CL_EXE_NAME)
                    if os.path.exists(cl_tool_path):

                        # map directory names to specication names
                        vc_host_target_name = _HOST_DIRNAME_TO_HOST_TARGETNAME_GREATER_THAN_14[vc_host_dir.lower()]
                        vc_target_target_name = _HOST_DIRNAME_TO_HOST_TARGETNAME_GREATER_THAN_14[vc_target_dir.lower()]

                        # folder structure: VC_DIR\Tools\MSVC\XX.XX.XXXXX\bin\HOST\TARGET\cl.exe
                        debug('found cl.exe: %s\\%s\\%s (%s, %s, %s)' % (
                            vc_specific_version, vc_host_dir, vc_target_dir, 
                            vc_specific_version, vc_host_target_name, vc_target_target_name)
                        )

                        # register specific version for target specification (host,target)
                        key = (vc_host_target_name, vc_target_target_name)
                        payload = (vc_specific_version, ' -vcvars_ver=' + vc_specific_version)

                        vc_tool_d.setdefault(key,[]).append(payload)

        # sort version number lists in descending order
        for vc_target_spec in vc_tool_d.keys():
            vc_tool_d[vc_target_spec] = sorted(vc_tool_d[vc_target_spec], key = lambda x: x[0], reverse=True)

        # register tool versions dictionary
        _VCVER_SPECIFIC_VERSIONS[msvc_version] = vc_tool_d

    return None

def _find_specific_version(env, msvc_version, vc_host, vc_target):
    """
    Find a specific toolset version.

    Returns vcvars_ver argument string or None
    """

    vc_specific_version = env.get("MSVC_SPECIFIC_VERSION", "").strip()
    if not vc_specific_version:
        return None

    if vc_specific_version == "14.0":
        # "14.0" is handled as a special case in vcvars.bat
        vcvars_ver = ' -vcvars_ver=' + vc_specific_version
        debug('vcvars_ver match found:%s %s' % (vc_specific_version, vcvars_ver))
        return vcvars_ver

    try:
        vc_tool_d = _VCVER_SPECIFIC_VERSIONS[msvc_version]
    except KeyError:
        debug('_VCVER_SPECIFIC_VERSIONS lookup failed:%s' % msvc_version)
        return None

    try:
        vc_tool_list = vc_tool_d[(vc_host, vc_target)]
    except KeyError:
        debug('vc_tool_d lookup failed:%s (%s,%s)' % (msvc_version, vc_host, vc_target))
        return None

    for version_t in vc_tool_list:
        specific_version, vcvars_ver = version_t
        if specific_version.startswith(vc_specific_version):
            debug('vcvars_ver match found:%s %s' % (vc_specific_version, vcvars_ver))
            return vcvars_ver

    debug('vcvars_ver lookup failed:%s (%s,%s) version %s' % (msvc_version, vc_host, vc_target, vc_specific_version))
    return None

Hook into _check_cl_exists_in_vc_dir(env, vc_dir, msvc_version):

    # make sure the cl.exe exists meaning the tool is installed
    if ver_num > 14:
        # find all vc toolset versions for 2017 and later
        _find_all_cl_in_vc_dir(env, vc_dir, msvc_version)

        # 2017 and newer allowed multiple versions of the VC toolset to be
        # installed at the same time. This changes the layout.
        # Just get the default tool version for now
        #TODO: support setting a specific minor VC version
        default_toolset_file = os.path.join(vc_dir, _VC_TOOLS_VERSION_FILE)

Hook into def msvc_find_valid_batch_script(env, version):

            # Get just version numbers
            maj, min = msvc_version_to_maj_min(version)
            # VS2015+
            if maj >= 14:
                if env.get('MSVC_UWP_APP') == '1':
                    # Initialize environment variables with store/UWP paths
                    arg = (arg + ' store').lstrip()
            # VS2017+
            if maj >= 14 and min >= 1:
                vcvars_ver = _find_specific_version(env, version, host_platform, tp)
                if vcvars_ver:
                    arg = (arg + vcvars_ver).lstrip()

scons-master-msvcver-patch.zip scons-master-msvcver-source.zip

Jun 10 '20 18:06 jcbrill

@jcbrill - please make a PR. large patches in issues are hard to manage/comment on/etc.. We can refine and/or at least pull your changes from your branch and continue work on them much more easily from a PR.

Jun 11 '20 06:06 bdbaddog

We might want to resurrect the MSVS_VERSION var, which was deprecated (right now forced to be the same as MSVC_VERSION).

Jun 11 '20 15:06 mwichmann

Are there legacy issues with existing SConstruct/SConscript files if MSVS_VERSION is resurrected and the definition of MSVC_VERSION changes/expands?

Jun 11 '20 16:06 jcbrill

Honestly, no clue. It feels like letting MSVC_VERSION also take a more specific version string wouldn't hurt anything, here's the current claim:

MSVC_VERSION
    Valid values for Windows are 14.2, 14.1, 14.1Exp, 14.0, 14.0Exp, 12.0, 12.0Exp, 11.0, 11.0Exp, 10.0, 10.0Exp, 9.0, 9.0Exp, 8.0, 8.0Exp, 7.1, 7.0, and 6.0.

What turning MSVS_VERSION back to relevant and allowed to specify a required VS version would do - it might affect things. I think it might still have a meaning in the vs module, the one that is used to generate project files but not actually hunt for usable compilers.

It was just a thought :)

Jun 11 '20 17:06 mwichmann

Note to self (or rather to create a github xref)... tried to capture the detection flow in a README file added by #3690 (unmerged as of this writing). Would want to update that if the above changes are added.

Jun 11 '20 17:06 mwichmann

A challenge with the current MSVC_VERSION definition is that "14.2", "14.1" and "14.1Exp" describe an msvs installation rather than an msvc toolset (14.16.27023) which can be present in all three.

For a user with all three versions (2019, 2017, 2017Exp) installed, an msvc version of 14.16.27023 is not unique. If a user specifies 14.16, should SCons use the 14.1, 14.1Exp, or 14.2 toolset?

My guess is that some users will want to explicitly specify the msvs installation and toolset.

Perhaps when migrating from an older version of msvs to a newer version of msvs and making sure the build processes are identical. Or perhaps a client/customer only has the express version installed and a developer would like to make sure there are no build issues between the two.

Still thinking about the problem...

Jun 11 '20 17:06 jcbrill

I apologize in advance as I am time limited for the rest of the day today and tomorrow morning.

Notes to self while it is still fresh that a bug report that needs to be issued.

During initial testing of the proposed changes above, and after installing 14.1Exp, a few issues were discovered.

There are at least three problems with the msvc detection in the master and one of them is in 3.1.2:

"14.1Exp" was added to the _VCVER list but was not added to the _VCVER_TO_VSWHERE_VER dictionary. This causes a key error when trying to look up the vswhere_ver and exits without finding the express version. Extending the dictionary results in the problem in the next item.
When both 14.1 and 14.1Exp are installed, the current query for vswhere is unreliable. The express version is returned for both 14.1 and 14.1Exp. The first result returned is not necessarily the desired result.

The problem has to to with using "-products * -version [15.0,16.0)" in the query which finds both the full version and the express version. With only -version [15.0, 16.0) the full version is found. With -products Microsoft.VisualStudio.Product.WDExpress the express version is found.

One solution is fairly straightforward.

_VCVER_TO_VSWHERE_VER = {
   '14.2'    : [ '-version', '[16.0, 17.0)' ],
   '14.1'    : [ '-version', '[15.0, 16.0)' ],
   '14.1Exp' : [ '-products', 'Microsoft.VisualStudio.Product.WDExpress' ],
}

...

   vswhere_cmd = [vswhere_path] + vswhere_ver + ['-property', 'installationPath']

MSVC 6.0 is not detected due to the directory walk and checking if "cl.exe" exists in the file list. Believe it or not, in msvc 6.0, the filename is "CL.EXE". This is in 3.1.2 and master.

A quick test for cl.exe existing in the bin folder (i.e., under vc_dir) prior to the directory walk is successful for 7.1, 7.0, and 6.0. If that ever fails, the case-sensitive comparison for 6.0 is a problem.

        # quick test for bin directory
        cl_path = os.path.join(vc_dir, 'bin', _CL_EXE_NAME)
        if os.path.exists(cl_path):
            debug('get_installed_vcs ' + _CL_EXE_NAME + ' found %s' % cl_path)
            return True

This is somewhat embarrassing: as recently as last week I thought I was doing c preprocessor tests against 6.0.

Vintage 32-bit windows 7 test box (circa 2007) with the following now discovered by locally "modified" Scons:

installed_vcs:['14.2', '14.1', '14.1Exp', '14.0', '12.0', '11.0', '11.0Exp', '10.0', '9.0', '8.0', '7.1', '7.0', '6.0']

There are no boxes here with git installed nor anyone with git experience. Minor patches for the above can be provided quickly. In the short term, pull request(s) a little longer...

Jun 11 '20 22:06 jcbrill

The issue with MSVC 6.0 mentioned above is a false negative:

The initial code to find installed vcs (get_installed_vcs) fails to find msvc 6.0 due to the case sensitive name test (cl.exe == CL.EXE). The debug logging shows that no compiler is found for 6.0 and it does not show up in the installed versions list.
However, when msvc_find_valid_batch_script is called later, it correctly finds the msvc 6.0 bin\vcvars32.bat file.
The diagnostic for the installed version check indicates that msvc 6 is not installed but the build process will use actually use msvc 6.

As described above, 14.1 Express will not work as intended in the master and the obvious fix will lead to a less-obvious problem with the vswhere arguments.

The introduction of 14.1 Express also makes the original issue report a little more thorny as now there are at least three msvs installations that support the 14.1 toolset.

@bdbaddog or @mwichmann Should the 14.1 Express and msvc 6 false negative issues go through the mailing list or is it OK to file an issue directly?

Joe

Jun 12 '20 14:06 jcbrill

For me, the 14.1Express support was a recent addition, and so "not fully debugged" isn't that surprising. My tests were on a VM that has only express, so I wouldn't have seen the overlap problems. Seems like including fixes for that as part of the other "finding" work is fine - but will wait for @bdbaddog to weigh in. I don't have disk space to test more combinations - my native Windows box is full (and old/slow), and the drive holding the VMs is also full - including 1TB of Windows VMs that do nothing but test scons combinations :)

Jun 12 '20 14:06 mwichmann

I have attached a second prototype of specific version support. This is pre-alpha quality lacking adequate testing, documentation and error checking. It may very well only work on my computer.

This prototype accepts a specific toolset version via the MSVC_VERSION variable. Internally, the specific version is converted to the existing _VCVER format for the existing code. This very well may break code outside of vc.py (e.g., test code) expecting the current definitions.

Probably a better implementation is to define an internal-user variable with the specific version.

An additional variable was added "MSVC_PRODUCT" to specify the installed version when there are choices. For example, when 14.1 and 14.2 are both installed, a MSVC_VERSION="14.1" will use the 14.1 installation. With MSVC_VERSION="14.1" and MSVC_PRODUCT="14.2", the "14.2" installation will use the 14.16 tools. This is a bit tricky as early on the MSVC_VERSION need to be "rewritten".

This version will also use installed 14.2 for MSVC_VERSION="14.1" when there are no instances of 14.1 installed which is the point of this current issue.

The desire to expand the definition of MSVC_VERSION has been satisfied. Trying to adhere to the "Law of Least Astonishement", if the product for the MSVC_VERSION is installed that will be used. It can be over-ridden via the MSVC_PRODUCT variable.

Due to the nature of the implementation, there is experimental support for choosing the product installation type: Ent (Enterprise), Pro (Professional), Com (Community), Exp (Express), and BT (BuildTools). Examples are shown below.

vswhere is run once for all installations and the json output is processed for fields of interest. Unfortunately, since this is done so early I don't believe a user's vswhere variable will have any effect.

Three exceptions were added and raised for invalid data. As the specific version information is passed as an argument to the vcvars batch files, ignoring any errors can/will result in a successful compilation with possibly a different (e.g., default) toolset than the user specification. For now, the exceptions have been useful in identifying corner cases.

Support for 14.0 with the later tools is precarious at best. The Express version of 14.1 makes the problem more annoyingly difficult.

scons-master-msvcver-source-2.zip

Example test configurations:

env = Environment() # 14.2 Community

# 14.2 Installed [Community]

#env = Environment(MSVC_VERSION="14.2")        # 14.2 Community
#env = Environment(MSVC_VERSION="14.26")       # 14.2 Community
#env = Environment(MSVC_VERSION="14.26.28801") # 14.2 Community
#env = Environment(MSVC_VERSION="14.25")       # 14.2 Community
#env = Environment(MSVC_VERSION="14.25.28610") # 14.2 Community
#env = Environment(MSVC_VERSION="14.20")       # 14.2 Community
#env = Environment(MSVC_VERSION="14.20.27508") # 14.2 Community
#env = Environment(MSVC_VERSION="14.27")       # 14.2 Community: MSVCProductNotFound exception

# 14.1 [Fake] Not Installed [Community or Express]

#env = Environment(MSVC_VERSION="14.1")        # 14.2 Community
#env = Environment(MSVC_VERSION="14.16")       # 14.2 Community
#env = Environment(MSVC_VERSION="14.16.27")    # 14.2 Community
#env = Environment(MSVC_VERSION="14.16.27023") # 14.2 Community
#env = Environment(MSVC_VERSION="14.16.27024") # 14.2 Community: MSVCProductNotFound exception

# 14.1 Installed [Community, Express]

#env = Environment(MSVC_VERSION="14.1")        # 14.1 Community [Ent->Pro->Com->Exp->BT]
#env = Environment(MSVC_VERSION="14.16")       # 14.1 Community
#env = Environment(MSVC_VERSION="14.16.27")    # 14.1 Community
#env = Environment(MSVC_VERSION="14.16.27023") # 14.1 Community
#env = Environment(MSVC_VERSION="14.15")       # 14.1 Community
#env = Environment(MSVC_VERSION="14.15.26726") # 14.1 Community
#env = Environment(MSVC_VERSION="14.14")       # 14.1 Community
#env = Environment(MSVC_VERSION="14.14.26428") # 14.1 Community
#env = Environment(MSVC_VERSION="14.16.27024") # 14.1 Community: MSVCProductNotFound exception

#env = Environment(MSVC_VERSION="14.1", MSVC_PRODUCT="14.1")    # 14.1 Community
#env = Environment(MSVC_VERSION="14.1", MSVC_PRODUCT="14.1Exp") # 14.1 Express
#env = Environment(MSVC_VERSION="14.1", MSVC_PRODUCT="14.1Com") # 14.1 Community
#env = Environment(MSVC_VERSION="14.1", MSVC_PRODUCT="14.1Ent") # 14.1 Enterprise:   MSVCProductNotFound exception
#env = Environment(MSVC_VERSION="14.1", MSVC_PRODUCT="14.1Pro") # 14.1 Professional: MSVCProductNotFound exception
#env = Environment(MSVC_VERSION="14.1", MSVC_PRODUCT="14.1BT")  # 14.1 BuildTools:   MSVCProductNotFound exception

#env = Environment(MSVC_VERSION="14.1", MSVC_PRODUCT="14.2")    # 14.2 Community
#env = Environment(MSVC_VERSION="14.1", MSVC_PRODUCT="14.2Com") # 14.2 Community

#env = Environment(MSVC_VERSION="14.16",       MSVC_PRODUCT="14.2") # 14.2 Community
#env = Environment(MSVC_VERSION="14.16.27",    MSVC_PRODUCT="14.2") # 14.2 Community
#env = Environment(MSVC_VERSION="14.16.27023", MSVC_PRODUCT="14.2") # 14.2 Community
#env = Environment(MSVC_VERSION="14.16.27024", MSVC_PRODUCT="14.2") # 14.2 Community: MSVCProductNotFound exception

# 14.1 Express

#env = Environment(MSVC_VERSION="14.1Exp")         # 14.1 Express
#env = Environment(MSVC_VERSION="14.16Exp")        # 14.1 Express
#env = Environment(MSVC_VERSION="14.16.27Exp")     # 14.1 Express
#env = Environment(MSVC_VERSION="14.16.27023Exp")  # 14.1 Express
#env = Environment(MSVC_VERSION="14.16.27024Exp")  # 14.1 Express: MSVCProductNotFound exception

# 14.0 [Fake] Not Installed

#env = Environment(MSVC_VERSION="14.0")                         # 14.2 Community -> 14.0
#env = Environment(MSVC_VERSION="14.0", MSVC_PRODUCT="14.2")    # 14.2 Community -> 14.0
#env = Environment(MSVC_VERSION="14.0", MSVC_PRODUCT="14.2Com") # 14.2 Community -> 14.0
#env = Environment(MSVC_VERSION="14.0", MSVC_PRODUCT="14.1")    # 14.1 Community -> 14.0
#env = Environment(MSVC_VERSION="14.0", MSVC_PRODUCT="14.1Com") # 14.1 Community -> 14.0
#env = Environment(MSVC_VERSION="14.0", MSVC_PRODUCT="14.1Exp") # 14.1 Express -> 14.0

# 14.0 Installed

#env = Environment(MSVC_VERSION="14.0")                         # 14.0 Community
#env = Environment(MSVC_VERSION="14.0", MSVC_PRODUCT="14.0")    # 14.0 Community
#env = Environment(MSVC_VERSION="14.0", MSVC_PRODUCT="14.0Exp") # 14.0 Community: MSVCProductInvalid [conflict]
#env = Environment(MSVC_VERSION="14.0", MSVC_PRODUCT="14.1")    # 14.1 Community -> 14.0
#env = Environment(MSVC_VERSION="14.0", MSVC_PRODUCT="14.1Com") # 14.1 Community -> 14.0
#env = Environment(MSVC_VERSION="14.0", MSVC_PRODUCT="14.1Exp") # 14.1 Express -> 14.0
#env = Environment(MSVC_VERSION="14.0", MSVC_PRODUCT="14.2")    # 14.2 Community -> 14.0
#env = Environment(MSVC_VERSION="14.0", MSVC_PRODUCT="14.2Com") # 14.2 Community -> 14.0

#env = Environment(MSVC_VERSION="14.0", MSVC_PRODUCT="14.1Ent") # 14.1 Enterprise: MSVCProductNotFound exception 
#env = Environment(MSVC_VERSION="14.0", MSVC_PRODUCT="14.1Pro") # 14.1 Professional: MSVCProductNotFound exception
#env = Environment(MSVC_VERSION="14.0", MSVC_PRODUCT="14.1BT")  # 14.1 BuildTools: MSVCProductNotFound exception

#env = Environment(MSVC_VERSION="14.0", MSVC_PRODUCT="14.2Ent") # 14.2 Enterprise: MSVCProductNotFound exception 
#env = Environment(MSVC_VERSION="14.0", MSVC_PRODUCT="14.2Pro") # 14.2 Professional: MSVCProductNotFound exception
#env = Environment(MSVC_VERSION="14.0", MSVC_PRODUCT="14.2BT")  # 14.2 BuildTools: MSVCProductNotFound exception

Jun 14 '20 15:06 jcbrill

Once the 2019 build tools were actually installed, a problem with identifying "synthetic" versions (e.g., Enterprise, Professional, Community, BuildTools) when there are multiple versions installed was revealed.

The example configurations below actually address the problem in the original post.

The MSVC_PRODUCT="14.2BT" may not be necessary if there is only one instance of 14.2. This has not been tested yet.

Will move the source to git soon ...

scons-master-msvcver-source-3.zip

Example test configurations:

env = Environment(MSVC_VERSION="14.1", MSVC_PRODUCT="14.2BT") # 14.2 BuildTools

#env = Environment(MSVC_VERSION="14.2",        MSVC_PRODUCT="14.2BT") # 14.2 BuildTools
#env = Environment(MSVC_VERSION="14.26",       MSVC_PRODUCT="14.2BT") # 14.2 BuildTools
#env = Environment(MSVC_VERSION="14.26.28801", MSVC_PRODUCT="14.2BT") # 14.2 BuildTools
#env = Environment(MSVC_VERSION="14.25",       MSVC_PRODUCT="14.2BT") # 14.2 BuildTools
#env = Environment(MSVC_VERSION="14.25.28610", MSVC_PRODUCT="14.2BT") # 14.2 BuildTools
#env = Environment(MSVC_VERSION="14.1",        MSVC_PRODUCT="14.2BT") # 14.2 BuildTools
#env = Environment(MSVC_VERSION="14.16",       MSVC_PRODUCT="14.2BT") # 14.2 BuildTools
#env = Environment(MSVC_VERSION="14.16.27023", MSVC_PRODUCT="14.2BT") # 14.2 BuildTools
#env = Environment(MSVC_VERSION="14.0",        MSVC_PRODUCT="14.2BT") # 14.2 BuildTools -> 14.0

Jun 14 '20 21:06 jcbrill

I have a "working" prototype version of vc.py that unifies both #3265 and #3664.

The prototype is not production ready but does illustrate that both #3265 and #3664 can be addressed with the same implementation. There is a great deal of new code that needs to be fully tested, documented, and modified based on feedback. The ramifications to code outside of vc.py are unknown.

If desired, A PR can be issued to move this discussion. The reality is that integration is likely a ways off in the future.

#3265 is an enhancement to allow a specific version of an msvc toolset. In #3265, there was a suggestion to allow specifying a specific version via the existing environment variable MSVC_VERSION.

#3664 (this issue) involves detecting a toolset that is installed in a later product version (i.e., the 14.16 toolset via the 14.2 product line). In this case, the 14.1 toolset via the 14.2 Build Tools.

The prototype implementation is "wired-in" to the existing code base in a few select locations. The existing code base continues to operate on an MSVC_VERSION that is compatible with the _VCVER version list (e.g., "14.2"). A single internal environment variable and dictionary was added to maintain the specific version and product state.

There is a single hard-coded variable at the top of the source file that selectively enables/disables the specific version support. When disabled, the functionality should be exactly the same as the current master. The hooks into the existing code were intentionally minimized to ease updating from the master. The bulk of the new code was added at the bottom of the file.

The MSVC_VERSION format was expanded to allow:

specifying an individual toolset at varying degrees of precision:
- "14.1"
- "14.16"
- "14.16.27"
- "14.16.27023"
specifying individual product types (attached to a specific toolset or a product):
- Enterprise: "14.26Ent"
- Professional: "14.2Pro"
- Community: "14.26.27Com"
- Build Tools: "14.2BT"
- Express: "14.1Exp"
a right assignment operator ("->") that maps a specific toolset to a defined product version with an optional product type:
- "14.0->14.2" use the the 14.0 toolset via a 2019 installation
- "14.1->14.2BT" use the 14.1 toolset via a 2019 Build Tools installation
- "14.1->14.2Com" use the 14.1 toolset via a 2019 Community installation

Without a product qualifier (e.g., "BT"), there is a subtle but significant change semantically: the product selected will be the newest/latest toolset that matches the user specification within installed toolsets for a given (host,target) combination. Effectively, without a product qualifier, any product type could be returned based on the installed toolsets. This only affects multiple installations of the same product version (e.g., 14.1 and 14.Exp and/or 14.2Com and 14.2BT).

This means that a user does not need to specify "14.1Exp" explicitly if there is a single 14.1 product installation. On the flip side, it also means that "14.1Exp" may be returned for a "14.1" request. In local testing, "14.1Exp" was returned for an "arm" target. In the local installations of 14.1 Community and 14.1 Express, the "newest" toolset for an arm target was in the 14.1 Express installation as it was installed recently.

With a product qualifier, a specific product version and type is used.

The motivating example for the current issue, is that no 14.1 products are installed and the 14.1 toolset is not found within the 14.2 Build Tools installation. In the prototype implementation, "MSVC_VERSION=14.1" will return an installed version of 14.1 if it exists or an installed version of 14.2 if the 14.16 toolset is installed for the host/target combination.

There are two attachments:

msvc-specific-142.txt contains edited testing output for the following SConstruct specification:
```
# 14.1 is not installed, 14.2 Build Tools is installed
env = Environment(MSVC_VERSION="14.1")
```

msvc-specific-141-142.txt contains edited testing output for the following SConstruct specifications:

# 14.1 Installed [Community, Express], 14.2 Installed [Community, BuildTools]
env1 = Environment(MSVC_VERSION="14.1")
env2 = Environment(MSVC_VERSION="14.16.27Exp") # 14.16.27->14.1Exp
env3 = Environment(MSVC_VERSION="14.1->14.2")
env4 = Environment(MSVC_VERSION="14.1->14.2BT")

The internal version tables (all installed and by version), internal toolset dictionary state at four locations, and select SConstruct environment variables are displayed.

I realize everyone is busy trying get 4.0 out the door...

msvc-specific-142.txt msvc-specific-141-142.txt

Jun 24 '20 17:06 jcbrill

Yes. Please push a PR mark it "[WIP]" in the title.

On Wed, Jun 24, 2020 at 10:49 AM Joseph Brill [email protected] wrote:

I have a "working" prototype version of vc.py that unifies both #3265 https://github.com/SCons/scons/issues/3265 and #3664 https://github.com/SCons/scons/issues/3664.

The prototype is not production ready but does illustrate that both #3265 https://github.com/SCons/scons/issues/3265 and #3664 https://github.com/SCons/scons/issues/3664 can be addressed with the same implementation. There is a great deal of new code that needs to be fully tested, documented, and modified based on feedback. The ramifications to code outside of vc.py are unknown.

If desired, A PR can be issued to move this discussion. The reality is that integration is likely a ways off in the future.

#3265 https://github.com/SCons/scons/issues/3265 is an enhancement to allow a specific version of an msvc toolset. In #3265 https://github.com/SCons/scons/issues/3265, there was a suggestion to allow specifying a specific version via the existing environment variable MSVC_VERSION.

#3664 https://github.com/SCons/scons/issues/3664 (this issue) involves detecting a toolset that is installed in a later product version (i.e., the 14.16 toolset via the 14.2 product line). In this case, the 14.1 toolset via the 14.2 Build Tools.

The prototype implementation is "wired-in" to the existing code base in a few select locations. The existing code base continues to operate on an MSVC_VERSION that is compatible with the _VCVER version list (e.g., "14.2"). A single internal environment variable and dictionary was added to maintain the specific version and product state.

There is a single hard-coded variable at the top of the source file that selectively enables/disables the specific version support. When disabled, the functionality should be exactly the same as the current master. The hooks into the existing code were intentionally minimized to ease updating from the master. The bulk of the new code was added at the bottom of the file.

The MSVC_VERSION format was expanded to allow:

specifying an individual toolset at varying degrees of precision:

"14.1"

"14.16"

"14.16.27"

"14.16.27023"

specifying individual product types (attached to a specific toolset or a product):

Enterprise: "14.26Ent"

Professional: "14.2Pro"

Community: "14.26.27Com"

Build Tools: "14.2BT"

Express: "14.1Exp"

a right assignment operator ("->") that maps a specific toolset to a defined product version with an optional product type:

"14.0->14.2" use the the 14.0 toolset via a 2019 installation

"14.1->14.2BT" use the 14.1 toolset via a 2019 Build Tools installation

"14.1->14.2Com" use the 14.1 toolset via a 2019 Community installation

Without a product qualifier (e.g., "BT"), there is a subtle but significant change semantically: the product selected will be the newest/latest toolset that matches the user specification within installed toolsets for a given (host,target) combination. Effectively, without a product qualifier, any product type could be returned based on the installed toolsets. This only affects multiple installations of the same product version (e.g., 14.1 and 14.Exp and/or 14.2Com and 14.BT).

This means that a user does not need to specify "14.1Exp" explicitly if there is a single 14.1 product installation. On the flip side, it also means that "14.1Exp" may be returned for a "14.1" request. In local testing, "14.1Exp" was returned for an "arm" target. In the local installations of 14.1 Community and 14.1 Express, the "newest" toolset for an arm target was in the 14.1 Express installation as it was installed recently.

With a product qualifier, a specific product version and type is used.

The motivating example for the current issue, is that no 14.1 products are installed and the 14.1 toolset is not found within the 14.2 Build Tools installation. In the prototype implementation, "MSVC_VERSION=14.1" will return an installed version of 14.1 if it exists or an installed version of 14.2 if the 14.16 toolset is installed for the host/target combination.

There are two attachments:

msvc-specific-142.txt contains edited testing output for the following SConstruct specification:

14.1 is not installed, 14.2 Build Tools is installed

env = Environment(MSVC_VERSION="14.1")

msvc-specific-141-142.txt contains edited testing output for the following SConstruct specifications:

14.1 Installed [Community, Express], 14.2 Installed [Community, BuildTools]

env1 = Environment(MSVC_VERSION="14.1") env2 = Environment(MSVC_VERSION="14.16.27Exp") # 14.16.27->14.1Exp env3 = Environment(MSVC_VERSION="14.1->14.2") env4 = Environment(MSVC_VERSION="14.1->14.2BT")

The internal version tables (all installed and by version), internal toolset dictionary state at four locations, and select SConstruct environment variables are displayed.

I realize everyone is busy trying get 4.0 out the door...

msvc-specific-142.txt https://github.com/SCons/scons/files/4827072/msvc-specific-142.txt msvc-specific-141-142.txt https://github.com/SCons/scons/files/4827073/msvc-specific-141-142.txt

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/SCons/scons/issues/3664#issuecomment-648970241, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAUCNB5IL26H5KMOZWX3ZTRYI4DFANCNFSM4NHLQUDA .

Jun 25 '20 13:06 bdbaddog

For posterity.

It has been deemed that this issue is not a bug and is the preferred behavior based on the response in https://github.com/SCons/scons/pull/3717#issuecomment-670822825:

One of the first principles of SCons..

The build should be repeatable regardless of user's environment.

So, if a user requests version X.1, then get version X.1, even if they have version X.2. If X.1 is not available, it should bail out.

We're not going to change a fundamental principle of SCons (reproducible builds) because you don't want to update your build scripts when there's new releases of the tools.

Now, if you were suggesting that we introduced a way for the build system to EXPLICITLY introduce flexibility, that would be acceptable.

But not without explicitly requesting it.

So MSVC_CHOOSE_VERSION(or even MSVC_VERSION if it's a callable) as a python function which gets passed the list of available versions and selects, that'd be fine.

But asking for 14.1 and accepting 14.2... nope.

BTW. I'm totally not saying that what you need SCons to do is unreasonable, just that it changes a fundamental SCons principle, and so wouldn't be merge-able without some explicit way to tell SCons to be flexible.

Curiously, the behavior deemed unacceptable is this reason for this issue and was laid out very clearly above prior to being requested in a PR.

But asking for 14.1 and accepting 14.2... nope.

Which is exactly the "bug" in the current issue.

Despite the fact that MSVC 2017 and MSVC 2019 are toolset oriented (i.e., a MSVC 2019 installation is a container for 14.1X, 14.2X, and to a lesser extent 14.0 toolsets), the desire is that the MSVC detection remain product oriented.

Prior to MSVS 2017, there was a 1:1 mapping between the msvc product and the msvc toolset. The toolset might be upgraded in an update or service pack but there was no way to select a specific toolset as side-by-side installations were not supported.

With MSVC 2017 and MSVC 2019, an SCons request of "14.1" or "14.2" is a toolset request. The current msvc behavior uses the default toolset which is generally the newest installed toolset version when the "--vcvars_ver" argument is not specified for the vcvars batch file.

For example, "14.1" (MSVC 2017) is likely to return "14.16.27023". Effectively, the product request is a toolset request similar to "find the newest toolset version that startswith('14.1')". However, "14.2" (MSVC 2019) will return a toolset based on the last update which may vary across users and computers.

The actual toolset version used for 2017 and 2019 is based on end-user's installation options and frequency of updates. While two different users may be using the same product (e.g., 2019) they may not be using the same toolset. Similarly, the installed toolset version is not necessary the same for all host/target combinations in a given installation.

The current implementation will use a default toolset within a product across different users and environments, which may not be the same, but will not use a toolset across products.

For whatever reason (e.g., enterprise and/or customer requirements), if a build depends on the 14.1X toolset it hardly seems as important in which product container it is installed (given a simple selection rule). The same issue will arise when the next MSVC product is released and end-users desire to use the 14.2X toolset combination.

There are two schools of thought:

If you want to use the 14.1X toolset in any version other that MSVC 2017: a user must specify the product and the toolset. This is the product-centric view.
If you want to use the 14.1X toolset: automatically select MSVC 2017 if it is installed, otherwise select the newest product with the toolset installed. An explicit product specification is optional. This is the toolset-centric view.

For the majority of end-users, there is likely only one msvc installation anyway so there is no ambiguity nor an explicit need to specify a product. This current issue falls into this camp.

A fundamental shift from a product-centric view to a toolset-centric view for the msvc tools would go a long way in supporting the msvc tools in the future and reduce the burden on an end-user.

The point of this particular issue is that a toolset is desired and not a necessarily product/toolset.

The build should be repeatable regardless of user's environment.

If the selected product for the requested toolset is supported by SCons, this principle should hold. The binary tools may be different. The binary tools are likely already different within the same product across users and computers.

It does not seem to be any different than the same build scripts using an installed version of mingw gcc 9.3 on one computer and using an installed version of mingw gcc 8.3 on another computer simply due to a lag in updates.

While this is certainly a noble tenet, in practice I'm not sure this always holds across two or more users in a windows environment or even across two or more windows boxes.

For example, the addition of cygwin to the default tools path caused some local builds to fail. The solution was to explicitly specify the tools used thus bypassing the effect of cygwin being added to the path. However, if the same build scripts were developed on a box without cygwin installed and sent to a customer who had cygwin installed the build would have failed.

Aug 09 '20 11:08 jcbrill

@jcbrill - No it has not been ruled that this is not a bug. If it had been I would have closed it with comments.

Please stop overreacting.

Aug 09 '20 16:08 bdbaddog

@bdbaddog I apologize if you believe that I'm overreacting. It is impossible to tell how much thought you may have put into supporting this issue and specific versions in general from one or two line comments here and there. I am merely trying to document some hard earned insight for your benefit in the future and for others who may attempt a solution.

Aug 09 '20 16:08 jcbrill

@jcbrill - As you're apparently unwilling to join any of the normal forms of discourse for this project, it's difficult to discuss this with you. Yes. I believe you are overreacting. I (and no volunteers here) work on this project 24/7. We do the best we can and are rarely rewarded with thanks and more often with complaints that bugs reported didn't immediately get our undivided attention until they were successfully resolved. Unfortunately that's just not possible. Listing the number of days/weeks an issue has been open is really only fair when you are PAYING for support. Which no-one is for this project.

Aug 09 '20 16:08 bdbaddog

@bdbaddog I understand completely. That is why actual solutions were provided that were worked on full time until completion. To be clear, I am not in need of any fixes, enhancements or reporting bugs. Merely trying to knock a couple off the list. Is there a mechanism where we can discuss this offline?

Aug 09 '20 16:08 jcbrill

@jcbrill - scons-devel mailing list and #devel on our discord server are the typical places. Which I believe I've mentioned before. If not sorry for the oversight.

Aug 09 '20 17:08 bdbaddog