Add handling of static versus dynamic linking of the C RunTime (CRT)

[volks73]

See #114 for the initial information. A Windows built Rust binary (executable) needs a C RunTime (CRT). By default, the CRT is dynamically linked to the executable, which keeps the executable size smaller. As of Rust v1.19, it is possible to statically link the CRT. This results in a slightly larger executable, but eliminates distribution of a dependency and/or eliminates installation of a prerequisite for running the Windows-built Rust binary.

In the case of the default, where the CRT is dynamically linked and not included in the executable, a CRT can be added to the MSI using the Merge WiX Toolset tag, or needs to be installed separately before running the Rust binary on Windows.

In the case of statically linking the CRT with the Rust executable, a rustflag can be defined in a environment variable, project configuration (Cargo.toml) or developer's Cargo configuration.

Adding static linking of the CRT is a simpler solution and recommended, but there are tradeoffs and developers may not be aware of this issue for their users since most Rust developers will have the required CRT already installed. Thus, some details and possible solutions based on these options should be added to the documentation.

A side thought, is it possible to detect from the cargo-metadata crate if static linking of the CRT is enabled? Maybe a warning should be printed if dynamic linking is being used and the CRT is not included in the WXS file? Or, is it possible to detect if dynamic linking is enabled at the time of the cargo wix init command and the template automatically adds the CRT to the installer? This would take the CRT version of the developer's computer. For now, a simple configuration in a project's Cargo.toml file handles all of this, but it is something to think about. I do recognize this is probably what #114 was asking to implement. It just took me a moment.

[roblabla]

A side thought, is it possible to detect from the cargo-metadata crate if static linking of the CRT is enabled?

Unfortunately, no. There are a lot of different ways to enable static linking of the CRT runtime, and cargo-metadata doesn't give us any of this information. What could work is checking the .exes for exports, e.g. parse the PE and check if it requires MSVCRT.dll (or any of its variants. Checking for a dll that starts with MSVCR should work well enough). There are some really good PE parsers (I highly recommend pelite that should make automated checking painless.

The hard part becomes finding which PE to parse. This is information cargo-metadata might be able to give us, but if the user modifies the WXS, we might want to instead parse the WXS to find which binaries are being included?

Or, is it possible to detect if dynamic linking is enabled at the time of the cargo wix init command and the template automatically adds the CRT to the installer?

What if it changes afterwards? What I do think might be a good idea is having a cargo wix init prompt the user if they want a dynamic linking or static linking setup. If the user choses dynamic linking, use a template with the Merge WiX tag, otherwise tell the user to add the appropriate config to the .cargo/config file. This way, the user makes an informed choice instead of using whatever defaults.

[volks73]

Unfortunately, no. There are a lot of different ways to enable static linking of the CRT runtime, and cargo-metadata doesn't give us any of this information.

Does the rustc --print cfg command or eventual --unit-graph feature help us out, again, for this case? A quick test of the output of the rustc --print cfg command:

debug_assertions
target_arch="x86_64"
target_endian="little"
target_env="msvc"
target_family="windows"
target_feature="fxsr"
target_feature="sse"
target_feature="sse2"
target_os="windows"
target_pointer_width="64"
target_vendor="pc"
windows

I see the target_feature="fxsr", target_feature="sse", and target_feature="sse2" line items. I tried,

C:\>$env:RUSTFLAGS="-C target-feature=+crt-static"; rustc --print cfg

But a target_feature=crt-static was not listed as a target feature. I think this could be because the crt-static target feature is only enabled and added to the configuration right before the compile/build, or as usual, I am missing something about the order of operations.

What could work is checking the .exes for exports, e.g. parse the PE and check if it requires MSVCRT.dll (or any of its variants. Checking for a dll that starts with MSVCR should work well enough).

The hard part becomes finding which PE to parse. This is information cargo-metadata might be able to give us, but if the user modifies the WXS, we might want to instead parse the WXS to find which binaries are being included?

Would this be further complicated by the --no-build flag, too?

There are some really good PE parsers (I highly recommend pelite that should make automated checking painless.

I looked at the pelite crate, and that is a pretty neat library. If we go this route, then it looks pretty painless to check with pelite, as @roblabla mentioned.

What if it changes afterwards? What I do think might be a good idea is having a cargo wix init prompt the user if they want a dynamic linking or static linking setup. If the user choses dynamic linking, use a template with the Merge WiX tag, otherwise tell the user to add the appropriate config to the .cargo/config file. This way, the user makes an informed choice instead of using whatever defaults.

I would like to avoid Q&A prompts in the CLI because no other feature within cargo-wix does this...yet, but flags or options would work.

I am a little lost in the order of operations relative to handling static vs dynamic linking of the CRT, so the following may be redundant, but bear with me. First, there is the cargo wix init and cargo wix print operations. Should we be checking for dynamic/static linking of the CRT at initialization/print of the WXS template? There would be no EXEs to inspect at this point. The user would be required to explicitly state what is happening with the CRT, either through a Q&A prompt or CLI flags/options. However, if the target-feature="crt-static" is defined in the package's manifest or the user's Cargo configuration, then this would be known at initialization/print time. The CLI flags would override these configurations and if the configuration cannot be found and the CLI flags not used, then a WARN statement could be displayed (Not sure what would be printed/generated for the WXS template in this case).

Then, there is the cargo wix installer creation operation, which may or may not build the Rust project and call rustc. The developer may or may not have altered the WXS file and/or the project's build configuration (enabled static linking). Since it is possible to change from dynamic to static linking at build-time, I guess we would need to pass some variable to WiX and the WXS template to automatically flip between including the CRT installer (Merge WiX) or not. Flags/options would need to be added to disable/enable/customize this feature. The WXS template would need to contain the Merge WiX section but be behind some preprocessor directives.

After this discussion with myself, it appears the WXS template is going to always need the Merge WiX implementation but it should be behind a preprocessor directive to enable or disable at installer creation time. I don't see a need for anything to happen at initialization/print time. In other words, a Q&A prompt, WARN statement, CLI flags/options, configuration check, etc. about the CRT at initialization/print do not appear to be needed as long as the WXS template always includes the Merge implementation behind a WiX variable.

Is any of this a concern with the -gnu toolchain? Is a different CRT used with the -gnu toolchain, so we would need to account for that as well?

How do we determine which CRT installer to include, i.e. version?

Action items:

1. Add to the WXS template a preprocessor block with the WiX tags for including the CRT installer.
2. Add check for static vs dynamic linking of the CRT during installer creation, i.e. cargo wix.
3. Add CRTLinkage WiX variable that is either static or dynamic and passed from cargo-wix to the WXS template at creation-time. The preprocessor block in Item 1 would include the CRT installer if the CRTLinkage WiX variable was dynamic.
4. Add --exclude-crt flag to the cargo wix command. If invoked, the CRTLinkage WiX variable in Item 3 would be set to static and the CRT installer would not be included for this invocation of the cargo wix command, i.e. build/creation.
5. Add WARN statement if --exclude-crt is used, but check from Item 2 fails to identify static linking.
6. Add INFO statement if the CRT installer is included. This would help explain why a follow-on installer for a run-time is being included/called.

For reference: Static and Dynamic C Runtimes

I should probably rename this issue as this is more than just documentation, but actually a feature. I appear to have closed #114 too soon. My apologizes.

[roblabla]

But a target_feature=crt-static was not listed as a target feature. I think this could be because the crt-static target feature is only enabled and added to the configuration right before the compile/build, or as usual, I am missing something about the order of operations.

RUSTFLAGS is handled by cargo, not rustc. You should simply run rustc -C target-feature=+crt-static --print cfg, which does give us a target_feature="crt-static"! And to get the various RUSTFLAGS, we can use cargo rustc -- --print cfg. So yeah, it turns out to be possible to check whether crt-static is enabled using --print cfg, good catch!

The hard part becomes finding which PE to parse. This is information cargo-metadata might be able to give us, but if the user modifies the WXS, we might want to instead parse the WXS to find which binaries are being included.

Would this be further complicated by the --no-build flag, too?

Well, if we go the cargo-metadata route, yes. If we go the WXS parsing route, no.

After this discussion with myself, it appears the WXS template is going to always need the Merge WiX implementation but it should be behind a preprocessor directive to enable or disable at installer creation time. I don't see a need for anything to happen at initialization/print time. In other words, a Q&A prompt, WARN statement, CLI flags/options, configuration check, etc. about the CRT at initialization/print do not appear to be needed as long as the WXS template always includes the Merge implementation behind a WiX variable.

Agreed, that seems like a fairly good course of action. Always have the merge, use the --print cfg trick to figure out if the binaries are being statically linked, and if so, set the variable, otherwise leave it unset.

Is any of this a concern with the -gnu toolchain? Is a different CRT used with the -gnu toolchain, so we would need to account for that as well?

AFAIK gnu toolchain has its own CRT and doesn't rely on msvcrt. I think they statically linked the CRT by default, but someone would need to double check. Either way, I think it makes sense to make a separate ticket for this.

How do we determine which CRT installer to include, i.e. version?

This is... complicated. I believe it depends on which version of the Visual Studio Build Tools you have installed. I don't know if there's a good way to know this without parsing the exe to find its DLL imports.

[volks73]

How do we determine which CRT installer to include, i.e. version?

This is... complicated. I believe it depends on which version of the Visual Studio Build Tools you have installed. I don't know if there's a good way to know this without parsing the exe to find its DLL imports.

Further questions to ponder and investigate after a brief exploration:

1. Is there an issue tracking Rust's toolchain upgrade from vcruntime140.dll (Visual C++ 2015 redistributable)?
2. I have 2019 build tools installed, does Rust still use the vcruntime140.dll (Visual C++ 2015 redistributable)?
3. Do the 2019 build tools come with the Visual C++ 2015 redistributable? 4, Do the 2019 merge modules come with the vcruntime140.dll?
4. Can the Universal CRT be used by cargo-wix instead?
5. The Universal CRT is a Windows 10 component and included with any Windows 10 installation. Can Rust use this and is there an issue tracking it?

It looks like the merge modules are optional components when installing VS and build tools 2019. The merge modules are included in 2015 and 2017 by default (of course!). This complicates matters even more because while 2019 Built Tools are needed when installing Rust, now an optional component must also be installed. Another caveat/dependency must be added to cargo-wix.

However, looking at my development environment, the vc_redist.x64.exe, vc_redist.x86.exe, etc. exist at C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Redist\MSVC\14.20.27508. While the merge modules do not exist by default, at least the redistributable exes are included. Maybe we use the EXE instead of the merge modules in the WXS template? Merge modules also appear to not be recommended.

We could just assume the developer has installed the merge modules and output an error message with appropriate directions and information for either installing the merge modules for 2019 and/or switching to static linking.

As for selecting the appropriate CRT version, a simple implementation would be to force the user to explicitly state the version/path to merge module if dynamic linking is used. An error message to require a flag/option/configuration could be added that nicely indicates we need this information.

The list of caveats, dependencies, questions, etc. to implement this keeps growing. The simplest resolution continues to be "enable static compiling". Let's do the following:

1. Assume the developer actually wants static linking but did not realize the CRT is dynamically linked by default on Windows.
2. Add check if static linking is enabled and output a WARN statement if it is not enabled and/or we cannot detect that it is enabled. With the rustc --print cfg statement, I think will have a reliable and robust mechanism for checking if its in enabled/disabled. A --suppress-static-linking-warning-like flag could be added, which I would take to be as a way to indicate to cargo-wix the developer has customized the installer in some way and knows what he or she is doing.
3. Add documentation about including the CRT using merge modules and adding to the WXS file. This would include information about installing the merge modules for the 2019 build tools.

These steps would avoid the whole merge modules, versions, redistributables, etc. problems, but I acknowledge not the most "out-of-the-box" UX for the cargo-wix subcommand and users.

I found Rust, Windows, and MSVC with discussion about the CRT, but it is long and I have not gone through all of it yet.

[volks73]

Added documentation and information about the CRT dependency and static linking as of bc06cb856f603a661050d9cc118bb71f973458bb.

This does not implement any resolution of the issue but it does resolve the original description of this issue and provides information to users until a more elegant resolution has been implemented.

[volks73]

I was experimenting with adding detection of the +crt-static compiler feature to at least notify/warn the user this might be an issue he or she needs to manually address/resolve. It is quickly spirally into a bigger issue (#135 and https://github.com/rust-lang/cargo/issues/8923). For now, the documentation will probably just need to be enough.