Failed to cross compile rust-openssl(openssl-src-rs) for windows msvc targets via cross-rs

[cavivie]

cross-rs: At present, cross-rs uses the Docker Ubuntu environment when compiling, and will download the wine and msvc tool chain in Ubuntu container to compile by mapping the source code into the container.

openssl: When the vendored feature is enabled, rust-openssl will call the openssl-src-rs to build the openssl c library. openssl-src-rs uses rust-lang/cc-rs to find the msvc tool when compiling the openssl library.

cc::windows_registry::find(target, "nmake.exe");

thread 'main' panicked at 'failed to find nmake', /root/.cargo/registry/src/github.com-1ecc6299db9ec823/openssl-src-111.25.1+1.1.1t/src/lib.rs:454

Although the correct VC environment has been set, the find result is still None. Then I checked ` find_tool' function:

#[cfg(not(windows))]
pub fn find_tool(_target: &str, _tool: &str) -> Option<Tool> {
    None
}

When compiling cross platforms, should we use below the code to check?

// use these
#[cfg(target_os=windows)]
#[cfg(not(target_os=windows))]

// not use these
#[cfg(windows)]
#[cfg(not(windows))]

Or should we manually pass --cfg windows to compile?

[ChrisDenton]

According to the docs, all target_* should be set correctly for the target. That would include both target_family=windows and windows. See: https://doc.rust-lang.org/reference/conditional-compilation.html#set-configuration-options.

Technically target_family is more general and could apply to other Windows-compatible OSes (though I don't think we yet have official support for any such target).

[cavivie]

I'm so sorry. I think my description is wrong. We should use CARGO_CFG_TARGET_OS option to check the target instead of cfg conditional compilation, because #[cfg(windows)] detects the runtime environment. So in order to support cross compilation, I think this conditional compilation should be modified.

/// Similar to the `find` function above, this function will attempt the same
/// operation (finding a MSVC tool in a local install) but instead returns a
/// `Tool` which may be introspected.
#[cfg(not(windows))]
pub fn find_tool(_target: &str, _tool: &str) -> Option<Tool> {
    // print some information for debugging
    println!("cargo:warning=target is not windows, but env `CARGO_CFG_TARGET_OS` is {:?}", "", std::env::var("CARGO_CFG_TARGET_OS"));
    None
}

When I call cc to build for the compilation target is x86_ 64-pc-windows-msvc on Ubuntu, this code will be executed and the following information will be output:

warning: target is not windows, but env `CARGO_CFG_TARGET_OS` is windows

But I hope it will follow the code, because it is needed to find the wine and msvc toolchain that I have installed on Ubuntu

/// Documented above
#[cfg(windows)]
pub fn find_tool(_target: &str, _tool: &str) -> Option<Tool> {
...
}

[cavivie]

I thinkcc should use the TARGET OS/FAMILY information in CARGO ENV to find toolchains, rather than the runtime OS/FAMILY information, because it cannot be found correctly in build scripts. As I said before, use wine and msvc toolchains to build a target on Linux environment such as Ubuntu.

[thomcc]

When compiling cross platforms, should we use below the code to check?

// use these
#[cfg(target_os=windows)]
#[cfg(not(target_os=windows))]

These will not work either. Note that inside a build script the cfgs are for the host platform, in your case linux.

Anyway, the code inside the windows_registry only works on windows hosts anyway, it doesn't work when cross compiling (and you don't need it to cross compile), as it's for locating MSVC tools when targetting --target=*-pc-windows-msvc. Given that MSVC and its tools do not run on non-Windows, we cannot cross-compile to those targets anyway.

Given that the nmake error you mention is in the branch of the code that is for msvc-only: https://github.com/alexcrichton/openssl-src-rs/blob/50efcdeac601febc3ab5a90f3eb729e472c124e1/src/lib.rs#L512-L521, I think you're trying to use a -msvc target.

When cross compiling from linux to windows, you must use the *-pc-windows-gnu targets (such as x86_64-pc-windows-gnu). Cross-compiling to these will may require installing and configuring mingw if it's not present, but looking at https://github.com/cross-rs/cross/blob/99b8069c0d977a14cd421ad8a3ef3255dc5802be/docker/windows-entry.sh#L14 it seems likely that it will be.

[cavivie]

msvc: https://github.com/cross-rs/cross-toolchains/blob/main/docker/msvc-windows-entry.sh

[cavivie]

I see that, despite this, we still cannot compile openssl with the target of msvc on a Linux host. So I have to give up and use Windows image to build openssl.

[cavivie]

It works for me:

/// Attempts to find a tool within an MSVC installation using the Windows
/// registry as a point to search from.
///
/// The `target` argument is the target that the tool should work for (e.g.
/// compile or link for) and the `tool` argument is the tool to find (e.g.
/// `cl.exe` or `link.exe`).
///
/// This function will return `None` if the tool could not be found, or it will
/// return `Some(cmd)` which represents a command that's ready to execute the
/// tool with the appropriate environment variables set.
///
/// Note that this function always returns `None` for non-MSVC targets.
pub fn find(target: &str, tool: &str) -> Option<Command> {
    // old version:
    // find_tool(target, tool).map(|c| c.to_command())

    // patch version:
    use std::env;

    // This logic is all tailored for MSVC, if we're not that then bail out
    // early.
    if !target.contains("msvc") {
        return None;
    }

    // Early return if it the current target is not a windows target.
    if let Ok(target) = env::var("CARGO_CFG_TARGET_FAMILY") {
        if target != "windows" {
            return None;
        }
    }

    // Early return if the environment doesn't contain a VC install.
    env::var_os("VCINSTALLDIR")?;
    env::var_os("VSINSTALLDIR")?;

    // Fallback to simply using the current environment.
    env::var_os("PATH")
        .and_then(|path| {
            env::split_paths(&path)
                .map(|p| p.join(tool))
                .find(|p| p.exists())
        })
        .map(|path| std::process::Command::new(path))
}

[cavivie]

Given that MSVC and its tools do not run on non-Windows, we cannot cross-compile to those targets anyway.

Wine can run the windows tools, but it seems to go to the #[cfg(not(windows))] code branch, that's why I'm confused.

[Emilgardis]

is there a reason https://github.com/rust-lang/cc-rs/blob/2447a2ba5f455c00b1563193e125b60eebbd8ebe/src/windows_registry.rs#L42-L49 can't use CARGO_CFG_TARGET_OS here?

[ChrisDenton]

Because if the host is not Windows-like then using COM or the Windows registry won't work. Which is the point of find_tool. It tries to find the required Visual Studio install using Windows-specific methods.

[Emilgardis]

would it be possible to implement the ability to specifically point out the tools via an env-var that we can then set on non-windows if we do have the tools and emulation needed?

would the project be open to such a change?

[ChrisDenton]

Instead of the current None for Unix, you could instead adapt find_msvc_environment for the unix case. However, it contains a fallback for when the environment target does not match the requested target whereas in the unix case you'd want to fail because there's no fallback available.

EDIT: The simplest way would be to test for VCINSTALLDIR and VSINSTALLDIR environment variables and, if they exist, then trust that PATH contains the right tool and that things like the LIB and INCLUDE variables are appropriately set.