Need a reliable way to get the target dir from the build script

[kennykerr]

The problem: I need to locate the target dir from the build script.

The solution: I'd love a CARGO_TARGET_DIR environment variable. There's clearly precedent for this as cargo doc depends on such a thing today.

@jyn514 suggested I reach out to the Cargo team for advice.

Currently the Windows crate has been using various hacky solutions that are problematic.

https://github.com/rust-lang/docs.rs/issues/1444#issuecomment-875188411

[kennykerr]

And grabbing the target_directory reported by "cargo metadata" doesn't work. If the crate is coming from crates.io then Cargo builds it in C:\Users\kenny\.cargo\registry\src\<random>\<some-crate> and the "target_directory" that cargo metadata reports is C:\Users\kenny\.cargo\registry\src\<random>\<some-crate>\target - what I need is the target directory for the crate the developer is building directly, not an intermediate target directly only known to the dependency.

[zeerooth]

I've been recently working on a project where there's need to link the generated binary in user project to another directory and I couldn't find a reliable way to get the target directory either. Currently, the code just guesses that the target directory is set as default <project dir>/target/ but if the project is located in a workspace or the build.target-dir is set to something else then this linking simply fails. Such feature to get the target dir reliably would be greatly appreciated :heart:

[jyn514]

The solution: I'd love a CARGO_TARGET_DIR environment variable. There's clearly precedent for this as cargo doc depends on such a thing today.

This has cause and effect backwards. cargo doc works fine without CARGO_TARGET_DIR set in the environment. Docs.rs uses that as an input to cargo because the source directory is sandboxed, and cargo happens to pass it through to any build scripts it invokes.

[kennykerr]

Yes, I since noticed that this is an optional input to Cargo:

https://doc.rust-lang.org/cargo/reference/environment-variables.html

Indeed, that's not what I want. I really need the target dir for the most dependent crate being directly built. I've tried numerous unsatisfying hacks. The one that works the most reliably is to read the first path from the PATH environment variable. This always appears to be:

<target_dir>\<profile>\deps

Although this only works on Windows. And I don't know how "supported" this is but I haven't found a better solution.

[ehuss]

Can you provide more information on exactly why you need the target directory? Cargo is designed so that build scripts are intentionally constrained on what they should do, and their only interaction should be through the OUT_DIR.

[riverar]

@ehuss Kenny is out this week but some scenarios are listed here https://github.com/microsoft/windows-rs/issues/979#issuecomment-880271566.

I'll take a stab here at paraphrasing:

The windows crate has a build script that generates bindings for the developer, sourcing from its built-in metadata or local workspace metadata. Things get tricky when this metadata is delivered via multiple crates in the dependency graph.

For example, envision a developer's crate depending on windows, windows-fizz, and windows-buzz each with their own unique metadata. The build script (macro) running in the context of the developer's crate needs to have access to all the dependent crates' metadata to generate the developer's desired bindings.

~~One proposed change is to align the normal build script behavior with the cargo doc build script behavior. CARGO_TARGET_DIR would then exist and point to the developer's crate target directory and all dependent crates could write their metadata to that path for consumption.~~

One proposed change is to provide a pointer of sorts to the developer's crate target directory and all dependent crates could write their metadata to that path for consumption. Open to other ideas, of course.

[jyn514]

One proposed change is to align the normal build script behavior with the cargo doc build script behavior.

@riverar you're making the same mistake as https://github.com/rust-lang/cargo/issues/9661#issuecomment-877413805, cargo doc is not related to the target dir. Docs.rs just happens to set CARGO_TARGET_DIR and you've never tried setting it locally.

[riverar]

@jyn514 Ah right, forgot. Thanks!

This doesn't affect the proposed change much. The bottom line is we'd like a pointer to the target directory for the aforementioned reasons.

[matklad]

One use-case I have is to store build-script's own caches. I can use a temporary directory as a scratch space (though it might still be nice to place this somewhere inside ./target rather than /tmp), I can use OUT_DIR for a real output, but I am not sure what's the right place for intermediate artifacts generated by the build script, which are not necessary output, but which I nonetheless want to preserve between invocations.

For example, if I am building some C code, I want to put intermediate .o files somewhere. Or, my specific use-case, I recursively invoke Cargo to compile some helpers to wasm. I mistakenly tried to use target dir for that, but that broke in all kinds of interesting ways down the line. Looking at how cc crate just puts .o into OUT_DIR, I think it's correct to use that for intermediate artifacts as well, but it isn't obvious from the docs. Will send a PR shortly (https://github.com/rust-lang/cargo/pull/10326).

[kmod-midori]

+1 for this issue as we are distributing dynamic libraries that are hard to build (especially on Windows) internally (by automatically downloading them from a server), and these libs needs to be in the same directory as the final executable.

Without a reliable way to determine TARGET_DIR (e.g. the user might be cross compiling, or using non-default toolchains), cargo run will not work out-of-the-box.

[dureuill]

FWIW, cxx_build has the following module to determine target_dir: https://docs.rs/cxx-build/latest/src/cxx_build/lib.rs.html#1-473.

It expects the path stored in the OUT_DIR environment variable as input. I'm not certain it handles all possible situations though.

[Be-ing]

I'm working on cxx-qt, a code generator build on top of cxx. I need a stable path to output generated C++ headers from build.rs for C++ build systems to be able to find them. I've seen what cxx_build does by walking up from OUT_DIR but that feels quite hacky to me. The least worst idea I've come up with is explicitly setting the CARGO_TARGET_DIR environment variable from the C++ build system so it gets passed to build.rs. I wish I did not have to do this though. Of course, this workaround isn't available if you're not working with an automated system that invokes cargo.

[Be-ing]

On further thought, for cxx-qt (and cxx), perhaps setting the CARGO_TARGET_DIR environment variable isn't the best idea. I don't want to depend on unstable implementation details of the structure of the target directory. I think it would be better to use an environment variable specific to this purpose. Or better yet, a designated place to put exported build artifacts (#5457).

I think this is quite a different use case from the windows crate. Please correct me if I'm wrong, but I think what @kennykerr and @riverar are asking for is a way to pass data from the build script of a dependency to downstream build scripts. I'm not sure exposing the target directory to build scripts is a great way to do that either. cxx_build uses a public static mut struct together with links in Cargo.toml for this purpose... which also feels hacky, but I guess it's less hacky than relying on the layout of the target directory? :shrug:

[sam0x17]

I think it is quite silly that at present it is difficult for a crate to discover where it is currently being built

[tommy-gilligan]

A use case I have: I want an easy way to run a target (a CLI bin) from an integration test (using the Rust test runner). Not a blocker, just a nice to have.

Some people with vaguely the same use case and some solutions: https://users.rust-lang.org/t/integration-tests-for-binary-crates/21373/9

[epage]

cargo already makes binaries available for integration tests. cargo_bin! is one example of this.

Trying to get the binary from just CARGO_TARGET_DIR has issues because the binary can be dropped in a couple of different places, depending on how you are building the project.

[tommy-gilligan]

Ah! Thank you! Sorry, I should have looked a bit more .

CARGO_BIN_EXE_ — The absolute path to a binary target’s executable. This is only set when building an integration test or benchmark. This may be used with the env macro to find the executable to run for testing purposes. The is the name of the binary target, exactly as-is. For example, CARGO_BIN_EXE_my-program for a binary named my-program. Binaries are automatically built when the test is built, unless the binary has required features that are not enabled.

https://doc.rust-lang.org/cargo/reference/environment-variables.html (in case it helps anybody who ends up here)

[sam0x17]

For anyone who might need it, this actually seems to work fine and I've been using it in docify for a while now (gets the crate root of the current proc macro caller when executed within the body of a proc macro):

https://github.com/sam0x17/docify/blob/ecb7cc87ec2f2670ce61d7dc3cc331310c42b2a6/macros/src/lib.rs#L72-L102

[marcmenem]

This seems to do the trick for me:

rustup toolchain install nightly
cargo +nightly config get build.target-dir -Z unstable-options

[ssrlive]

This variable CARGO_TARGET_DIR is guaranteed in the cargo documentation, and I'm curious why all the project maintainers refuse to add it. Another words, why don't you remove the description of CARGO_TARGET_DIR from the documentation?

[weihanglo]

If you had read it carefully, in the doc, it is under the section of "Environment variables Cargo reads", which is different from the latter section "Environment variables Cargo sets for build scripts". You want your Cargo to sets the variable for your build script, and that is never guaranteed.

From https://github.com/rust-lang/cargo/issues/12673#issuecomment-1721123826:

This implementation could be: if the CARGO_TARGET_DIR variable already exists, do nothing, if it does not exist, add it. Is there anything wrong with this?

There is nothing wrong for a feature request like this. But again, if you had looked into issues you linked carefully, there are indeed some open questions needing to address. For example, the concurrent issue for accessing target-dir mentioned here https://github.com/rust-lang/cargo/pull/8710#issuecomment-695047331. Also, we may need to figure out the implication and interaction between this and other features, such as https://github.com/rust-lang/rfcs/pull/3371 and per-user build cache.

[sam0x17]

This is the workaround I currently use when I need the macro caller crate root in proc macros: https://github.com/sam0x17/docify/blob/main/macros/src/lib.rs#L75-L112

This works in workspaces, outside of workspaces, and regardless of whether we are a crates.io crate or a local crate.

This could probably be adapted to reliably get the proper target directory as well.

Until we get better support for this sort of thing, this is probably the best you can do

[ssrlive]

Through the information displayed in this image, we can infer the exact path of CARGO_TARGET_DIR, it's d:\mytoy\target\x86_64-pc-windows-msvc\release. My problem is solved.

Since OUT_DIR is unique, then CARGO_TARGET_DIR is also unique.

Now my question is: why you guys are so stubborn in not providing CARGO_TARGET_DIR .

Here is my code to find out CARGO_TARGET_DIR, hope it can help somebody.

fn get_cargo_target_dir() -> Result<std::path::PathBuf, Box<dyn std::error::Error>> {
    let out_dir = std::path::PathBuf::from(std::env::var("OUT_DIR")?);
    let profile = std::env::var("PROFILE")?;
    let mut target_dir = None;
    let mut sub_path = out_dir.as_path();
    while let Some(parent) = sub_path.parent() {
        if parent.ends_with(&profile) {
            target_dir = Some(parent);
            break;
        }
        sub_path = parent;
    }
    let target_dir = target_dir.ok_or("not found")?;
    Ok(target_dir.to_path_buf())
}

In addition, I also found a bug in your recent implementation: the two strings d:\mytoy\target\release\deps and d:\mytoy\target\release you added to the Path environment variable are Wrong, it should be d:\mytoy\target\x86_64-pc-windows-msvc\release\deps and d:\mytoy\target\x86_64-pc-windows-msvc\release.

[kwinz]

I support this feature request. I need this functionality if I want to copy some assets to my target folder. I am using https://doc.rust-lang.org/book/ch14-03-cargo-workspaces.html so I can't just hardcode a path.

CARGO_TARGET_DIR and/or CARGO_BUILD_TARGET_DIR and/or another name that you define should be an environment variable that is set before build.rs is called so I can read it just like I am able to read PROFILE.

fn get_cargo_target_dir() from @ssrlive is a great workaround. Thank you very much!

I still hope that we can get this feature request with an officially supported env that is less hacky.

[epage]

Note that directly exposing CARGO_TARGET_DIR was attempted in the past but was rejected, see https://github.com/rust-lang/cargo/pull/8710#issuecomment-695047331

• Each package needs an isolated space for concurrent builds
• The guarantees around the target/ layout is limited and we are looking at making changes for #5931 + #12633

Cheers to weihanglo for finding this context earlier in the thread

So the way to move this forward is for us to look at the use cases and try to identify solutions without CARGO_TARGET_DIR

Doing a quick summary of use cases. If your use case is not listed here, it likely is because it lacked details for me to understand what is happening / why this is needed

• Build-to-build cache directory: https://github.com/rust-lang/cargo/issues/9661#issuecomment-1021029381
  • Seems trivial for us to provide a per-package CARGO_BUILD_TMP_DIR
• Staging files for the --out-dir so they are next to the cargo-built final artifacts: https://github.com/rust-lang/cargo/issues/9661#issuecomment-1049452886
  • I see this use case also brought up in #545 though the fact that build script is involved deals with having a stable way of knowing where the build output went (see next item)
• build script generating C++ header files from the Rust code: https://github.com/rust-lang/cargo/issues/9661#issuecomment-1223301577
  • The thing we need to understand is if a build script is the appropriate tool for this or if it was the most convenient. Generating C++ headers is not needed for the current package (e.g. "if a package depended on yours through the registry, would it be needed?". This seems like more of a build orchestration concern.
• find binaries for running end-to-end tests: https://github.com/rust-lang/cargo/issues/9661#issuecomment-1636573626
  • Already supported: https://github.com/rust-lang/cargo/issues/9661#issuecomment-1636577484

[Be-ing]

build script generating C++ header files from the Rust code: https://github.com/rust-lang/cargo/issues/9661#issuecomment-1223301577 The thing we need to understand is if a build script is the appropriate tool for this or if it was the most convenient. Generating C++ headers is not needed for the current package (e.g. "if a package depended on yours through the registry, would it be needed?". This seems like more of a build orchestration concern.

To frame this use case more generally: generate an artifact (anything other than code that gets compiled into the library/executable; for example, C/C++ header, man page, icons...) during a Cargo build which is put somewhere that external tools (for example, a C++ build system, or a package generator) can reliably find them. A build script doesn't have to be the solution for this use case; so long as there is some way to run Rust code during a build which can output files to a path that external tools can predict, that could work.

For the particular cases of CXX(-Qt), this mechanism for generating artifacts should be available to any crate in the dependency graph, not only the final library/executable. It would be a bit inefficient if this was somewhere other than the build script because CXX(-Qt) need to generate and compile the corresponding C++ source file for the output C++ header which needs to be done in the build script, but if there was a bit of redundant code run in both the build script and some artifact-generation-script, that wouldn't be the end of the world.

[epage]

I would very much rather not frame these discussions more generally; the concrete details for how a solution is wanting to be used and why can make a big difference.

[nanoqsh]

@ssrlive

Here is my code to find out CARGO_TARGET_DIR, hope it can help somebody.

Thanks for the solution, but unfortunately it doesn't work with custom profiles The PROFILE variable sets to either release or debug, regardless of an actual profile name.

Thus, if I use a custom profile, for example:

[profile.production]
inherits = "release"
lto = true

My OUT_DIR will be /project-dir/target/production/build/crate-name-17eab3514163ed76/out. I needed another way to get the target directory, so I thought, why not just skip the required number of directories?

The only caveat is that if I compile a target other than host's, cargo adds a subdirectory with a name of target triple, for example wasm32-unknown-unknown. So I need to skip one more directory in this case.

I wrote the following code and so far everything works

fn get_cargo_target_dir() -> Result<std::path::PathBuf, Box<dyn std::error::Error>> {
    let skip_triple = std::env::var("TARGET")? == std::env::var("HOST")?;
    let skip_parent_dirs = if skip_triple { 4 } else { 5 };

    let out_dir = std::path::PathBuf::from(std::env::var("OUT_DIR")?);
    let mut current = out_dir.as_path();
    for _ in 0..skip_parent_dirs {
        current = current.parent().ok_or("not found")?;
    }

    Ok(std::path::PathBuf::from(current))
}

Perhaps there are details that I didn't take or there are ways to make the code more reliable. So let me know if you have any ideas. Regarding the feature discussed, I definitely need a reliable way to get the target directory, so +1

[masonk]

+1 to the use-case of "Staging files [...] next to the cargo-built final artifacts".

In my case I want to copy a config.toml file from my crate root into a new file in the same directory as the the final .exe of the binary.

[WesleyAC]

I want to chime in and support the usecase of "generating header files for FFI" — I'm surprised that this isn't a solved problem. In my case, I could do this at a higher level in the build system (I'm doing FFI between Rust and Dart, and need to get header files for Dart's ffigen program, so I could have a Makefile that calls a binary target in my rust code that generates the headers and puts them in a expected location before calling the dart build or something, but that seems a lot more complicated than just generating the headers when the code is built, which has the nice property of preventing the headers from getting out of sync with the .so file, for instance)