--out-dir Tracking Issue

[ehuss]

Original issue: #4875 Implementation PR: #5203 Documentation: https://doc.rust-lang.org/nightly/cargo/reference/unstable.html#out-dir Issues: https://github.com/rust-lang/cargo/labels/Z-out-dir

Summary Adds --out-dir=PATH flag to cargo build to specify a directory to place final artifacts.

Unresolved questions

• [ ] #6100 — concern about similarity with --target-dir, proliferation of flags
• [ ] #6313 — concern about name collisions in workspaces or with examples
• [ ] What exactly is the use case for this? In what scenario would it be used where you don't already have an outer build system or script that can copy the files into the correct location?
• [ ] Confusion with rustc's --out-dir
• [ ] Discrepancy or confusion with build.rss OUT_DIR (depends on if build.rs is generating intermediate or final artifacts)

[ultrasaurus]

I find this feature very helpful, since I get output in a directory named like this ./target/debug/build/mylib-79d5eadf36eadf53 and with a build.rs file there are two similarly named directories.

However, it doesn't seem that OUT_DIR environment variable is set to the value of --out-dir when build.rs is executed. Is that intentional? is there some other way to discover the value of --out-dir?

[haze]

+1 on this issue. I have a program that uses a template directory for rust compilation and would like to now have to recompile everything. Yes I could symlink all the folders within target, but it would be nicer to be able to specify the binary output path. I'm not sure why this is such an overlooked issue.

[robinbudd]

When will this feature be available in the stable release? It would greatly simplify my build/CI pipeline.

[Chris--B]

+1 on this. This greatly improves the workflow on our CI, and we'd like to avoid using a nightly compiler just for this.

What needs to happen for this to move forward?

[lf-]

This issue makes nmattia/naersk unusable on stable rust. (another build system issue)

[golddranks]

This would streamline some of my Dockerfiles and build scripts quite nicely.

[Diggsey]

Is there a reason this doesn't work with cargo rustc?

[benesch]

I just published a crate called cargo-out-dir to crates.io, which can ease some of the frustration while we wait for this to stabilize. All it does is print out the current' crate's out directory, as determined by running cargo check --metadata-format=json:

$ cargo install cargo-out-dir
$ cargo out-dir
/home/benesch/path/to/crate/target/debug/build/foo-9d9f15d3d084bc3a/out

[simonbuchan]

Another potential use case for out-dir is making cargo run work nicely with building MacOS apps, which expect a structure like:

My App.app/Contents/
  Info.plist
  MacOS/actual-binary
  Resources/foo.png

in order for native APIs like (the moral equivalent of) [NSImage imageNamed: @"foo"] to work.

But in this case it would be more helpful to be able to put out-dir = "dist/My App.app/Contents/MacOS" in Cargo.toml somewhere, presumably in the [bin] section. Not sure if that makes anyone more or less happy?

[workingjubilee]

--out-dir should not have this name because --out-dir on rustc performs almost the exact function of --target-dir, and to have this be a copy directory command on cargo instead would be confusing. A name like --copy-dir would suffice, however. I believe one of the proposed solutions for #6100, such as having the ability to independently specify "scratch" and "final" build artifact directories is worth investigating.

[piegamesde]

What exactly is the use case for this? In what scenario would it be used where you don't already have an outer build system or script that can copy the files into the correct location?

I run into this regularly (multiple times a year), and I am not the only one. At the moment, people hard-code ./target/release/binary or some variant of it like $CARGO_TARGET_DIR/$TARGET/$APPNAME for copying their binaries in installation scripts. This quickly breaks apart in the following cases:

• The cargo target dir has been moved to a custom value
• Cross compilation.

Cargo's current file handling is unfortunate, and stabilizing this value would help a lot. Specifically, all temporary build files should be freely movable to an external directory, while the build outputs (i.e. the binaries or shared objects) should land in a fixed path for easy packaging.

(In an ideal world, ./target would only contain what --out-dir currently provides by default, and all other build artifacts would land somewhere user-configurable with a default to $XDG_CACHE_HOME/cargo-target/appname or something)

Related: https://github.com/rust-lang/cargo/issues/6100#issuecomment-821022329

[bugproof]

@piegamesde exactly, I'm used to having bin and obj nice and clean. There should be an environment variable or make it configurable from file as well (--out-dir )

[vabka]

Why is it still unstable? I need to use some dark magic to build in debug/release profiles in one dockefile

...
# If release is set to any value then PROFILE env var becomes 'release' and then left unchanged
# Otherwise PROFILE env var becomes empy and then changes to debug 
ARG release
ENV PROFILE=${release:+release}
ENV PROFILE=${PROFILE:-debug}

RUN cargo build --frozen --offline --target x86_64-unknown-linux-gnu ${release:+--release}

FROM docker.io/ubuntu:22.04 as prod
COPY --from=build /target/${PROFILE}/executable /
...

[dpaoliello]

What exactly is the use case for this? In what scenario would it be used where you don't already have an outer build system or script that can copy the files into the correct location?

The key scenario for this is integrating with other build systems, especially where you are cross-compiling: since Cargo will place the build artifacts into a target-triple directory, you need to teach the outer build system about Rust's target-triples so that you know where to look to find the build artifacts.

I'd mostly be happy if there was an option to suppress the target-triple directory (and the release or debug directory).

[workingjubilee]

The key scenario for this is integrating with other build systems, especially where you are cross-compiling: since Cargo will place the build artifacts into a target-triple directory, you need to teach the outer build system about Rust's target-triples so that you know where to look to find the build artifacts.

...? But in that case you need to have invoked them using --target in the first place, which means you already know what the triple is.

[detly]

...? But in that case you need to have invoked them using --target in the first place, which means you already know what the triple is.

Just from experience with various (non-embedded) CI and embedded build systems, it's often easier to determine a path for the binary artifacts early and up-front and then use that everywhere, than try to reverse engineer your build tool's scheme for placement. Especially if there's no agreement that the build tool's placement will remain stable.

Consider also that with CI/CD you're configuring things in YAML, so no functions to call like cargo_path_for_binary(app_name, target_triple, profile) - it's copy and paste everywhere. GNU Make at least has functions, but they're not fun to figure out.

There may be more concrete reasons too, that's just what came to mind.

[workingjubilee]

CI configuration languages generally accept variables, indeed this is virtually required for making a CI matrix work. And in a project I have recently been working on which requires constructing a custom build system, I made it so cargo is always invoked with a --target, even if it is just echoing the host tuple, precisely because I don't like the fact that binary artifacts appear in /target/release or /target/release/blah-blah-blah-blah/ based on whether you invoke cargo with --target or not. But neither does it work well to potentially admix the artifacts of different targets.

[piegamesde]

...? But in that case you need to have invoked them using --target in the first place, which means you already know what the triple is.

One problematic use case I ran into was with some application that was built with Meson. The problem is that it just copied ./target/release/appname into the installation folder. Now I wanted to add cross compilation support for it, and had to hack my way around so that it would take ./target/target-triple/release/appname instead. I think the situation would be half as bad if we didn't have this weird two-level stuff going on based on whether the project is cross-compiled or not.

[dpaoliello]

...? But in that case you need to have invoked them using --target in the first place, which means you already know what the triple is.

Just from experience with various (non-embedded) CI and embedded build systems, it's often easier to determine a path for the binary artifacts early and up-front and then use that everywhere, than try to reverse engineer your build tool's scheme for placement. Especially if there's no agreement that the build tool's placement will remain stable.

Consider also that with CI/CD you're configuring things in YAML, so no functions to call like cargo_path_for_binary(app_name, target_triple, profile) - it's copy and paste everywhere. GNU Make at least has functions, but they're not fun to figure out.

You might know the target triple within the Rust project, but if you are wrapping calls to Cargo from another build system then there are likely other non-Rust projects in that build system that are unaware of Rust's target triples. It's much easier in a build system to assume that the built artifact is $OUTPUT/$PROJECT.so than to have to thread the target triple (and knowledge that a subset of projects are Rust) throughout the rest of the project so that you can generate $OUTPUT/target/$TARGET_TRIPLE_IF_RUST/$FLAVOR_IF_RUST/$PROJECT.so.

Things only get worse if a project can customize its target triple: it might be trivial to create some Make/CMake function to generate the target triple based on the target architecture of the build, but if an individual project can customize that triple (e.g., MUSL vs GNU, MSVC vs GNU, or using a none target for no_std) then the external projects need to rely on implementation details of how that project is configured to know where its output is.

[hashworks]

As a practical example, this is a problem when one builds and packages mdcat - besides the binary target/release/mdcat additional files are produced, which one needs to locate in a (IMO) unstable way:

$ git clone https://codeberg.org/flausch/mdcat
$ cargo fetch --locked
$ cargo build --frozen --release --target-dir=target
$ ls target/release/mdcat
target/release/mdcat
$ ls -d target/release/build/mdcat-*
target/release/build/mdcat-2a83e790d2b3f7a6  target/release/build/mdcat-7decb70a642f31fb
$ find target/release/build -type d -path "target/release/build/mdcat-*/out"
target/release/build/mdcat-7decb70a642f31fb
$ ls target/release/build/mdcat-7decb70a642f31fb/out
completions  _mdcat  mdcat.bash  mdcat.fish  _mdcat.ps1

[mcandre]

I need this --out-dir feature in order to automate archiving binary artifacts (e.g. tar -czvf ..., zip -r ...), when I release my Rust apps.

Not a fan of how the flag is teased in non-nightly cargo's help menu, only to bail when invoked. Don't bother me. Either let me use the flag, or at least drop it from the CLI documentation.

Alternatively, I could use nightly, and require my users to use nightly. For DevOps users and other fans of pinned, immutable versions, nightly is not a valid option.

Alternatively, I would have less need for this feature if cargo simply nested binary artifacts in a more archive amenable tree structure, like target/bin, by default. Currently, cargo flattens out binary artifacts--the most important part of the build--polluting recursive archives with sibling junk files.

Alternatively, one could write a custom shell script to accomplish this. But that wouldn't work very well across all Rust projects, and it would needlessly restrict Windows devs, and proper robust shell programming is rather a dark art, and shell scripts substantially depart from the Rust ecosystem.

Alternatively, one could configure archive commands to strip out the junk sibling files, but that's a maintenance nightmare, as cargo will surely (as any package manager reasonably would) introduce additional file patterns in the build tree internal directory over time.

Alternatively, one could configure archive commands to look exclusively for file patterns matching target/(debug|release)/<all the binaries mentioned in Cargo.toml>(\.(exe|js|wasm))*. Another maintenance nightmare.

I don't want to get too involved with low level details of cargo internal build files. I just want more reasonable defaults. Or failing that, landing --out-dir in the next Rust release.

[valadaptive]

It would be really really nice if this feature would be stabilized. I'm trying to integrate Cargo with Meson, whose developers steadfastly refuse to support artifacts being output into any subdirectory, so being able to output the artifacts into any directory would be extremely helpful.

[valadaptive]

I'm trying to figure out what the blockers for stabilizing --out-dir are. So far, I've been able to gather the following:

• Some people think calling the argument "out-dir" is ambiguous and confusing (what exactly is going "out" into it?), especially given that the OUT_DIR environment variable already exists and means something completely different.
  • If naming is a problem, why not call it --artifact-dir instead? That makes it more clear that we're specifying the directory that the final build artifacts will be placed into.
• Some people want to see a much larger refactoring of how Cargo structures all of its directories, and are concerned that --out-dir would end up as a piece of historical cruft once that refactoring happens.
  • However, as others have pointed out, changing the existing target directory structure in any way would likely be a breaking change, given how many projects have come to rely on certain files being in certain places.
  • In addition, I see this type of "let's wait until we can fix everything" as a form of cookie-licking. In the 4 years since this issue was opened, it seems that no substantial progress has actually been made on this supposed large rethinking, and it's only ever brought up as a reason not to stabilize --out-dir.
• Some people claim that access to build artifacts is solved by allowing people to better specify the location of the target/ directory, in particular via https://github.com/rust-lang/rfcs/pull/3371.
  • However, this RFC doesn't actually address many of the use cases for --out-dir in the first place. For example, as I mentioned above, Meson will not let you pull build artifacts out of any subdirectory. The files themselves must be in the same directory as your meson.build files, no ifs, ands, or buts.

I'm not familiar with the Cargo governance structure--what's needed to push this feature over the finish line?

[poliorcetics]

Some people claim that access to build artifacts is solved by allowing people to better specify the location of the target/ directory, in particular via https://github.com/rust-lang/rfcs/pull/3371.

As the author of that RFC: it doesn't solve the problem here from a quick read through. Or at least, that's not the intended effect and won't help anymore than setting CARGO_TARGET_DIR in the cargo config.

Meson will not let you pull build artifacts out of any subdirectory. The files themselves must be in the same directory as your meson.build files, no ifs, ands, or buts.

Well, another build system being recalcitrant to change is not a good argument for changing cargo IMO: cargo is made to play nice with itself, maybe some more Rust-specific tooling and not much else

If meson wants not to do something that's on them, not on every other tool on earth to fix it.

Outside of that, being able to specify an artifact directory (I like --artifact-dir) would indeed be nice !

[valadaptive]

Well, another build system being recalcitrant to change is not a good argument for changing cargo IMO: cargo is made to play nice with itself, maybe some more Rust-specific tooling and not much else

It's true that Meson's decision in particular is a bit silly, but I feel like some basic concessions to allow Cargo to integrate into other build systems are necessary.

Either:

1. Cargo is a build system for multi-language projects
2. Cargo makes itself usable as a part of build systems for multi-language projects
3. Cargo doesn't integrate in any way with projects that use languages other than Rust. If you want to incorporate Rust into other projects, use another Rust build system.

Option 1 is explicitly far outside of Cargo's scope for good reasons. Option 3 seems unnecessarily restrictive, and right now (as far as I know) there is no "another Rust build system" that supports external crates. That leaves option 2.

[eli-schwartz]

Option 3 seems unnecessarily restrictive, and right now (as far as I know) there is no "another Rust build system" that supports external crates.

Well, meson has a couple of in-flight changes with the intention of supporting external crates in the near future, and at least two major stakeholders whose choice of build system is "meson or bust" that want to utilize this. I can't promise on a precise timeline but we have been chipping away at the issues involved, pieces have landed, and there are practical PoCs.

It will definitely not suit everyone's taste, in particular the chances are approximately 0.000001% that we will ever support build.rs, but the hope is that those will be small enough in number that people can write a meson.build 'wrap file' for a handful of projects that need it and let meson automatically parse Cargo.toml for the rest.

I suspect we'll be finished before rust adds a way to specify the precise output file locations of its build artifacts.

[GrantGryczan]

I believe this comment addresses most of the concerns against stabilizing this issue (I don't know of any others), and I don't see anyone raising objections to it.

This issue is still labeled as "waiting on feedback", so what further feedback is needed? This flag would simplify my Docker setup, so I'd love to see it stabilized. What's left to be done until it can be?

[soloturn]

i do hope the one discussing about this flag will not be dead until this gets enabled in stable :)

[workingjubilee]

Can someone PR a rename of the flag from --out-dir to --artifact-dir as a first step to stabilizing this? It seems there was consensus on that.

[soloturn]

Can someone PR a rename of the flag from --out-dir to --artifact-dir as a first step to stabilizing this? It seems there was consensus on that.

where did you get the impression from that out-dir is not good enough, and what is the purpose to block it even longer?