cargo
cargo copied to clipboard
rust-lang
Possibility to set the package version on command line
Hello
I'm building a rust application using internal teamcity CI job. I want the build number to be part of the version of the binary (so eg. ./app --version knows from which build it came).
The only way I found so far is to let the build first edit the Cargo.toml (sed -i -e 's/^version = .*/version = "%build.version%"/' Cargo.toml), which seems ugly and fragile.
It would be great if I could somehow override the version from Cargo.toml through the command line ‒ either as a parameter or an environment variable.
Would something like that make sense? Are there plans to support it?
I think this could be created as a cargo extension using https://github.com/ordian/toml_edit. You could use https://github.com/killercup/cargo-edit as inspiration.
Actually, I don't want to modify the Cargo.toml at all if possible, just use that different value of version during a single cargo build. So something like cargo build --config package.version="0.1.2".
But yes, you're right, there are certainly less fragile ways to edit Cargo.toml.
I think there's a ticket for being able to modify options (I guess I'm both Cargo.toml and .cargo/config) via a command, like git config, but I don't know if it extends to temporary overrides.
It's very common use case to specify the version from command line while releasing on a CI server that tracks and auto increments the build number. GNU/make, Maven and Gradle natively support it, manual editing of a build-spec file is not practical and error prone.
make target VERSION=%build.version%
mvn release:prepare -DreleaseVersion=%build.version%
./gradlew build -Pversion=%build.version%
mvn release:prepare is based on the maven release plugin. The equivalent is https://github.com/sunng87/cargo-release, and I don't know if it supports specifying the version on the command line.
I don't think it is equivalent for my use case. This is for releasing on crates.io and modifying Cargo.toml.
What I want is somewhat completely the opposite. I want to leave Cargo.toml as it is, not touch git (tags, push) at all ‒ that happens externally. I want to just build the binary, but the binary should be fed with a version different from the one in Cargo.toml, because for this use case, Cargo.toml is not the authoritative source.
What does
the binary should be fed with a version different from the one in
Cargo.toml
mean? Is it just env!("CARGO_PKG_VERSION") (and variants, like CARGO_PKG_VERSION_MAJOR) usage?
I'm not 100% sure there are no other places the version is passed into the compilation, but I guess so. I haven't tried yet (I might), but I guess cargo will set the env var unconditionally and overwrite it even if I pass it from outside.
Looking at built, it seems to be really nice and useful thing and I'll keep this in mind. But I don't think this will help me here ‒ this doesn't help me push the right version into eg. clap or structopt.
Can you not define the version in clap/structopt in terms of built_info::PKG_VERSION?
By default, they take the version from CARGO_PKG_VERSION. I could set that manually to built_info::PKG_VERSION but I don't see how that would help me, because built also takes the value from there.
You're right, sorry. I think the solution is to not override the environment variables, then.
Is there any progress? Also looking for the feature to externally manipulate the package version. (CI is setting the version, Cargo.toml should not be changed).
An Env-Variable would be nice: CARGO_TOML_PACKAGE_VERSION
This could also be done for every other part of the Cargo.toml:
- CARGO_TOML_PACKAGE_NAME
- CARGO_TOML_PACKAGE_AUTHORS
- ...
The place within the code is here: https://github.com/rust-lang/cargo/blob/master/src/cargo/util/toml/mod.rs#L68 There should be an env-override after parsing the toml and before checking for unused.
This should be analogous to https://doc.rust-lang.org/cargo/reference/config.html#environment-variables
I could add the implementation if we find a good name for env variables...
Adding another +1 - we're setting the version upon release, and it'd be great to not need to edit Cargo.toml each time
By way of another +1 and showing analogous functionality in other toolchains, the .NET SDK allows for overriding version numbers at the command line with commands like dotnet build /p:Version=1.0.1 (https://andrewlock.net/version-vs-versionsuffix-vs-packageversion-what-do-they-all-mean/). I've often found that overriding package versions that way can be useful in using CI pipelines to control how packages are versioned.
absolutely same for me. CI and stuff.
I was going to use "built"'s GIT_VERSION for that, but we build in docker containers not passing .git inside. So this info is lost.
This seems to be working on nightly right now: https://github.com/rust-lang/cargo/issues/6699
e.g. cargo build -Z unstable-options --config 'package.version="1.5.2"'
@Humandoodlebug
cargo build -Z unstable-options --config 'package.version="1.5.2"'
Will not work because --config overrides the cargo configuration, not the build definition in Cargo.toml
Editing the Cargo.toml isn't practical in some scenarios anyway - in Nix we don't really have conventional network access during build phases, and so it's not possible rewrite toml and regenerate lock files to workaround this.
I do see that there is a package macro that makes it easy to override the build version, falling back to the Cargo version, but it wouldn't be fun adding this to lots of Rust projects if Cargo could do it: https://docs.rs/git-version/latest/git_version/macro.git_version.html
Very surprising that this is still an open issue in feb 2022. 3 years have passed! probably lost/forgotten in the 1.2k issues. Theres no good way to programmatically replace the version in cargo.toml.
Could we @ some of the active maintainers to get some visibility on this issue?
+1 to adding version override in CLI like dotnet.
Is there a way to get cargo build to blend Cargo.toml version with [[bin]] name output? so that the cargo build --release makes a project-name-version.dll for example?
+1 here. I don't understand where the practice of statically specifying a version in a checked in manifest file comes from. I just want to release whatever is already checked in without having to commit again to change any manifests. It's very error prone unless you're using some fancy automated tools. It's so easy to overwrite a version that was already published previously if you release forgetting to change the manifest.
Being able to specify the version in the command line and completely omitting the version from the manifest would be my ideal workflow!
I suspect this is a large enough semantic change that this would need a major change proposal or an RFC and need a person to champion the proposal and implementation. It is unlikely for someone else to pick up and do all of that leg work on behalf of someone interested in this work. One challenge though is cargo is currently on a soft feature freeze and the cargo team is wanting to avoid distractions, including mentoring / shepherding, to be able to handle basic maintenance, finish currently committed work ,and to reduce technical debt so we can increase our capacity.
Things off the top of my head that would need to be figured out
- Should the version field be made completely optional?
- What are cargo's semantics without a version field and when no version is specified?
- What all commands should accept a version?
- How do we pass in versions, especially when running against multiple packages?
- How do we help people when they miss specifying the version for one of the multiple packages?
- How do we help workspace users with path dependencies?
- Currently, to publish a crate with a path dependency, it also needs to have a version requirement specified. That version requirement must match the crate its pointing to. Tools like
cargo-releasehandle updating version requirements of dependents so they don't become stale
- Currently, to publish a crate with a path dependency, it also needs to have a version requirement specified. That version requirement must match the crate its pointing to. Tools like
- Is there any prior art in other ecosystems that we can learn lessons from?
@epage This is a nice summary to figure out a considerate solution to this problem! Thanks for that!
It's a pity that it's unlikely that this effort would come from the core team, but it's understandable.
On the other hand, maybe an easier and good enough solution could be adding a flag (or env var) to pass the desired version to cargo so it could ignore the one specified in Cargo.toml? I'm absolutely unaware of cargo's internals, but if there's only one place where Cargo.toml is read to extract the version, then it seems like a pretty easy change to make.
In my (somewhat biased) opinion: there should be no static file defining the "current" version of a package at all. This should be determined at release time by looking up the VCS info or elsewhere.
On the other hand, maybe an easier and good enough solution could be adding a flag (or env var) to pass the desired version to cargo so it could ignore the one specified in Cargo.toml? I'm absolutely unaware of cargo's internals, but if there's only one place where Cargo.toml is read to extract the version, then it seems like a pretty easy change to make.
That still needs most of the questions I brought up answered.
In my (somewhat biased) opinion: there should be no static file defining the "current" version of a package at all. This should be determined at release time by looking up the VCS info or elsewhere.
Going back to your earlier comment
+1 here. I don't understand where the practice of statically specifying a version in a checked in manifest file comes from
I'd recommend doing a thought exercise to try to understand why so many ecosystems have static version fields. It will both give you more empathy for other ways of doing things but could also help when proposing solutions.
@epage
I'd recommend doing a thought exercise to try to understand why so many ecosystems have static version fields. It will both give you more empathy for other ways of doing things but could also help when proposing solutions.
I really did the thought exercise, and what I posted was my considerate conclusion :)
In my career (about 10 years) I've worked closely with several ecosystems: PHP, Python, JavaScript, Go (the longest time among others), and now I'm digging a bit into Rust. Among these, only Rust is so strict about having a static version in a file.
- Go doesn't have it at all, it uses VCS for determining the package version. By looking at tags normally. To be fair it doesn't have a package repository either, and uses VCS for that too (they do have a proxy cache now). And I'm a big fan of this decision.
- PHP doesn't need a static version file either.
- In JavaScript there're tools to either bump the version automatically, or dynamically pass it at release time (if I'm not missing anything).
- In Python there's a version field, but the config is defined in
setup.pyfile, which can run arbitrary code. So one could fetch the version from elsewhere if it's necessary. Again, if I'm not missing anything.
So in my experience the need for a static version file is roughly 50/50 across different ecosystems.
My conclusion is: I don't understand where the practice is coming from, nor do I see any usefulness in it. I never appreciated having it, and in my, admittedly personal and biased opinion, having a static version file checked in to the VCS has zero benefits.
I accidentally discovered this issue today, and to my surprise looks like I'm not the only person wondering about these design decisions, which makes me glad that I'm not the only weirdo complaining about it :)
And, don't get me wrong, I'm sure there's reasons why people decided to make static version files a requirement. It's probably because of my ignorance that I don't see it. But, being unable to overwrite it dynamically is what surprises me, and it's what I'm advocating for to be implemented in one way or another.
Trying to be useful, I give my opinionated answers to the questions you posted, in case it could be helpful if someone is willing to drive this forward:
- Should the version field be made completely optional?
Yes. I think it should possible to omit the version in Cargo.toml.
- What are cargo's semantics without a version field and when no version is specified?
Fail to release. Display a warning about a missing version providing options to either specify it in Cargo.toml or dynamically via flag or env var.
- What all commands should accept a version?
I'm unaware about the internal of Cargo, but probably any command that is reading Cargo.toml to extract the version in the current implementation.
- How do we pass in versions, especially when running against multiple packages?
I understand that this is the tricky one here. I see several options with different levels of user-friendliness:
- Require static version in case of a multi-package workspace.
- Be able to define versions for each package separately similar to
--version=package-1=1.0.0,package-2=2.0.0or anything like that. - Have an explicit integration with different VCS and have conventions about how to name tags (or similar concepts). E.g.
<package-name>/<version>. - Only allow releasing one package at a time (probably not practical when there're dependencies between local packages).
- Probably many other options and ways to do it...
- How do we help people when they miss specifying the version for one of the multiple packages?
Similar to the second answer above. Fail to release and provide guidance to fix it.
- How do we help workspace users with path dependencies?
- Currently, to publish a crate with a path dependency, it also needs to have a version requirement specified. That version requirement must match - the crate its pointing to. Tools like cargo-release handle updating version requirements of dependents so they don't become stale
That's even trickier. I don't have answer for it from the top of my head. Maybe in this case dynamic version release should be impossible. Or maybe it could depend on some hash identifier of the path dependency instead of semver (which I don't know if is a thing in Cargo at all).
- Is there any prior art in other ecosystems that we can learn lessons from?
I described my experience with different ecosystems in the previous comments: https://github.com/rust-lang/cargo/issues/6583#issuecomment-1131675725
My favorite one is Go. Let the VCS be the source of truth, have conventions on how to extract versions from VCS, have a proxy in front of the VCS + checksum database for public packages to provide availability, reliability, redundancy, security and other properties. I understand this is way different from many other ecosystems, and probably impractical for Cargo at this point.