RFC: cargo-script

[epage]

Rendered FCP

eRFC rust-lang/rfcs#3424 was previously approved without any direction on syntax or behavior having been decided. This RFC is to finalize the decisions made through the implementation to prepare the way for stabilization as we work through the remaining tasks in rust-lang/cargo#12207

#3503 covers the T-lang side of this conversation

Example:

#!/usr/bin/env cargo
---
[dependencies]
clap = { version = "4.2", features = ["derive"] }
---

use clap::Parser;

#[derive(Parser, Debug)]
#[clap(version)]
struct Args {
    #[clap(short, long, help = "Path to config")]
    config: Option<std::path::PathBuf>,
}

fn main() {
    let args = Args::parse();
    println!("{:?}", args);
}

$ ./prog --config file.toml
Args { config: Some("file.toml") }

Note: most of this is available behind -Zscript (see rust-lang/cargo#12207 for limitations). Support for the code-fence frontmatter was added in rust-lang/cargo#12681 and it may be a couple of days before cargo's submodule in rust-lang/rust gets updated and built into a nightly.

[ggggggggg]

"These will be accepted with just like Cargo.toml files" doesn't read right.

[bstrie]

The frontmatter syntax wouldn't need to be added to Rust's grammar, correct? In other words, the frontmatter parsing would be a preprocessing stage solely within Cargo, which would strip out the frontmatter before passing it to rustc?

[jhpratt]

I think it should be forbidden to publish a script crate without an edition specified. Otherwise we're inevitably looking at a crate that builds when it's published and fails to build later on.

[epage]

The frontmatter syntax wouldn't need to be added to Rust's grammar, correct? In other words, the frontmatter parsing would be a preprocessing stage solely within Cargo, which would strip out the frontmatter before passing it to rustc?

This is what cargo master is doing but

• It is no longer rust which is a priority
• Error messages, cargo-metadata, etc will expose the lie and I do not want to hack around every case.

[epage]

I think it should be forbidden to publish a script crate without an edition specified. Otherwise we're inevitably looking at a crate that builds when it's published and fails to build later on.

Publish is out of scope for this RFC. There are other aspects of the design that will play into this that a conversation will need to consider. For example, I expect it to turn it into a mult-file package which will require setting the edition.

[jhpratt]

Publish is out of scope for this RFC

I now see that note towards the top. It was the mention of publish = false that left me concerned for the reason stated.

[bstrie]

This is what cargo master is doing but

Does the "but" here indicate that the goal would be to get this syntax into Rust's grammar? If so, then I'd like to consider the alternative of stuffing this data into comments rather than introducing new syntax.

[epage]

This is what cargo master is doing but

Does the "but" here indicate that the goal would be to get this syntax into Rust's grammar? If so, then I'd like to consider the alternative of stuffing this data into comments rather than introducing new syntax.

#3503 is the RFC for the syntax, as referred to in the PR description and in the embedding syntax section of rationale.

This solution was what we came up with in discussions with a subset of the cargo team, rust educators, and the language team. It was started and finished on zulip with the middle being handled at RustConf.

[epage]

I now see that note towards the top. It was the mention of publish = false that left me concerned for the reason stated.

Ah, yes, I missed some sections when removing publish support from the RFC.

[0x61nas]

Did we can use a cargo script in another cargo script?

If I have more than one cargo script and most of them have common functions/structs/enums/types or traits, it will be more convenient if I can put them into a separate cargo script that doesn't have necessary have a main function and it just defines these common things and export them as pubic items, so I can use them somehow in the other cargo scripts.

I think that we can't use the use keyword (or at least its syntax) because we don't have the crate concept here, and we might want to use the absolute path.

[epage]

Did we can use a cargo script in another cargo script?

If I have more than one cargo script and most of them have common functions/structs/enums/types or traits, it will be more convenient if I can put them into a separate cargo script that doesn't have necessary have a main function and it just defines these common things and export them in the pubic so I can use it somehow in the other cargo scripts.

I think that we can't use the use keyword (or at least its syntax) because we don't have the crate concept here, and we might want to use a cargo script to use its absolute path.

At this point you should either

• Make a common "script-tools" package and have a dependency on that (could be a path dependency)
• Use a multi-file package

[epage]

In case people find it of interest, the Python community has a parallel effort to this with two competing solutions and Microsoft paid for some user interviews to be done. Maybe I overlooked something but unfortunately there wasn't much that seemed relevant to us as it seemed more focused on the differences in formats inside of the frontmatter and is more akin to manifest subset vs cargo-deps: difference from rust-script.

[timClicks]

Thank you for your diligence and patience to take on so many points of view.

[ijackson]

I would like to see some discussion (in Future, at least) of how a script could contain its lockfile (or equivalent information).

[epage]

I would like to see some discussion (in Future, at least) of how a script could contain its lockfile (or equivalent information).

We have a whole section on that. If there are specific questions on it or areas you feel are missing, feel free to open discussions on the relevant lines.

[epage]

To the best of my understanding, all of the concerns have been acknowledged and are addressed within the RFC body, one way or the other.

With the syntax approved, I think this is ready to move forward.

@rfcbot fcp merge

[rfcbot]

Team member @epage has proposed to merge this. The next step is review by the rest of the tagged team members:

• [x] @Eh2406
• [x] @Muscraft
• [ ] @arlosi
• [x] @ehuss
• [x] @epage
• [x] @joshtriplett
• [x] @weihanglo

No concerns currently listed.

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

[gmorenz]

Is this the right RFC to define how argv[0] is set?

Currently cargo -Zscript passes it as current_exe (e.g. /home/greg/.cargo/target/42/2ab348f3d46e86/debug/test-), I think it makes more sense to pass it as the script itself, treating the file that the user ran as the executable. Mirroring the behaviour of a python/bash/... script.

[epage]

@gmorenz yes. This is also being discussed in rust-lang/cargo#12870

[rfcbot]

:bell: This is now entering its final comment period, as per the review above. :bell:

[jwnrt]

I'm not sure if this is worth raising here but I just started trying the frontmatter syntax and had the following problem:

The current syntax of a script does not allow comments between the shebang line and the frontmatter since this would need more sophisticated parsing to handle. Some of the projects I contribute to require a copyright/license header in a comment at the top of the file (also after an optional shebang).

I can understand this not being considered an issue since (A) the copyright comments are a constraint introduced by external projects and not intrinsic to Rust, (B) it's more of a cultural problem than a technical one and could in theory be waived by linting tools (e.g. REUSE allows copyright headers to go anywhere), and (C) the scripts can be converted to full Cargo packages if checked into a repo with such a requirement.

Still, it would be nice to be able to have a copyright header at the top of cargo-scripts.

[gmorenz]

@jwnrt

Would the licenses allow for

#!/usr/bin/env cargo
---
# License Goes here.
# In a TOML comment.

[package]
...

?

[jwnrt]

If the linting script is changed to ignore --- I think that's the closest you could get, yeah, thanks

[coolreader18]

Coming from TDCIC _{(This Development Cycle in Cargo)} - wrt choosing the profile to run a script with, what about the suggested run profile, but called script? That way there's no need to change the behavior of cargo run, and it's clear what it's for.

---
[profile.script]
inherits = "release"
---

[epage]

In the spirit of keeping this focused, so long as we can make a change later, let's do that rather than ensuring this RFC has every feature in it.

[ijackson]

I came here via TWIR, the blog post and https://github.com/epage/cargo-script-mvs/issues/154#issuecomment-2112059383. I have two concerns.

Firstly, conflating the script filename namespace with the cargo subcommand namespace is IMO a serious mistake. There's extensive discussion in the RFC about how to handle it, but there is a much better answer with prior art from many other script interpreters:

cargo should require an option to indicate that the argument is a script name. So the shebang should be #!/usr/bin/cargo --bikeshed or #!/usr/bin/env -S cargo --bikeshed.

Prior art:

• To write a script for jq, use the -f/--from-file option. (#!/usr/bin/jq -f) The default is to treat the argument as jq filter text (ie, source code in the jq language, rather than a filename). awk has similar behaviour.
• To run a file containing Emacs Lisp as a script, use emacs --script. So #!/usr/bin/emacs -script. (The default behaviour is of course to run emacs as an interactive text editor and treat the argument as a file to edit.)
• Debian's debian/rules is a Makefile. make's default behaviour is to treat the arguments as targets and look for a Makefile in the cwd (which is analogous to cargo's default behaviour). So, debian/rules starts #! /usr/bin/make -f.
• To run a file containing a PostgreSQL script, use psql -f, since the arguments are the database name etc.
• If you want to write a #! script for execution by gdb, you must write #!/usr/bin/gdb -x or similar. (By default the argument is the program to debug.)

I'm sure there are many other examples.

[ijackson]

My second concern is:

The .rs extension should be optional.

Encoding the implementation language of a script in its filename is an antipattern (notwithstanding that it seems to be extremely common).

The name of a script is how you invoke it. So it is the script's API (along, of course, with its behaviour). The language its written in is an implementation detail.

This is not merely a theoretical problem of principle. It causes practical difficulties. It is very common to start out with a script in a simple language, and then to rewrite it in a more sophisticated language as the demands on it grow.

If the filename mentions the implementation language, this can't be done without updating every call site, possibly in different repositories, or even public documentation, and possibly teaching many humans to run a different command.

In the case of cargo script, insisting on a .rs extension makes it impossible to replace a script foo that once started #!/bin/bash and now starts #!/usr/bin/env python3, with a Rust implementation.

I don't think this is a blocker for the cargo script feature, since the .rs restriction could always be relaxed later. But it is a significant limitation that ought to be discussed in the RFC. (The RFC text does discuss possible other extensions, and seems to have been written on the (false) assumption that an extension is necessary at all.)

[bjorn3]

So the shebang should be #!/usr/bin/cargo --bikeshed or #!/usr/bin/env -S cargo --bikeshed.

macOS doesn't support -S as noted in the "Naming" section. #!/usr/bin/cargo doesn't work with non-system installs of cargo as is the most common.

[epage]

@ijackson

• As mentioned env -S is not universal as covered in the RFC
• At least for myself, that prior art feels "obscure". I have never written or edited a file with a shebang like in those cases.

[rfcbot]

The final comment period, with a disposition to merge, as per the review above, is now complete.

As the automated representative of the governance process, I would like to thank the author for their work and everyone else who contributed.

This will be merged soon.