rfcs
rfcs copied to clipboard
rust-lang
RFC: Syntax for embedding cargo-script manifests
This is for the T-lang side of #3502
Example:
#!/usr/bin/env cargo
---
[dependencies]
clap = { version = "4.2", features = ["derive"] }
---
use clap::Parser;
#[derive(Parser, Debug)]
#[clap(version)]
struct Args {
#[clap(short, long, help = "Path to config")]
config: Option<std::path::PathBuf>,
}
fn main() {
let args = Args::parse();
println!("{:?}", args);
}
In PL/Rust (a Rust subset that works as a postgres procedural language handler) we use a somewhat hacky syntax like (see https://github.com/tcdi/plrust/blob/main/doc/src/dependencies.md for example source)
[dependencies]
rand = "0.8"
[code]
use rand::Rng;
Ok(Some(rand::thread_rng().gen()))
I greatly prefer the approach in this RFC (and will likely push PL/Rust transition to it if the RFC is accepted) but it's probably worth noting as prior art.
I would like to consider alternative 7, the extended shebang. I don't think the backticks and the redundant cargo specifier should be necessary, producing this:
#!/usr/bin/env cargo
# [dependencies]
# foo = "1.2.3"
fn main() {}
I think this looks quite good. It's fewer lines than the proposed syntax, and mirrors both the shebang syntax and the attribute syntax.
I understand that people might want this to be generalizable/extensible, but this could suffice for now and any discussion about how to generalize the "info" portion can be left for a future discussion. If people think that it's important to make it generalizable right now, then I'd be interested to hear some concrete use cases.
EDIT: although I suppose an unmentioned downside of this is that # [dependencies] might look a bit too much like an attribute.
If you think this RFC a little bit further, then you could generalize this to make rustc ignore all ``` delimitered blocks, to say that rustc should just ignore all these sections and leave them to other tools similarly to how it does it already for the shebang. And then you'd see how similar triple backtick blocks are to /* */ comments. From there to just adding /* */ comments to the AST it's only a small step.
IMO it's bad style to add cargo specific extensions to Rust's syntax. Just say that ``` delimitered blocks are ignored by rustc and that parser implementations are suggested to add them.
In PL/Rust (a Rust subset that works as a postgres procedural language handler) we use a somewhat hacky syntax like (see https://github.com/tcdi/plrust/blob/main/doc/src/dependencies.md for example source)
Thanks!
I've added this as prior art and would love more interoperability (one of the stated motivations for the cargo script RFC)
If you think this RFC a little bit further, then you could generalize this to make rustc ignore all ``` delimitered blocks, to say that rustc should just ignore all these sections and leave them to other tools similarly to how it does it already for the shebang. And then you'd see how similar triple backtick blocks are to
/* */comments. From there to just adding/* */comments to the AST it's only a small step.
Is this meant to positive suggest something or to point out a slipper slope? I'm not too sure the intent.
IMO it's bad style to add cargo specific extensions to Rust's syntax. Just say that ``` delimitered blocks are ignored by rustc and that parser implementations are suggested to add them.
I'm not seeing how this is different than the suggested future state. We are starting with it being locked to cargo initially as we work out the design / usage and then can remove that restriction which is noted Future Possibilities.
I feel like 'front matter' would be a better syntax:
-
People know that it has to be the first thing in the file, which is even implied from the name. With
```cargo, it is not immediately obvious that it can't appear in the middle or at the end of a file -
It better conveys the purpose of the syntax: Front matter is used to add metadata to a file, while markdown code blocks are typically used to highlight some content as code
-
The proposed syntax only allows a subset of CommonMark code blocks. The CommonMark spec allows both
```and~~~as code fences; it allows code fences to be indented by up to 3 spaces; and it allows the closing code fence to be longer than the opening code fence. Deviating from the CommonMark spec could lead to confusion, and implementing the spec would add a lot of complexity. Front matter on the other hand is much easier to parse correctly (if the content is just parsed as "raw text" by rustc) -
The proposed syntax blurs the line between source code and documentation comments (where markdown is already supported). So people might incorrectly assume that doc-tests can appear outside of doc comments, or that
```cargoalso works inside a doc comment.
To address the mentioned downsides:
- Too general that people might abuse it
I actually like that it is more general (it doesn't use the word 'cargo'), so it can be used for other build systems as well. Rust the language shouldn't require any particular build system. What is considered 'abuse' is subjective; allowing front matter might enable various other use cases, and I wouldn't call them 'wrong' just because they're not what this feature was originally intended for.
- We've extended the frontmatter syntax, undoing some of the "familiarity" benefit
On the other hand, familiarity with markdown syntax might backfire because the semantics are so different here.
- People are used to YAML going in frontmatter (though some systems allow other syntaxes)
We could use +++ front matter instead of ---.
- Doesn't feel very rust-like
Care to elaborate?
Alternative 8: several loaders
#!/usr/bin/env cargo
[dependencies]
foo = "1.2.3"
#!/usr/bin/rustc
fn main() {}
Is this meant to positive suggest something or to point out a slipper slope? I'm not too sure the intent.
My intent was to suggest that the feature suggested by this RFC is quite close already to the solution I have been advertising multiple times already, using special kinds of comments.
I'm not seeing how this is different than the suggested future state.
The suggested future state is to allow multiple specific specifiers and have them mean different things to cargo.
What I am suggesting is that cargo should still error when it finds the backtick blocks in other places than the beginning and with other headers than cargo. But the error should not be at the parser level, but at the cargo level. rustc shouldn't hardcode the string "cargo", but just leave triple backticked things to preprocessors like cargo.
We are starting with it being locked to cargo initially as we work out the design / usage and then can remove that restriction which is noted Future Possibilities.
I don't think much is gained from restricting it at the parser level in rustc: I doubt that we ever want to do rustc specific processing of any of the backtick blocks. If you want to forbid them initially, you can also forbid them at the cargo level.
I'm fine with adding cli args to rust that were added because of cargo, but they were usually formulated quite generically. cargo passes a bunch of cli args to rustc that could be expressed much more concisely if one could just say "cargo mode" (e.g. json error output, etc). But it doesn't, and that's the better design choice. Similarly, here, I think it is fine to add triple backticks to the language so that tooling like rustfmt understands that it should not touch it, and so that parser developers like syn have something specified to work with.
I don't think that lexical structure, the "foundation" of the language, should be extended to support a build system feature.
From the language point of view the new added data is a comment. Rust already supports all kinds of comments, regular and doc, at crate and module roots, the new data should also fit into the existing setup.
It can always be reconsidered if the feature proves to be super popular in the future, in which I'm not sure.
@est31
If you want to forbid them initially, you can also forbid them at the cargo level.
I don't think that's possible, because cargo doesn't look at the Rust source files, only the manifest (except when the manifest happens to be a Rust script).
The latest comments just got me thinking: even if we don't want to use comment syntax, does rustc really need to parse the header? Could cargo possibly just pass an offset (idk if bytes or lines is better) into the file at which rustc should start parsing? That would allow experimentation with all kinds of build system headers in different build tools without further changes to rustc.
I don't think that lexical structure, the "foundation" of the language, should be extended to support a build system feature.
From the language point of view the new added data is a comment. Rust already supports all kinds of comments, regular and doc, at crate and module roots, the new data should also fit into the existing setup.
It can always be reconsidered if the feature proves to be super popular in the future, in which I'm not sure.
We need a block of text that is generally ignored by the language but has special semantics. We already have a similar concept baked into the language, doc comments. Those are for a specific set of tools/applications (rustdoc). We are needing something similar. We can make a new form of structured comment which would be a breaking change and require an edition to start using which puts a specific deadline on this feature or requires us waiting 4 more years for this. Another problem with basing it off comment syntax is they need a full rust lexer / parser to extract which is fine for rustdoc as it needs to be grammar aware but is terrible for random tools. We could restrict this new structured comment syntax but that would break from users expectations in working with them.
The latest comments just got me thinking: even if we don't want to use comment syntax, does rustc really need to parse the header? Could cargo possibly just pass an offset (idk if bytes or lines is better) into the file at which rustc should start parsing? That would allow experimentation with all kinds of build system headers in different build tools without further changes to rustc.
If we did something like that, I can only see that existing for the purpose of rust compiling bundled code or as a nightly-only feature for experimenting. For this to be used widely by tools, this is making a wild west where we throw interoperability out the window.
Interoperability is one of the goals with cargo-script. We need a syntax that we can get cargo, the play ground, rust analyzer, random editor plugins, etc to be able to work with.
I also feel like doing something like this means the file is no longer "rust" and we'd need a new extension. I don't think that cognitive load is worth it (and there is little precedence for doing so among the various solutions). This also has an ecosystem cost as tooling has to be updated and percolate out with support for the new extension. The delays from this also impacts the usability of this feature.
@Aloso
If you want to forbid them initially, you can also forbid them at the cargo level.
I don't think that's possible, because cargo doesn't look at the Rust source files, only the manifest (except when the manifest happens to be a Rust script).
Yeah and only there do these things have meaning, or we care about them. I think that's the setup we should aim for.
@jplatte
Could cargo possibly just pass an offset (idk if bytes or lines is better) into the file at which rustc should start parsing?
In fact, there is such parameters already, namely env vars UNSTABLE_RUSTDOC_TEST_LINE and UNSTABLE_RUSTDOC_TEST_PATH. Named that way as they were added for rustdoc.
Note that even with these parameters, one still needs to think about rustfmt, and there is some independent parsers of rust as well.
@epage
We need a block of text that is generally ignored by the language but has special semantics. We already have a similar concept baked into the language, doc comments. Those are for a specific set of tools/applications (rustdoc). We are needing something similar.
100% agree with this.
We can make a new form of structured comment which would be a breaking change and require an edition to start using which puts a specific deadline on this feature or requires us waiting 4 more years for this.
No edition is needed. cargo-script is a novel feature, so in theory, any input to cargo-script would be possible. If you want, you could make it parse java :). It would be nice however if rust with embedded manifests would still be something that e.g. rustfmt can process. With comments, code paths/tools outside of cargo-script would just treat them as comments. Say when this is found in a multi-file package, or via some other build system, those will treat these sections as comments and ignore them.
Another problem with basing it off comment syntax is they need a full rust lexer / parser to extract which is fine for rustdoc as it needs to be grammar aware but is terrible for random tools. We could restrict this new structured comment syntax but that would break from users expectations in working with them.
The restrictions you would need wouldn't be larger than the restrictions existing for //! rustdoc tags already, or the restrictions that this RFC puts on ```s. I cannot speak for @petrochenkov but personally I would be really happy if every time this RFC mentions triple backslashes, you'd replace the mention with /*% or */ for where you would have placed the closing triple backslash.
The rustc_lexer crate is 100% stable Rust so that re-use is easy. I think it has everything cargo needs for parsing the /*% ... */ blocks.
Cargo can then, for future compat sake, make a full lexing run over the .rs manifest and error for any /*% ... */ block that is not at the top of the file right after the shebang. rustc would still treat them as comments, as would rustfmt. Eventually, rustfmt could gain support to format these manifests like is proposed for Cargo.toml, at which point rustc would need some minor parsing support, but that could be totally opportunistic and not result in an error if something invalid is found.
No edition is needed. cargo-script is a novel feature, so in theory, any input to cargo-script would be possible. If you want, you could make it parse java :). It would be nice however if rust with embedded manifests would still be something that e.g. rustfmt can process. With comments, code paths/tools outside of cargo-script would just treat them as comments. Say when this is found in a multi-file package, or via some other build system, those will treat these sections as comments and ignore them.
While cargo script is a novel feature, my comment was about giving syntactic meaning to these comments like we do for doc comments. That applies everywhere.
The restrictions you would need wouldn't be larger than the restrictions existing for //! rustdoc tags already, or the restrictions that this RFC puts on ```s.
We can put those kinds of restrictions on them but then we are breaking users expectations for formatting of comment syntax.
The rustc_lexer crate is 100% stable Rust so that re-use is easy. I think it has everything cargo needs for parsing the /*% ... */ blocks.
If we need to pull in rustc_lexer, then that doesn't work for random editor plugins. It also adds logistical problems (e.g. cargo gets published but that crate isn't regularly published, publish order between the two repos, etc)
@est31
The suggested future state is to allow multiple specific specifiers and have them mean different things to cargo.
I had over looked this comment before. I think I was unclear; that is not a suggested future state but a "in case we need it". I tried to clarify that in 97ca09a
While cargo script is a novel feature, my comment was about giving syntactic meaning to these comments like we do for doc comments. That applies everywhere.
To the end user? Yes. But the "end user" view of a language is broken all the time. Say when todo!() is added, suddenly each place that uses unimplemented!() is legacy. the times someone has /*% */ comments in their code should be quite limited. There is no need for an edition boundary there, but even if, we have the 2024 edition around the corner.
While cargo script is a novel feature, my comment was about giving syntactic meaning to these comments like we do for doc comments. That applies everywhere.
To the end user? Yes. But the "end user" view of a language is broken all the time. Say when
todo!()is added, suddenly each place that usesunimplemented!()is legacy. the times someone has/*% */comments in their code should be quite limited. There is no need for an edition boundary there, but even if, we have the 2024 edition around the corner.
I'm not referring to user perception but how we parse the syntax.
And yes, we have an edition right around a corner but it would be very limiting if a feature like this can only b used with edition="2024" or newer.
I think it's worth noting that the proposed syntax exactly reverses the meaning of fenced code blocks: these "turn code off" but in markdown they turn code on.
I also think it's worth leaving a fly-by comment at least to advocate for not using markdown fenced code blocks (I will try to respond; personal life means I may not, thus fly-by. I care but time).
It seems to me that the biggest motivation for this is perceived learnability, but the effect is that in all contexts that use markdown not taking the extra step of escaping means breaking the code. This is called out in the RFC but if I don't have a lot of time and I want to give someone a repro of something I want to just copy/paste it and not remember that there's this extra specific Rust step. Someone has to deal with that. I am almost sure that even Rust veterans reporting issues who happen to be in a hurry at the moment are going to forget. It's an annoying chore for the reporter or an annoying chore for the maintainer and it's often already hard enough to get people reporting/finding time to deal with things. Of course if this is ever extended that goes from annoying to potentially much more problematic: should it ever not only be used at the top of files, then it's time to play hunt the problems and, from that perspective, begins closing doors to any future extensions. The only advantage I can see for making this choice is learnability, and I'm skeptical that the effect is as large as it seems (for one thing Jira doesn't really use markdown as it is; the implicit assumption is that a new coder knows markdown).
The static site generator alternative of using --- is just as learnable: "this is like insert-blog-thing, we will learn about all the content later". But without the foot-guns.
I will add that I was involved in another discussion like this back on the un-indented strings RFC, was told in that discussion that fences could still be escaped, and had to look it up for this comment anyway. Not only for fences, but how to write ``` by itself in a sentence. Not to mention that markdown isn't exactly a standard beast so who knows if what I used on GitHub works elsewhere? I don't personally know of problematic implementations but given the lack of standards/conformation to spec, are we even sure there isn't some issue tracker somewhere where "overloading" markdown like this isn't possible because it doesn't respect the escaping? It seems to me that the unspoken assumption is "if x claims to be using markdown, then x will work in this specific way" and that's never really been true.
TLDR: it really feels messy when there's other non-overloaded syntactical constructs that would work, most of which are even in the RFC. Learners only learn it once, the rest of us deal with it forever
TLDR: it really feels messy when there's other non-overloaded syntactical constructs that would work, most of which are even in the RFC. Learners only learn it once, the rest of us deal with it forever
If advocating for another solution, please also address the concerns with that format.
it's obviously up to the cargo team to decide on the final format, though IMHO the doc-comment alternative would be much more beneficial and easier to support across the ecosystem since no parser changes are necessary to support files with an embedded manifest.
Sorry, forgot to call this out earlier but I think an important note for anyone reviewing this RFC is that this is not a cargo team decision but a language team decision. There are subjective aspects of this. There are aspects where people will prioritize things differently than others. I've geared things towards what I expect will work for the language team and will adjust as they direct otherwise. This does not mean that input isn't useful but it has already improved the RFC and can help provide more perspectives for my recommendation and for the language team in their decision. Let's make sure we recognize that multiple experienced, well reasoned people can come to different conclusions on this and it might not look like our ideal (myself included).
I would be happy with any other option. I'm not addressing specific alternatives because all other alternatives have two extremely useful properties:
- There are no questions over what a markdown implementation does.
- At 2 AM or whatever, when I'm in a hurry, etc, and I have a repro or something like a useful devops script/utility, ctrl+a ctrl+v works for me and also doesn't introduce problems for the maintainer because I forgot a step which is unnecessary in any other language I know (or reversed, if I'm the maintainer, etc).
I am not aware of any other language which knowingly decided that requiring editing the program in order to paste into issue trackers etc. would be necessary because it chose to embed a subset of markdown because (as the RFC currently reads to me)it decided that was the most learnable option. And indeed, here is what I would consider a likely problem with learnability anyway:
- Student 1 figures out something neat or a bug, or whatever and wants to share.
- Student 1 goes to whatever's using markdown to share with student 2, but makes the mistake of not fixing the markdown.
- Student 2 copy/pastes and now the program doesn't work and they have to fix it.
I'm not the person to ask about readability. I'm actually blind and in addition I'm one of those weirdos who doesn't find C++ that bad so any opinion I have there is likely not great. For the record I favor --- because it's the same as a code block (e.g. can copy/paste manifests back and forth) but will embed in any text markup I'm aware of as long as it's inside a code block (which we must do for other code anyway). I'm not a huge personal fan of prefixes because I can't easily select the non-prefixed text with the mouse, but eh, the 99% matter so if that's what people like go for it. My preference for isn't what I consider important, it's my strong preference against.
As an example of a "fast" (e.g. ctrl+a ctrl+v) system where I'd expect this to break, I've seen Slack treat markdown in a copy/paste as formatting. I believe you have to toggle something on to get markdown (I forget; it's useful for me because I can type formatting without clicking around). Snippets exist and clearly are the right answer but me and my coworkers don't bother with them half the time.
One great thing about Rust is that while it has edge cases, most of them don't compile and most of the rest don't break the program; markdown fences, by contrast, silently break sharing with others.
@bestouff that is a slight variant of one of the options and I'd recommend talking to the downsides if proposing it.
Note that you left off the manifest being a string. That is dependent on whether the attribute parsing code can correctly handle TOML syntax (now and in the future) being embedded with in it which I've not tried to verify. In using strings, unless we try to shift people's style to single quotes, it will likely require a raw string literal.
@epage Yes sorry I found it simpler to learn than Alternative 3 but not sure it's rustc-parseable. I removed my comment but you were too quick !
For the record my proposal was:
#!/usr/bin/env cargo
#[cargo(
[dependencies]
clap = { version = "4.2", features = ["derive"] }
)]
use clap::Parser;
#[derive(Parser, Debug)]
#[clap(version)]
struct Args {
#[clap(short, long, help = "Path to config")]
config: Option<std::path::PathBuf>,
}
fn main() {
let args = Args::parse();
println!("{:?}", args);
}
With my lang hat on, I don't see a reason we should RFC a feature that only allows cargo front matter, without specifying a path to generalizing it to other tooling. If we want to be conservative in what we stabilize, let's approach that in the stabilization rather than in the RFC.
At the language level we should acknowledge that not all projects get to use cargo, and the generalization here seems trivial to do in the RFC. Note that I'm fine with the RFC being conservative in other ways (only allowing one, right after the shebang, etc).
Taking off my lang hat now – using cargo sure makes it awkward to extend to embedding a second front matter with Cargo.lock. It'd be nice if the RFC addressed that. (Maybe just say "we can use cargo-lock for consistency with cargo"?)
With my lang hat on, I don't see a reason we should RFC a feature that only allows
cargofront matter, without specifying a path to generalizing it to other tooling. If we want to be conservative in what we stabilize, let's approach that in the stabilization rather than in the RFC.At the language level we should acknowledge that not all projects get to use cargo, and the generalization here seems trivial to do in the RFC. Note that I'm fine with the RFC being conservative in other ways (only allowing one, right after the shebang, etc).
Had considered loosening this up before any more official word from the lang team and realized there are syntax questions we don't really have an answer to (and our source of inspiration doesn't have good answers for). I expanded on this and also gave a suggested starting point for syntax if we decide to bring those decisions into this RFC.
Taking off my lang hat now – using cargo sure makes it awkward to extend to embedding a second front matter with Cargo.lock. It'd be nice if the RFC addressed that. (Maybe just say "we can use cargo-lock for consistency with cargo"?)
I figured what the string should be would be best left for #3502 (as noted here) which goes into more detail. Feel free to add your thoughts there!
If you feel that is a t-lang or a joint t-lang + t-cargo decision, we can talk about it!
So, I've been following along a bit and had a couple of comments.
I am also an instructor and a CS Ed researcher. I think that no matter what we do, we are going to increase the cognitive load for students unless it's extraordinarily explicit in its immediate interpretation. I don't think that code fences give you that.
The reason for that is 2-fold.
First, the code fence means that you have no distinguishing factor inline with the embedded toml. This means that students will basically have to switch modes when they are looking at different parts of the file. The only way this becomes even slightly viable is with syntax highlighting but even then, now you have to mix syntax highlighting for toml (which isn't a lot but does exist) and rust into one file adding to the complexity of ensuring you have a diverse enough color palette to maintain good contrast and instant recognition.
Second, code fences are not explicit in their intent. There's nothing about them that inherently says, oh, by the way, we have an embedded file here. I don't think anyone has really managed to do that phenomenally from what I've seen but I will grant that I haven't looked that hard for languages that embed other languages inside of them. JSX, Bash here docs, and Doc comments come to mind but that's all I've got currently.
My personal opinion is that if the goal is to ensure minimal increase in cognitive load, there are a few general approaches that I would suggest:
-
Comments. Already discussed this extensively, my two cents is that comments already tell students and programmers that this isn't something that will be in the code. You can disregard it unless you're trying to do something with it at the moment. I will add that a danger I see with comments is that sometimes people see comments as vestigial and ignorable while an embedded cargo.toml is anything but that.
-
If we're already adding support for the bash shebang, why not make it possible to embed files beneath it as well. Something like
#!/blah
#= cargo.toml
#+ stuff
#+ goes
#+ here
But a bit more thought out and considered in the extension to the syntax.
-
Allow specifying dependencies on the use statements instead and have that be collected into a file format that can be used by cargo or other build tools.
-
Extend the extern keyword to allow either externing an embedded file for users to link to or consume somehow or to simply allow for users to again specify dependencies by saying extern crate lib with name and version and features and stuff.
All of these feel to me like they address the idea of remaining explicit and not locking people into cargo.
Final thoughts, even if we do use code fences or whatever, many students don't fully comprehend what a use or import or whatever your language adds to support including libraries and other code until much later. They just type it out because they know that if they don't have it, their code will break. I don't think it's wrong to simply provide a well documented template file to students at the beginning. Make sure it covers the common libraries they will be using. Then, through the semester/quarter you take opportunities to come back to it and refine their understanding of why the things in the template are there until they can begin adding dependencies on their own.
As it is, as an instructor, I prefer to autograde students programs and I would never in a million years use their dependency list. That's a quick way for a clever student to decide that they want to access the container that the autograder is running in and then try to either manipulate it, access the answers when they shouldn't using something I'm not familiar with, or do something more malicious or mischievous. It's the same reason playground doesn't let you use your own cargo.toml file. It's just not worth the risk. Especially when the machines I'm using are either mine, my universities, or a free services. All of which would probably not take kindly to a student mucking around inside of their stuff or not take kindly to what may appear to be a spike of usage from me.
@ProfDoof thanks for taking the time to share those thoughts!
It looks like I left out of this RFC the guiding principles for why we chose to embed the manifest. That is now included in 6d8782c.
We are not optimizing for one specific angle (e.g. university student cognitive load) but trying to balance out several. Annotating use or extern is a solution. Some examples of problems we run into with that
- You still need a way for specifying other fields from a manifest, if desired, like
package.edition - We'd need to have separate spaces for documentation for these very different approaches
- Answers on stackoverflow, reddit, etc would not be immediately applicable but the users would need to understand both and how to translate between them
- Conversion between the two systems is more work and would likely require machine assistance (my general rule for design is machines are to augment but not be required)
- Cargo would need to parse arbitrary rust syntax. We'd either need to rely on rustc somehow to extract the data (heavy, slow process) or we'd need to use an alternative parser (e.g.
syn) which will then mask any of the high quality rustc errors, replacing them with lower quality ones
It'sthe same reason playground doesn't let you use your own cargo.toml file.
Hmm, I thought I had talked with @shepmaster about support for this in the playground and they were open to it but the only related thread I see doesn't include that.
Had a recent observation in my use of #3502 that I thought would be relevant to share.
Its been around 1.5 years that I've been using this feature regularly, first as rust-script, then cargo-script-mvs, and now I run my scripts out of cargo master. Pretty much every bug report for clap or other libs I maintain gets re-written into a script. I probably average 2-3 scripts per month as a conservative estimate (I occasionally wipe dump/ so I can't say for sure).
I started off using single-line doc-comments and thought that was fairly lightweight, especially compared to the macro options. However, I still ended up copy/pasting from previous script and I blamed it on the whole experience (she-bang, file permissions, commonly used dependencies, embedded manifest syntax) but felt there wasn't any more room to get this lighter to avoid the copy/pasting. For context, one of the limitations I put on this effort is to not consider script scaffolding (e.g. cargo new) support until we have everything designed, forcing us to use this regularly to see how successful we were in making this "lightweight".
Some people suggested multi-line doc-comments should be the recommended style and so I switched to that to see what the experience would be like first hand. It made copy/pasting manifest content in-and-out easier but I was still copy/pasting the scripts themselves.
I recently wrote my first script using code-fences (which this experience would equally apply to more traditional frontmatter). I didn't copy/paste and it felt ... good. Something about it shifted the dynamics in me enough that I'm ok with handling the shebang and permissions and writing these from scratch.
Yes, I am the author of this RFC but for that entire time I was using doc comments, I was advocating for that solution, so I don't think there is much personal bias influencing this as the biases during each period were the same, I went into with the assumption that I was still going to need to copy/paste, and I'm observing what actions I took in response and. Each person will also experience that tipping point between templates and hand-written differently.
No problem! I'm really enjoying seeing where this is going!
I felt like I should chime in with my 2 cents since I saw multiple references to concerns from an instructor and wanted to offer up a different viewpoint.
And just to clarify what I meant with extern, I was more thinking
pub extern "cargo" static depfile: &str = "your actual toml here";
Or something silly like that. Basically allowing you to embed files and then tell rustc to do something with them while also still maintaining something that looks like rust. But I'm not familiar enough with the internals to know how extern actually works and what it's doing to know whether something like that would be viable and/or simple to accomplish. Once I'm done with my PhD, maybe I'll finally have time to learn about the internals of rust.
I think that gives you the extendability for the future as well because now part of rustc can be rustc communicating with build systems.
However looking at it now, I can see the amount of extra work that would probably and and it's probably not worth it.
It could also just be an attribute attached to a static string if that can make a temporary file somewhere that build systems or other consumers of the embedded files can easily find.
The nixpkg style could also be interesting but could require a lot of buy in from the community and a lot of work implementing something that cargo already does wonderfully so seems out of scope. Just a thought that popped into my head.
If there's a good spot to chat I'd love to discuss the concerns of the instructor more as that's more my background.
(Also super embarrassed that I missed the extended shebang in the alternatives already.)