compiler-team
compiler-team copied to clipboard
rust-lang
Arbitrary annotations in compiletest
Proposal
This MCP proposes to add a new compiletest declaration, annotations, that allows the person writing the test to add arbitrary annotations to the test. While those annotations won't impact the test execution, compiletest will emit a JSON file at the end of the test run containing all the annotations it found, and the checksums of all the files related to those tests.
For example, if you add this comment to the bastion of the turbofish test:
// annotations: foo
...compiletest will generate a JSON file like this at the end of the test run:
{
"parser/bastion-of-the-turbofish.rs": {
"annotations": ["foo"],
"sha256": {
"parser/bastion-of-the-turbofish.rs": "840ae8552ba944ae468df113f1ee4123b260fbb32a43f71af907950586848333"
}
}
}
In Ferrocene we need annotations because we need to establish a "traceability matrix" between some requirements and their related tests, to document that all of our requirements are properly tested and which files are part of the test. Compiletest annotations would generate all the necessary information we need on the test suite side of things, allowing our traceability matrix tool to parse the JSON output.
Note that even if this MCP is accepted we likely won't upstream the annotations themselves, as they'd reference internal documents and would be impossible to maintain outside of Ferrocene.
Still, the generic annotations mechanism could be useful in the Rust project as well, and we'd love to upstream it if this MCP is accepted. Some uses of annotations I could see are annotating which tests would need to be un-ignored when a target approaches Tier 1, or annotating tests with the issues they fix.
Mentors or Reviewers
@Mark-Simulacrum
Process
The main points of the Major Change Process are as follows:
- [x] File an issue describing the proposal.
- [ ] A compiler team member or contributor who is knowledgeable in the area can second by writing
@rustbot second.- Finding a "second" suffices for internal changes. If however, you are proposing a new public-facing feature, such as a
-C flag, then full team check-off is required. - Compiler team members can initiate a check-off via
@rfcbot fcp mergeon either the MCP or the PR.
- Finding a "second" suffices for internal changes. If however, you are proposing a new public-facing feature, such as a
- [ ] Once an MCP is seconded, the Final Comment Period begins. If no objections are raised after 10 days, the MCP is considered approved.
You can read more about Major Change Proposals on forge.
Comments
This issue is not meant to be used for technical discussion. There is a Zulip stream for that. Use this issue to leave procedural comments, such as volunteering to review, indicating that you second the proposal (or third, etc), or raising a concern that you would like to be addressed.
This issue is not meant to be used for technical discussion. There is a Zulip stream for that. Use this issue to leave procedural comments, such as volunteering to review, indicating that you second the proposal (or third, etc), or raising a concern that you would like to be addressed.
cc @rust-lang/compiler @rust-lang/compiler-contributors
