ty icon indicating copy to clipboard operation
ty copied to clipboard

Improve the developer experience of working on ty

Open AlexWaygood opened this issue 4 days ago • 3 comments

We discussed in a recent ty planning meeting that it can sometimes feel a bit painful to work on ty right now as a developer. Things that we discussed which might help include:

  • Improving the typing-conformance workflow. All lines in the typing-conformance test suite where a type checker should emit an error have an # E comment next to them; a type checker is not expected to emit an error on any other lines. It should therefore not be too hard to improve this workflow so that it automatically tells you whether a diagnostic being added or going away is a good or bad change.
  • Make the ecosystem-analyzer and/or mypy_primer workflow more "positive". If you've worked for several days on a PR, it can be a bit dispiriting to then be greeted with an overwhelming list of new diagnostics being emitted on the ecosystem that you have to dig through and analyze. It would be nice to also have more positive feedback, such as: "You reduced the percentage of @Todo or Unknown types in the ecosystem by 37%, well done!".
  • Make the ecosystem reports easier to analyze:
    • Could we run other type checkers on this code as well, and also have an indication in the report about whether our diagnostics are in line with those of other type checkers? It's often useful to know that other type checkers also emit an error on a certain line of code.
    • Fix https://github.com/astral-sh/ty/issues/1670

We didn't discuss it explicitly in the meeting, but I'd also add to this list:

  • Reduce the number of generated files that it's necessary to check in when making a PR. We're adding/removing/renaming/redocumenting diagnostics at a very quick velocity right now. Having to run cargo dev generate-all increases the amount of time it takes to prepare a PR (I have to compile a whole different crate locally before submitting it), and it's hard to remember to do it, so you often end up filing the PR, then only remembering to run cargo dev generate-all when CI starts to fail. Generating the rules.md docs and the ty.schema.json file feel like they could be done as part of the release workflow on this repository, rather than having to be checked in as part of PRs to the Ruff repository? Alternatively, we could look at whether these files could be generated automatically behind the scenes when running mdtest.py or cargo test -p ty_python_semantic; then you wouldn't have to remember to execute a separate command before filing the PR.

AlexWaygood avatar Dec 24 '25 13:12 AlexWaygood