typeshed icon indicating copy to clipboard operation
typeshed copied to clipboard

Single script to run PR checks locally

Open Avasam opened this issue 2 years ago • 3 comments

After running scripts/create_baseline_stubs.py, we are informed of the following

Suggested next steps:
1. Manually review the generated stubs in {stub_dir}
2. Run "MYPYPATH={stub_dir} python3 -m mypy.stubtest {package}" to check the stubs against runtime
3. Run "mypy {stub_dir}" to check for errors
4. Run "black {stub_dir}" and "isort {stub_dir}" (if you\'ve made code changes)
5. Run "flake8 {stub_dir}" to check for e.g. unused imports
6. Commit the changes on a new branch and create a typeshed PR

There's a few issues with running those commands as such:

  1. Setting the MYPYPATH environment variable and running a script afterward is done differently per shell (think of powershell for example, where its $Env:MYPYPATH="{stub_dir}"; python3 -m mypy.stubtest {package})
  2. These will ignore some configurations such as METADATA.toml, stubtest_allowlist.txt, etc.
  3. Does not include other checks defined in https://github.com/python/typeshed/blob/master/tests/README.md
  4. It takes multiple commands to autofix and check everything
  5. Overall, the results will be different from what's on the CI

For those reasons, I suggest adding a new script under scripts that a dev who's contributing to typeshed can run locally to easily and completely check their stubs. The documentation in CONTRIBUTING.md and the output of scripts/create_baseline_stubs.py should then be updated accordingly.

Avasam avatar Sep 05 '22 14:09 Avasam

I would prefer:

1. Manually review the generated stubs in {stub_dir}
2. Run tests locally if you want (see CONTRIBUTING.md)
3. Commit the changes on a new branch and create a typeshed PR

The CI will check everything anyway, and is set up so that you don't have to run anything locally. This would also be consistent with CONTRIBUTING.md.

Akuli avatar Sep 05 '22 18:09 Akuli

@Akuli I'm mainly asking this because of the constant back and forth this produces with pushing changes, waiting on the CI (which has additional overhead), opening the errors page and dropdowns, then searching for the error. I'd much rather be able to quickly run all autofixes and checks locally.

My editor being VSCode, I could just setup a task for this project to run everything with the click of a single button. (but that's only gonna be beneficial to me)

At the end of the day, it's not a major pain point, and most contributions will only include some changes in already compliant stubs. I just happened to be using quite a handful of popular untyped libraries of which most were not yet ready to include types directly.

Avasam avatar Sep 07 '22 23:09 Avasam

I think we can maybe do both.

I agree with @Akuli that the language in create_baseline_stubs.py is definitely outdated — nowadays, we're pretty keen to emphasise that contributors shouldn't have to run any tests or formatting locally before submitting a PR.

But that doesn't mean that we can't also have a convenience script to e.g. run pycln/isort/black/flake8/mypy in quick succession, for contributors who want to run the tests locally first.

AlexWaygood avatar Sep 07 '22 23:09 AlexWaygood