Add support for PEP 484 Type Hints

[kkirsche]

Would adding type hints to the gunicorn codebase be something the team would be open to? If so, what can I do to help with this initiative?

[tilgovi]

I closed #2374 with a positive answer. I think it would be a great addition.

[kkirsche]

Awesome! Regarding contributing types, do you have a preference for how that's delivered? For example, do you prefer to have smaller, focused merge requests that add various pieces or a larger merge request which lays the full foundation?

[tilgovi]

I think smaller contributions are more likely to get reviewed and merged.

[kkirsche]

Perfect, thanks for the feedback. So looking at the codebase, the following is the approach that makes sense to me to take, but wanted to outline it so that others are aware and can provide feedback, if desired.

The gunicorn codebase is decently large and comes with complexity. To avoid errors in the type hints, the safest approach is to work from the innermost "edges" of the gunicorn API and work outward to the user-facing API. To elaborate, this means identifying functions, classes, modules, etc. which are self-contained or use other libraries that have existing types. This will allow these methods to be typed appropriately based on the requirements of those functions. This can be done either on a per-file or per-module basis, I lean towards per-file to keep it nice and small.

During this stage, the py.typed marker will be added, mypy configuration, etc. Once the innermost edges are complete, I'll start to work outwards to functions that call the typed functions, and work up the API until coverage is complete.

I think the config module may be a good place to begin this process from, but I'll review the code more to determine if there is a better starting point. Once identified, I'll create an initial pull request to begin this process and will put a checklist into this issue of the various files to track completeness of the type hints

[kkirsche]

It's worth noting, that mypy's variable type hints for classes are only supported on Python 3.6+, so that may delay certain annotations until support for EOL Python 3 versions (per https://devguide.python.org/versions/#versions) is dropped or pyi stubs are used instead of inline type hints.

EDIT: clarified statement about dropping EOL support with added alternative of using pyi stubs

[kkirsche]

Clarified the above statement with alternative that using .pyi stubs rather than inline stubs would be an alternative to bypass this issue.

[benoitc]

what would be the results of this types? How would they be used?

[kkirsche]

Can you be more specific? I don’t understand your question.

With that said, Type hints are a feature of Python that allows you to explicitly declare the data type of a variable when declaring it. They are only available in Python 3.5 and later.

Type hints provide benefits to both maintainers and users who choose to use gunicorn as a library via subclassing or other means. First, they help people reading your code to know what types of data to expect. Second, they can be used by tools such as mypy to detect type-based errors in a codebase that either currently exist or are introduced (such as via refactoring). Third, are available to users of gunicorn as a library rather than command-line tool to understand the expected structure, and the attributes, of functions such as child_exit in config modules or other locations (such as subclassing the server to customize its behavior or start it programmatically. Fourth, this integrates type data with editor language servers, such as pylance in VS Code, to provide inline highlighting of usage errors.

Types are documented here on Python's website: https://docs.python.org/3/library/typing.html

Hopefully this helps answer some of your question.

EDIT: edited for grammar

[mockodin]

A bit more/different-level of work probably but to point out some projects like requests use an external typing/stubs module to implement typing without impacting legacy python versions.

[benoitc]

@kkirsche I was wondering how types could benefit for the project. Anyway If it can help somes developers to contribute , I'm open for a PR :)

[asciifaceman]

@benoitc type hinting is a massive boon to any project, especially an open source library. It removes a lot of implementation confusion, is supported as a pop up in many IDEs, and in the case of many devs allows mypy to do strict static analysis on code bases to assert quality. I found myself here because we use mypy strict static analysis ourselves:

error: Skipping analyzing "gunicorn": module is installed, but missing library stubs or py.typed marker  [import]
error: Class cannot subclass "BaseApplication" (has type "Any")  [misc]

I would wager this will be a growing demand from developers as type hinting and demand for type analysis has grown exponentially in recent years.

admittedly, it's a very tall order to implement.

A good starting point might be stubgen, and then tweaking it to ensure accuracy

https://mypy.readthedocs.io/en/stable/stubgen.html