experiment with shipping type annotations

[mmerickel]

It'd be cool if pyramid could ship type annotations for its public APIs. I talked to the pycharm folks at their booth last week and there are apparently 3 ways to do this given that we want to keep working on py2 and py3:

1. Use the comment-based type annotation syntax in pyramid source which is compatible with py2.
2. Use separate stub files that are shipped to the typeshed repo. https://github.com/python/typeshed/
3. Use separate stub files distributed in the pyramid package.

It would be helpful to evaluate the pros and cons of the various approaches with the primary goal being to minimize the maintenance effort required. For example, a build hook that could check that type hints were defined for public APIs.

On a side note, I think the commant syntax is ugly and that it should be considered the worst of the 3 options.

Related https://github.com/Pylons/pyramid/issues/2387

NOTE: This issue is vague because I only sort of know what I'm talking about.

[stevepiercy]

Option 1 and 3 require a release of Pyramid (con), whereas 2 does not (pro). That alone makes option 2 the most appealing to me.

[mmerickel]

Option 1 and 3 require a release of Pyramid (con), whereas 2 does not (pro). That alone makes option 2 the most appealing to me.

These are type annotations, not documentation. They are inherently coupled to the source of the release. I think this is not a concern.

[stevepiercy]

Are you saying that there shall be no bugs or other mistakes in the type annotations when added to the Pyramid package, or that if there are then it is no big thing?

[mmerickel]

I think if there are bugs with them then they could simply get fixed in a later release. Again that's just my first impression as I have not used type annotations myself.

[tseaver]

-1 to option 2: annotations have to stay tightly in-sync with code: in fact, we should find a way to "test" them so that we don't see them drift. I see no benefit in creating an out-of-tree "authority" for them which can drift (and will, inevitably) from changes as Pyramid releases are made.

[iivmok]

I would love to see Pyramid type annotations, personally I like the 3rd option the most. I use PyCharm and type annotations a lot, and marking request as WebOb Request class helps a whole lot, especially when I first started using Pyramid.

[ztane]

I would like to see the option 4: stop supporting Python 2 and use real annotations.

Though a bigger problem is that nothing supports interfaces.

[mmerickel]

Though a bigger problem is that nothing supports interfaces.

Yeah zope.interface is a big issue for us in Pyramid. It looks like https://github.com/python/mypy/issues/1240 is slowly working on this.

[domenkozar]

Most of pyramid "exposed" api isn't zope.interface and thus would benefit from having types :)

[mmerickel]

I'm not sure that's true. We have two major problems in terms of typing / linting.

1. Almost all of our APIs are defined in terms of zope interfaces such as a view signature which accepts (context: Any, request: pyramid.interfaces.IRequest) or config.set_authentication_policy(policy: pyramid.interfaces.IAuthenticationPolicy).

2. Some of Pyramid's public api is configurable at runtime as well which is difficult to type at all (speaking as someone with minimal mypy experience). For example config.add_directive adds extra methods to the configurator and config.add_request_method adds extra methods to the request.

[domenkozar]

1. Yes, those could be defined as Any until zope.interface plugin exists.

2. Could define http://mypy.readthedocs.io/en/latest/protocols.html for default Configurator, then the rest of custom methods typing is up to the user.

(close button is too close)

[pior]

Is there a plan to work on typing already?

I don't see any mention of it in the 2.0 feature list #2362.

I would love to help with stubs/comments for 1.x since I'm using it at work, but it would also be interesting to plan for Py3.

[ztane]

Please when talking with pycharm folks, tell them we want to type with zope interfaces too :D

[mmerickel]

Here is, in my eyes, what needs to happen to get typing into Pyramid. The goal would be to support typing via annotations and not via stub files or comment syntax. This is open for debate but this is the roadmap I have in my head.

• [x] refactor the repo structure first #3022 (this should be coming in the upcoming 1.10 release)
• [x] drop support for Python 2 #2903 (this should be happening after the 1.10 release)
• [ ] experiment / come up with a strategy to handle the configurator and z.i issues (using Any or something else) and test it out on pycharm and mypy (no one has really taken this on yet that I know of)
• [ ] actual PR to pyramid to add typings including mypy steps to our tox -e lint config

If I were someone interested in doing this and starting from scratch I would probably look at what websauna is doing. It is a framework built on top of pyramid and is py3-only with annotation-based typing.

[jtrakk]

Yes, those could be defined as Any until zope.interface plugin exists.

Here it is. https://github.com/Shoobx/mypy-zope