Add mypy stub files to PyAV

[tooxo]

Overview

To simplify developing and linting with PyAV, stub files should be included. explanation of stubs

Desired Behavior

Every object / method should have defined stubs, which are detectable by IDEs such as PyCharm aswell as typical linters like MyPy.

Example API

Not relevant for this problem (?)

Additional context

For futher explanation see: https://www.python.org/dev/peps/pep-0484/

A possible solution could be using a stub generator like stubgen.

[jlaine]

I'd very much enjoy having such stubs! However, having already written stubs for a fairly large project (cryptography) I know this is going to take more time than I currently have. Would you consider submitting a PR?

[tooxo]

Yes sure, if I get to it I will submit a PR.

[mikeboers]

Is there any way to auto-generate this from the Cython source? I'm not looking forward to maintaining the stub files in the future, and if it can be a near-automatic process it is more likely to be maintained by me.

[tooxo]

Yes and no, there is a mypy stub generator for cpython files, but that requires pyx files with every type indicated. If it's not the case the stubgen will create stub files with any type indicated as any, which doesn't really help.

For example this will be the result on the descriptor.pyx file:

from typing import Any

class Descriptor:
    name: Any = ...
    options: Any = ...
    @classmethod
    def __init__(self, *args, **kwargs) -> None: ...
    def __reduce__(self) -> Any: ...
    def __setstate__(self, state) -> Any: ...

So the (only) possible solution would be to refactor the whole library to use these type hints.

[tooxo]

Another (smaller) step would be to follow up on the promise here https://github.com/PyAV-Org/PyAV/blob/develop/av/init.py#L14 and import anything into the __init__.py, and not just the things that are human constructable, so that IDEs like PyCharm can at least auto-complete the class names.

[moi90]

So the (only) possible solution would be to refactor the whole library to use these type hints.

Couldn't we just start slowly? PyAV supports Python from 3.5 on, so adding type annotations shouldn't break anything.

[synodriver]

Hi. Add stub files would not be so hard(I think). After install pyav and mypy, just stubgen -p av and that would generate pyi files. Althogh some of their types are Any.

[jlaine]

Hi. Add stub files would not be so hard(I think). After install pyav and mypy, just stubgen -p av and that would generate pyi files. Althogh some of their types are Any.

Great, I look forward to your pull request!

[synodriver]

Hi. Add stub files would not be so hard(I think). After install pyav and mypy, just stubgen -p av and that would generate pyi files. Althogh some of their types are Any.

Great, I look forward to your pull request!

But when function signatures changed or new functions been add, the stubgen command should be run again to generate new pyi files against new modules. Should we do that in CI?

[moi90]

stubgen is surely a good approach. But why not include type annotations directly in the code?

[jlaine]

@moi90 I'm unfamiliar with how to include type annotations for Cython code, but on principle yes I would prefer them to be straight in the code if that is possible.

[moi90]

@jlaine Ah, yes. Compiled modules will require pyi files. However, stubgen support for Cython seems to be currently limited, see https://github.com/python/mypy/issues/7542.

Anyways, the annotations produced by stubgen can serve as a starting point and can then be manually improved. (This seems to be the preferred way in typeshed as well.)

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[moi90]

This shouldn't be closed, I think this could be very valuable for users of the library.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[yozachar]

I get missing stub files.

Related: #1065

Would adding a py.typed, to this repository resolve this?

[moi90]

Would adding a py.typed, to this repository resolve this?

No, the problem is, that no type annotations are available at all. It would be tedious to add them manually and keep them up to date.

And, so far, is not easy to extract typing information from Cython code automatically.

Apart from stubgen (which is only able to inspect the compiled result, not the Cython code itself) https://github.com/Vizonex/CyStub could be another solution, but it does not work with Cython 3.