Idea: build poetry wheels using mypyc

[uSpike]

• [x] I have searched the issues of this repo and believe that this is not a duplicate.
• [x] I have searched the documentation and believe that my question is not covered.

Feature Request

https://mypyc.readthedocs.io/en/latest/index.html is a pretty cool tool that can turn type-annotated code into compiled C extensions.

I think it's worth investigating whether poetry/poetry-core could be sped up by compiling native binary wheels and (optionally?) using compiled code to execute.

@ichard26 wrote a very interesting series of blog posts about doing this same optimization to black: https://ichard26.github.io/blog/2022/05/31/compiling-black-with-mypyc-part-1

[dimbleby]

MRs welcome, I expect, but whenever I've tried - more in hope than expectation - to use mypyc I've found that it is way too alpha and it's really hard to get it working on a project of any substance.

Would be glad to be shown wrong!

[Tatsh]

I do have a few projects that will work with Mypyc but it is tricky to add mypyc to the packaging system of Poetry. Support for mypyc would be just one more of things like Cython and Maturin: Building extension modules.

Poetry would have to have to dynamically generate the equivalent of this content normally placed in setup.py:

from setuptools import setup
from mypyc.build import mypycify
setup(
    name='mylib',
    packages=['mylib'],
    ext_modules=mypycify([
        'mylib/__init__.py',
        'mylib/mod.py',
    ]),
)

From Using setup.py

mypycify returns list[setuptools.extension.Extension].

Definitely can be done with a custom script specified in tool.poetry.build.script but it would be cool if it were simpler than that:

[tool.poetry]
mypyc-modules = ["module1", "module2/single_file.py"]

On the custom script part, if I get something working I can make a PR to add to the documentation.

[Tatsh]

These are the modifications required:

pyproject.toml:

[build-system]
requires = ["poetry-core", "mypy[mypyc]"] # and you must add all dependencies required for Mypy to succeed like type stub packages

[tool.poetry]
exclude = ["**/*.c"]
include = [
  { path = "package_name/**/*.so", format = "wheel" },
]

build.py:

from __future__ import annotations

from pathlib import Path
import shutil

from mypyc.build import mypycify
from setuptools import Distribution
from setuptools.command.build_ext import build_ext


def build() -> None:
    distribution = Distribution({'ext_modules': mypycify(['package_name/lib.py'])})
    cmd = build_ext(distribution)
    cmd.ensure_finalized()
    cmd.run()

    for output in cmd.get_outputs():
        relative_extension = Path('build') / Path(output).relative_to(cmd.build_lib)
        shutil.copyfile(output, relative_extension)
        relative_extension.chmod((relative_extension.stat().st_mode & 0o444) >> 2)


if __name__ == '__main__':
    build()