ENH: upstream setuptools patches to upstream setuptools

[mattip]

This would simplify building projects that have migrated to HPy. This task would be something like

• find the places in the current code that need more support from setuptools
• submit patches to upstream to incorporate them into the official setuptools

[antocuni]

This is a summary of what we do after merging #338 :

• The final goal is to be able to write something like this

from setuptools import setup, Extension
setup(
    name="hpy-simple-example",
    hpy_ext_modules=[
        Extension('simple', sources=['simple.c')]),
    ],
    setup_requires=['hpy'],
)

• another requirement is to be able to build "universal modules": these require a different set of compilation options, and produces a file with a different extension: simple.hpy.so. Currently, we do that by doing setup.py --hpy-abi=universal build.

• we are using a setuptools entry point to detect hpy_ext_modules=[...] and hook into setuptools:

https://github.com/hpyproject/hpy/blob/2211087ff360ec5134baa8318300d64a18892901/hpy/devel/init.py#L128-L142

• inside the hook, we automatically patch the distribution to use our own build_hpy_ext command for build_ext:

https://github.com/hpyproject/hpy/blob/2211087ff360ec5134baa8318300d64a18892901/hpy/devel/init.py#L95-L99

https://github.com/hpyproject/hpy/blob/2211087ff360ec5134baa8318300d64a18892901/hpy/devel/init.py#L261

• this is where we modify the Extension to add the extra c files, include dirs, etc.:

https://github.com/hpyproject/hpy/blob/2211087ff360ec5134baa8318300d64a18892901/hpy/devel/init.py#L285-L301

On top of these, there are various hacks and bad monkey-patching to make things works. For example, we have to use this ugly hack to be able to distinguish hpy and normal extensions:

https://github.com/hpyproject/hpy/blob/2211087ff360ec5134baa8318300d64a18892901/hpy/devel/init.py#L176-L195

[mattip]

We could think about supporting pyproject.toml build systems as well or instead of setuptools. The scientific python projects have moved to meson/meson-python.

[fangerer]

Agreed. Btw. this relates to #435 .