cython-package-example icon indicating copy to clipboard operation
cython-package-example copied to clipboard

Add example `pyproject.toml` using `poetry` instead of `setuptools`

Open FedericoStra opened this issue 3 years ago • 4 comments

Take inspiration from:

  • https://github.com/python-poetry/poetry/issues/11
  • https://github.com/python-poetry/poetry/issues/137
  • https://github.com/python-poetry/poetry/issues/1516
  • https://stackoverflow.com/a/63679316
  • https://github.com/ADicksonLab/geomm/tree/v0.1.7 (example of build.py)
  • https://github.com/zoj613/htnorm (more modern example of build.py)
  • https://github.com/decred/tinydecred/ (example of build.py)
  • https://github.com/iamishalkin/cyrtd (Cython + Sphinx + poetry example)

FedericoStra avatar Feb 09 '22 09:02 FedericoStra

The state of the art best practices for packaging python modules with compiled extensions is just a desolate land of pain and suffering.

The crucial detail to defeat https://github.com/python-poetry/poetry/issues/1516 appears to be to require setuptools and Cython in the build system defined in pyproject.toml:

[build-system]
requires = ["poetry-core>=1.0.0", "setuptools>=57.0.0", "Cython>=0.29.27"]
build-backend = "poetry.core.masonry.api"

FedericoStra avatar Feb 09 '22 11:02 FedericoStra

There is an initial implementation in the branch poetry. One of the downsides is that installing with pip install . leads to the following folder structure

cypack/
├── answer.c
├── answer.cpython-310-x86_64-linux-gnu.so
├── answer.pyx
├── data
│   └── zen.txt
├── fibonacci.c
├── fibonacci.cpython-310-x86_64-linux-gnu.so
├── fibonacci.pyx
├── __init__.py
├── __pycache__
│   └── __init__.cpython-310.pyc
├── sub
│   ├── helper.c
│   ├── helper.h
│   ├── helper.pxd
│   ├── __init__.py
│   ├── __pycache__
│   │   └── __init__.cpython-310.pyc
│   ├── wrong.c
│   ├── wrong.cpython-310-x86_64-linux-gnu.so
│   └── wrong.pyx
├── utils.c
├── utils.cpython-310-x86_64-linux-gnu.so
├── utils.pxd
└── utils.pyx

instead of the cleaner and intended

cypack/
├── answer.cpython-310-x86_64-linux-gnu.so
├── data
│   └── zen.txt
├── fibonacci.cpython-310-x86_64-linux-gnu.so
├── __init__.py
├── __pycache__
│   └── __init__.cpython-310.pyc
├── sub
│   ├── helper.h
│   ├── helper.pxd
│   ├── __init__.py
│   ├── __pycache__
│   │   └── __init__.cpython-310.pyc
│   └── wrong.cpython-310-x86_64-linux-gnu.so
├── utils.cpython-310-x86_64-linux-gnu.so
└── utils.pxd

FedericoStra avatar Feb 09 '22 15:02 FedericoStra

I found this issue via search. I'm trying to cytonize parts of a package and also face the problem, that the resulting package will include the original Python, the C-source and the compiled binary.

My target is, that the wheel only contains the compiled binary.

rnestler avatar Apr 14 '22 12:04 rnestler