rust-python-ext
rust-python-ext copied to clipboard
novocaine
Options for distributing without rustc?
Hi, I'm interested in your project because I'm looking for an alternative to C++ extensions. I'm now wondering how the odds are for distributing an extension that I would write. In C++, I could build a wheel for Windows and Mac and then expect that Linux users would have a compiler ready. With Rust, however, this may not work that way. The wheel building part for Win/Mac may still work out (will it?) I'm most worried about the Linux part since rustc will just not be available everywhere for the foreseeable future. How well does Lbinary distribution work for Linux with Rust? It's not going to be better than C, is it?
Kind regads, Matthias
Hey @MatthiasKauer,
It looks like binary Rust wheels are not easy either (https://github.com/pypa/setuptools/issues/435).
Great Idea @MatthiasKauer !
Until rust is as mainstream as C compilers, asking your users to compile rust to use your extension is likely to alienate them just a little bit. So as @kmike suggests you would really want to distribute binary wheels.
I don't see why in principle wheels can't be made to work, but I haven't tried. If you work out how to hook distutils, I'd love to incorporate it into this project as long as it's clean.
You should be aware that working out how to use rust-cpython is very daunting and it only runs on rust nightlies. While I am optimistic about its future given the obvious benefits of the approach, I don't believe it's really ready for writing production libraries just yet, which is why I haven't done much more in this space lately.
(maybe I should revise the promises I give in the README.md?)
Hi, thanks for the clarification. Simon Sapin's discussion on bitbucket is a big issue that I hadn't even foreseen.
I was more worried about this PEP: https://www.python.org/dev/peps/pep-0491/ If I read this right, wheels do not work for Linux right now since the C libraries on various distros are too different. If that's an issue for C, it's most likely going to be an issue for rust too, no?
@MatthiasKauer yeah, almost: wheels work on Linux, but you can't safely share them and distribute them if machines are different. There is https://www.python.org/dev/peps/pep-0513/ in works to overcome it.
exactly, that's the one we would need @kmike
And regarding rust-cpython: I was under the impression that its goal is the other direction, i.e., calling Python from Rust. Am I mistaken there too?
You would need also equivalent guarantees from the rust packager. I don't know if that's on the table for rust.
On the up side maybe linux users are more amenable to installing rustc especially if there is an appropriate version in their package manager.
rust-cpython is just a wrapper around the Python C API, meaning it is suitable wherever the Python C API is - either embedding Python in an application, or calling C libraries from python. It does contain examples of how to do either. My personal interest is in the latter.
Hi, author of bdist_wheel here. I've been working on a different way to make wheels using SCons that I hope will be useful. IMO it is simpler and easier to understand than bdist_wheel because it does not use the setuptools/distutils architecture at all, but it also emulates setup.py to create pip-compatible sdists. Here's an example build file for Simon's cffi Rust example. https://github.com/dholth/hello-pyrust/blob/master/SConstruct
I'd be interested in hearing whether this is useful to you.
cool. I guess the equivalent of rust-python-ext for your new system would be a Builder for rust libs?
How is scons going with python 3 support?
Yes. As you can see on line 28 invoking cargo with an env.Command() is really easy too, it is not like GCC where you have to understand a ton of options and a multi-step compile - link toolchain. Or you could publish a pypi project that adds a rust builder and install it before setup on line 3 of this file https://github.com/dholth/hello-pyrust/blob/master/pyproject.toml
SCons works on Python 3 for everything I've tried to do with it. enscons automatically pulls in the correct SCons from https://pypi.python.org/pypi/import-scons
oh, neat. with some sort of trunk scons? pip3 install scons for me tries to install 2.5.0 and errors because it explicitly doesn't support py3
https://github.com/timj/scons/tree/u/timj/python3
pip install import_scons && python -m SCons
@dholth Just to clarify: Your scons solution does indeed build a binary wheel such that the end user doesn't need a rust compiler anymore? That would be a great step.
That's right. It builds wheels and gives you lower level control over how the wheel is built compared to setuptools. After the wheel is built it's no different and you can distribute the wheels normally. On Sep 24, 2016 09:48, MatthiasKauer [email protected] wrote:@dholth Just to clarify: Your scons solution does indeed build a binary wheel such that the end user doesn't need a rust compiler anymore? That would be a great step.
—You are receiving this because you were mentioned.Reply to this email directly, view it on GitHub, or mute the thread.