boltons
boltons copied to clipboard
mahmoud
Update package metadata
Background
Hello there! The Python packaging ecosystem has now standardized on the interface for build backends (PEP 517/PEP 660) and the format for metadata declaration (PEP 621/PEP 631).
As such, I'm spending my free time updating important projects so that they are modernized and set an example for others 😄
Summary of changes
This implements PEP 621, obviating the need for setup.py, setup.cfg, and MANIFEST.in. Support has not landed in setuptools, so builds will now use hatchling.
I've done this for such projects as pipx, all Datadog Agent integrations, etc.
Notes
- This also drops support for EOL Python 3.x versions since
hatchlingand other build backends likesetuptoolshave already done so. This will not break anything becausepiponly looks for version candidates that satisfy therequires-pythonfield. Here are stats for the curious: https://pypistats.org/packages/boltons - This also resolves https://github.com/mahmoud/boltons/issues/290 as I saw that had not yet been handled, @mahmoud let me know if that's okay!
Hatchling seems very new (still 0ver ;) ). What's the plan for maintenance/stability?
It is quite stable and actively maintained, the only reason the version is 0.x is because 1.0.0 will drop support for Python 2.
Does Hatch/Hatchling offer anything like packaging tests we can add to the tox.ini?
If you mean like checking the contents of the sdist *.tar.gz file then no, I don't think any of the newer backends do that because there is no more MANIFEST.in which was the confusing and error prone part.
Would` you mind triggering the CI with that button below?
Done :) All green.
Hatchling seems very new (still 0ver ;) ). What's the plan for maintenance/stability?
It is quite stable and actively maintained, the only reason the version is 0.x is because 1.0.0 will drop support for Python 2.
I guess my main concern is that Hatchling first came out (0.3) two months ago? And I love the velocity, but it also makes me nervous should something break. Presumably I'd own the fix at that point :)
Longer term, what's your recommendation when setuptools support lands?
If you mean like checking the contents of the sdist *.tar.gz file then no, I don't think any of the newer backends do that because there is no more MANIFEST.in which was the confusing and error prone part.
MANIFEST.in could certainly be tricky, but even in this modern approach, couldn't the includes here go out of sync if a new file is added?
Finally, maybe a very simple question, but if someone were to seek to install boltons from a source distribution built with this modern approach, how would it be recommended they do that without a setup.py? Does pip know what to do these days?
should something break. Presumably I'd own the fix at that point
No it would be on the maintainers, just as if you used setuptools or some other build backend. Since we use it at Datadog for shipping business critical things, our CI would be the first to yell at me 😄
Longer term, what's your recommendation when setuptools support lands?
I still would not use it. I can briefly tell you why we moved:
Our goal is to remove all setup.py files since their use is now heavily discouraged and build with a PEP 517 compatible build system while defining metadata with PEP 621.
setuptools
Disadvantages:
- Does not yet support
pyproject.toml-based metadata declaration nor editable installation standards (PEP 660) - Based on a deprecated standard library module that it now vendors
- Slow to get things merged since the code quality issue is exacerbated by the fact that it is mostly maintained by just one person
Flit
flit-core is the build backend behind Flit.
Disadvantages:
- No mechanism for extending behavior by design, so we could never do things like versioning with Git tags
- Very opinionated about project layouts since it targets beginners e.g. only supports implicit namespace packages, must define certain data like
__version__inside the root__init__.pyfile, etc.
Poetry
poetry-core is the build backend behind Poetry.
This was actually a no-go option for us due to its persistent unwillingness to adopt new standards if they are deemed suboptimal (see comments on PEP 621 and PEP 665)
Hatchling
- In comparison to
setuptools, Hatchling is far smaller, making it less susceptible to security issues and more auditable. For example, the logic to build a wheel resides in this one file. - In comparison to
flit-core, Hatchling is extensible by design. For example, there is already a plugin for setuptools_scm (which despite the legacy naming is actually now decoupled fromsetuptools) and a plugin for compiling code with Mypyc. - In comparison to
poetry-core, Hatchling never strays from community standards and is in fact eager to adopt new ones (it's the only backend that currently supports PEP 639).
couldn't the includes here go out of sync if a new file is added
Well yes of course. If you want everything added to the source distribution that section can just be omitted, or define exclude.
if someone were to seek to install boltons from a source distribution built with this modern approach, how would it be recommended they do that without a setup.py? Does pip know what to do these days?
Yes, since version 19.0.
You could also just use the setuptools build backend.
build-backend = "setuptools.build_meta"
It's pretty standardized and I don't think there are any concerns about maintenance. Also, since a build backend has a standard to follow, if Hatchling really takes off, it wouldn't be hard to switch to it.
The mentioned lack of support for pyproject.toml metadata in setuptools is also incorrect (it was at right at one point, but it's supported it for a while). It's actually possible to have a pyproject.toml as the only configuration file.
[build-system]
requires = [
"setuptools ~= 61.0",
"wheel",
"build",
]
build-backend = "setuptools.build_meta"
[project]
name = "boltons"
version = "21.0.1dev"
description = "When they're not builtins, they're boltons."
urls = { "homepage" = "https://github.com/mahmoud/boltons" }
authors = [
{ "name" = "Mahmoud Hashemi", "email" = "[email protected]" },
]
license = { "text" = "BSD" }
classifiers = [
"Topic :: Utilities",
"Topic :: Software Development :: Libraries",
"Intended Audience :: Developers",
"Development Status :: 5 - Production/Stable",
"License :: OSI Approved :: BSD License",
"Operating System :: OS Independent",
"Programming Language :: Python :: 2",
"Programming Language :: Python :: 2.6",
"Programming Language :: Python :: 2.7",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.4",
"Programming Language :: Python :: 3.5",
"Programming Language :: Python :: 3.6",
"Programming Language :: Python :: 3.7",
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: Implementation :: CPython",
"Programming Language :: Python :: Implementation :: PyPy",
]
[project.optional-dependencies]
test = [
"tox",
]
That's all you'd theoretically need; no need for setup.py or setup.cfg at all.
It's actually possible to have a pyproject.toml as the only configuration file.
Support landed a few days ago and is experimental. See notes and fixes: https://setuptools.pypa.io/en/latest/history.html
I don't think there are any concerns about maintenance
True but in comparison to setuptools, Hatchling has reproducible builds, supports the modern PEP 660 editable (pip install -e ...) installation standard, has a more intuitive plugin system, and the codebase is an order of magnitude smaller thus making it more auditable and less prone to bugs (setuptools is based on a deprecated standard library module that it now must vendor).
I'm not sure having distutils in setuptools is a huge issue, it was actually removed from Python in part BECAUSE it was already in setuptools anyway and there wasn't a reason to maintain it in the standard library. Full support for pyproject.toml WAS only recently merged, but it's been possible to include metadata there for a while but it didn't validate it and defaulted to setup.cfg in any conflict; I've been doing so for a spell and with the new release the whole thing is working even better.
Really though @ofek I'm not disagreeing with your analysis; my swing through Hatchling actually impressed me and I'm going to take it for a spin on a personal project so I have a reminder to keep an eye on it. It may be (probably even IS?) the superior project. I hope it takes off!
Really though, technical analysis aside, my preference for widespread important projects like boltons tends to be conservative. For better or worse setuptools is the major player here. It's not the only one but it's the old hat in the ring and it's the conservative choice. Were Hatchling to really catch on and take off I'd say it would be a really cool thing to look at upgrading to. Swapping build backends isn't too hard anyway.
Of course it's not my call at all :smile: - I was just popping by to leave an unsolicited opinion :wink:
Small update: Hatch will likely be adopted by PyPA in the next week
- https://mail.python.org/archives/list/[email protected]/thread/J2BL5XTTOCXUCELUE7L5P5HHJ7NXZGZ5/
- https://mail.python.org/archives/list/[email protected]/thread/ZWSSNKQJ3GRIAK73FJS6KHOJNUJZYOVX/
Nothing major will change, just the repo location