pyopenssl
pyopenssl copied to clipboard
pyca
Remove X509Extension, which has been deprecated for a year
I'm a bit nervous about the impact of this. Notwithstanding that it's been deprecated for a while, it effectively ends using pyOpenSSL to generate certs, and I don't have a sense of how widely that's relied upon.
certbot 4.0 which removed the usage of this has been released.
@wgreenberg, if we merge this and do a release, will that break older installs of certbot, or do they bound their pyopenssl versions?
Assuming no bounds, is there any way for us to track what versions of certbot people use? (Is it in the user-agent? Happy to chat with our friendly neighborhood Lets Encrypt SREs :-))
ack, unfortunately we don't have an upper bound on pyopenssl versions, so this would indeed break older, pip-based certbot/acme installations (but not snap). not that that's a blocker necessarily, but should be noted! theoretically we could backport version bounds for older versions; i'll discuss that w/ the team.
i'll get back to you on version metrics
Thanks.
Unclear how much a backport would do. (We've seen a lot of people who pin their deps, but not transitive ones, and those still break.)
We'd like to avoid breaking too many folks, so I think tracking version is the most important thing we could do.
so the bad news is we don't have stats on whether people have installed their certbot via snap, a distro-managed package, or pip (the only method that would be affected by the breakage), but some basic back of the envelope estimates puts the number of pip installations anywhere from 50k-2M users. of these, any version <4.0 might be affected
given this, i think it'd be reasonable to post an announcement on the letsencrypt community forum to give folks a heads-up -- i'd be happy to do that, since the call to action is a little nuanced based on people's certbot installation method
Sure -- I think the action item is basically "if you're installing with pip and going to pin to a certbot<4, make sure to also pin your pyopenssl".
50k-2M is a lot of people, so we're going to either need to build confidence most people are on distro versions, or go slow here.
so the bad news is we don't have stats on whether people have installed their certbot via snap, a distro-managed package, or pip (the only method that would be affected by the breakage), but some basic back of the envelope estimates puts the number of pip installations anywhere from 50k-2M users. of these, any version <4.0 might be affected
How were those numbers calculated? IIRC, PyPi stats include downloads by mirrors, but not those using mirrors... so many CI systems do not hit this. They're showing 450k monthly downloads - which is pretty high (https://pypistats.org/packages/certbot). The most popular certbot environments are py3.10 and py3.11; same with pyopenssl. pyopenssl is currently supporting py3.7+ ; certbot is supporting py3.9+. The potentially affected platforms are a minority of "actively maintained" installs ("actively maintaned" as in people occasionally update them; most people who hit the letsencrypt forums with a pip install have never updated it).
A safe approach might be for Certbot to create a pin release for the versions that last supported py3.7 and py3.8 - an update or install on earlier systems won't pull in this release, and anything py3.9+ shouldn't care about this because it would use the newer certbots that don't reference this code. Looking at the changelog, Certbot dropped py3.8 in v3.1 , and py3.7 in v2.8.
An important fact here is that nothing we do in a new certbot release matters, the backwards compatibility issues here exist for people doing pip install certbot<4 (for example), which installs an old certbot but will install a new pyopenssl.
Sorry if I was not clear with this: What I meant to convey is that a v3.1.1 release and a v2.8.1 release with pins should cover most legacy platforms, because of how the minimum python versions of both PyOpenSSL and Certbot align across python environments. Anything py36 or under will not be affected, because pip won't pull in an affected PyOpenssl due to the minimum python version; so we would just need to handle py3.7 and py3.8.
For anyone running py3.9 above – which seems to cover >80% of actively maintained installations based on pypi stats of 3.10 and 3.11 alone – pip install certbot<4 should pull in Certbot 3.3.0. Certbot >= 3.2.0 shouldn't ever call this code – it switched to make_self_signed_cert, though gen_ss_cert is still in there as deprecated for compatibility through 3.3.0. It is possible some third party extension or in-house tech is still leveraging this, but those numbers should be negligible. I handled some of this deprecation and removal in Certbot/Acme, which is why all this stuff is still fresh in my mind.
There will of course be issues for people trying to install a specific version that wouldn't be caught by the 2 suggested pins that will cause issues. Perhaps people are running specific pins like this for various reasons, such as a popular plugin. I don't have any insight into that.
Based on my understanding of Certbot/PyOpenSSL code and the platform/library version distributions though, I don't think pip install certbot<4 is likely to affect anyone on py3.9 or above due to the previous api migrations - and patch version bumps against 3.1.0 and 2.8.0 with only a pyopenssl pin should address most legacy Certbot installs.
How were those numbers calculated?
we used user-agent stats from LetsEncrypt to estimate the installation distribution, but again, it's a very very rough estimation.
also thanks for the analysis @jvanasco! though given that our minimum supported Python version is 3.9.2, i don't think it's likely we'll put effort into backporting pinned releases unless we start getting many reports of breakages.
btw i posted an update on this to the LE community forum here: https://community.letsencrypt.org/t/upcoming-pyopenssl-deprecation-will-impact-older-4-0-certbot-versions/237067