Add support for Python 3.12 and drop EOL 3.7

[hugovk]

Re: https://devguide.python.org/versions/

Also upgrade Python syntax with https://github.com/asottile/pyupgrade and convert to f-strings with https://github.com/ikamensh/flynt.

[codecov[bot]]

Codecov Report

Attention: 6 lines in your changes are missing coverage. Please review.

Comparison is base (3cd57b6) 98.05% compared to head (66845c3) 98.04%.

Files	Patch %	Lines
glom/core.py	94.23%	3 Missing :warning:
glom/test/test_mutation.py	75.00%	0 Missing and 2 partials :warning:
glom/matching.py	97.36%	1 Missing :warning:

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #283      +/-   ##
==========================================
- Coverage   98.05%   98.04%   -0.01%     
==========================================
  Files          27       27              
  Lines        4362     4356       -6     
  Branches      765      765              
==========================================
- Hits         4277     4271       -6     
  Misses         54       54              
  Partials       31       31              

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[mahmoud]

Hey Hugo! This is awesome. I knew about pyupgrade, but this is the first I'm seeing flynt, thanks for this. So what I usually do before removing a version is ask:

1. What proportion of users still use the EOL version?
2. What do we gain by dropping the version? (i.e., is there a feature we want to build with)

For #1, I look at download stats. There's a site I always use, written/maintained by a really cool dev:

I think you know it ;) Anyways, 3.7 is weirdly popular with glom right now. Funky traffic pattern. Before recently it was ~4% which is nice and low.

For #2 the f-strings are gonna be nice. Curious if there's anything else you particularly like about 3.8+ vs 3.7+, in the event you had thoughts.

Overall I'm leaning +1, just because you did such a great job refreshing the codebase. I'll let @kurtbrose weigh in, as well. Thanks again!

[hugovk]

I do know https://pypistats.org/, but I don't maintain it, just the CLI: https://github.com/hugovk/pypistats :)

Yeah, there's something a bit odd with 3.7, for some reason lots of downloads are coming from Amazon Linux:

https://dev.to/hugovk/why-are-there-still-so-many-downloads-for-eol-python-37-30cp

Same for glom. Here's 30 days:

❯ pypinfo --days 30 --limit 20 --percent --markdown "glom" system distro pyversion
Served from cache: False
Data processed: 2.71 GiB
Data billed: 2.71 GiB
Estimated cost: $0.02

system_name	distro_name	python_version	percent	download_count
Linux	Amazon Linux	3.7	33.36%	584,316
Linux	Ubuntu	3.10	32.99%	577,959
Linux	Ubuntu	3.8	4.42%	77,411
Linux	Debian GNU/Linux	3.9	3.98%	69,783
Linux	Amazon Linux	3.10	3.27%	57,270
Linux	Debian GNU/Linux	3.6	3.10%	54,374
Linux	Ubuntu	3.11	3.09%	54,060
Linux	Ubuntu	3.9	2.89%	50,665
Linux	Debian GNU/Linux	3.8	2.89%	50,624
Linux	Debian GNU/Linux	3.10	2.06%	36,022
Linux	Debian GNU/Linux	3.11	1.98%	34,726
Linux	Debian GNU/Linux	3.12	1.25%	21,890
Linux	Ubuntu	3.7	0.88%	15,492
Linux	Debian GNU/Linux	3.7	0.76%	13,227
Linux	Fedora Linux	3.11	0.63%	11,098
Linux	Ubuntu	3.12	0.56%	9,888
Linux	Alpine Linux	3.11	0.55%	9,650
Darwin	macOS	3.11	0.51%	8,862
Linux	Amazon Linux AMI	3.7	0.46%	8,140
Linux	Amazon Linux	3.9	0.35%	6,203
Total				1,751,660

Limiting to just one day to avoid that peak, things look more "normal":

❯ pypinfo --days 1 --limit 20 --percent --markdown "glom" system distro pyversion
Served from cache: False
Data processed: 138.00 MiB
Data billed: 138.00 MiB
Estimated cost: $0.01

system_name	distro_name	python_version	percent	download_count
Linux	Ubuntu	3.10	45.06%	23,064
Linux	Debian GNU/Linux	3.8	6.00%	3,069
Linux	Ubuntu	3.8	5.90%	3,022
Linux	Amazon Linux	3.7	5.51%	2,821
Linux	Debian GNU/Linux	3.9	5.08%	2,601
Linux	Ubuntu	3.9	4.79%	2,450
Linux	Ubuntu	3.11	4.48%	2,295
Linux	Amazon Linux	3.10	4.04%	2,069
Linux	Debian GNU/Linux	3.10	3.38%	1,732
Linux	Debian GNU/Linux	3.11	3.25%	1,661
Linux	Debian GNU/Linux	3.6	3.10%	1,586
Linux	Debian GNU/Linux	3.12	1.98%	1,013
Linux	Debian GNU/Linux	3.7	1.49%	765
Linux	Ubuntu	3.12	1.12%	575
Linux	Ubuntu	3.7	1.04%	531
Linux	Alpine Linux	3.11	1.03%	526
Linux	Fedora Linux	3.11	0.86%	440
Darwin	macOS	3.11	0.77%	394
Linux	Amazon Linux	3.11	0.57%	292
Linux	Alpine Linux	3.8	0.54%	275
Total				51,181

Anyway, my projects tend to follow the EOLs of Python itself:

https://devguide.python.org/versions/

One clear benefit is reducing resource use on the CI, allowing us to add test newer versions without worry about resource waste. And 3.13 will be in beta soon, so it will be good to add that. Another is being able to adopt newer syntax.

More that this, I find it much easier to follow upstream, than to have an annual discussion over the pros and cons. People shouldn't be using EOL Pythons, they no longer receive security updates, so we shouldn't be encouraging it.

But most of all, following upstream is good for communication: there's a clear schedule published by a third-party that we can all follow.

(The Scientific Python community go one further, and have agreed on a standard to drop Pythons even sooner. Currently they recommend 3.10+: https://scientific-python.org/specs/spec-0000/)

[mahmoud]

Hey Hugo, I really appreciate the refresher on the Python release cycle, and the great dev.to posts you've done on the relevant stats. Still deciding here, but figured I'd share some relevant thoughts.

Personally, having worked in many enterprise environments where a given Python is supported long after the core devs decide it's EOL, I'm inclined to retain library support longer, until there's a clear advantage to dropping a version. (For my application work, I'm more of an early adopter.) I think the comments on this point capture the concerns well: https://dev.to/hugovk/why-are-there-still-so-many-downloads-for-eol-python-37-30cp

In short (for anyone else reading this), it's not accurate to say that because the core team aren't backporting security fixes that a given version doesn't get security fixes, because the Python provider might be backporting their own security fixes. I do wish more distros were more public about how long they planned to support various Python versions. I still have libraries that support Python 2, and pypistats is my main source of info on the topic!

[hugovk]

Here's a summary of some new things in 3.8: https://www.nicholashairs.com/posts/major-changes-between-python-versions/

There less "big" reasons to drop 3.7 and move up to 3.8 than for earlier releases, so happy to drop those commits and only add support to 3.12.

I still have libraries that support Python 2 [...]

🙀 Well, there's many, many compelling reasons to drop 2.7, but I'm sure you know and won't go into it here, beyond to say that library's download numbers for 2.7 (0.41% for v23.1.1) probably doesn't justify holding on :)

[hugovk]

I saw https://github.com/mahmoud/boltons/issues/339 so I opened https://github.com/mahmoud/boltons/pull/362 :)

[branchvincent]

+1 to at least testing/declaring official support for 3.12

[hugovk]

It's also time to start testing 3.13: https://discuss.python.org/t/python-3-13-0-release-candidate-1-released/59703

But let's get this dealt with first. I can ditch the dropping of 3.7 if you like.

[branchvincent]

I'm just a drive-by commenter (not affiliated with this project). FWIW I agree with dropping 3.7 (pip doesn't even support it now), but if that's not acceptable it would be great to land 3.12 (and soon 3.13) support