Dropping python 2 and 3.5 and earlier?

[jquast]

Thoughts,

• some version of setuptools/pip have added support to restrict installations by python version, https://packaging.python.org/en/latest/guides/distributing-packages-using-setuptools/#python-requires
• and so we should be able to safely drop support for these minority languages
• [ ] are the last releases of pip+setuptools for py2.7 able to honor this variable?
• if we drop 2.7, we should also drop 3.5, from this graph from https://pypistats.org/packages/wcwidth

I am a strong believer in backporting and so on, having worked in many restrictive environments, I can sympathize with the small number of the tens of thousands of wcwidth downloads performed every day by python 2.7. And, though they should be version-pinning when working with such legacy software versions, there are cases where they are not.

The reason I held onto python 2 support so long was because it wasn't too difficult, we had found simple 2/3 switches that work in the codebase and the tests have provided nearly 100% coverage.

However I think a majority of the bugs of wcswidth calculations have been resolved, and python 2.7 users can now benefit from this. And that I cannot expect any python 2.7 users will want to make use of any new API's that are discussed in open issues. And for that reason, I am a proponent of dropping support of older python releases.

[vfazio]

For informational purposes: https://devguide.python.org/versions/

[GalaxySnail]

The latest version of setuptools which supports python 2.7 is setuptools 44.1.1 [1], the latest version of pip which supports python 2.7 and 3.5 is pip 20.3.4 [2].

[1] https://setuptools.pypa.io/en/latest/history.html#v45-0-0 [2] https://pip.pypa.io/en/stable/news/#v21-0

If we decide to drop python 2, we can maintain a legacy branch 0.2.x for python 2, and backport bug fixes only (including unicode table updates). The question is: what can we get from dropping python 2 and 3.5 (or even 3.6, 3.7)? Typing on python 2 is difficult, but I don't think #71 is an unsolvable problem, maybe it just need some more investigation.

If we drop python 2 only (requires python>=3.5):

• we can get rid of all of str unicode confusion
• lru_cache and typing are available in stdlib
• syntax for function annotations (PEP 3107): def wcwidth(wc: str, unicode_version: str = 'auto') -> int: ...
• use python -Im instead of python -m in CI
• the latest version of mypy which supports --python-version 3.5 is mypy 1.5 [3][4]

[3] https://mypy-lang.blogspot.com/2023/10/mypy-16-released.html [4] https://github.com/python/mypy/commit/14743a1cdd2a07ecc56ce01cc9d54130fb32931e

If we drop python<=3.5 (requires python>=3.6):

• PEP 526: syntax for variable annotations

If we drop python<=3.6 (requires python>=3.7):

• from __future__ import annotaions so that we can use newer syntax such as tuple[int, ...] and str | None
• unpin virtualenv<20.22.0 and use the latest virtualenv [5]
• we can use the latest version of mypy

[5] https://tox.wiki/en/4.11.3/faq.html#testing-end-of-life-python-versions

IMO currently it seems not worth to maintain 2 different branches. We can reconsider this if #71 is proven unsolvable unless dropping python 2, or we encounter more problems in the future.

[jquast]

I agree with everything you say. I don't have much inclination to do the work to remove python 2 support myself, I would only be inclined to remove it when I am unable to reasonably test it or continue to support it.

On 'What do we get?',

there is little to gain, only to drop the maintenance burden for tasks like #71. It is minor burden for the changes I expect to make, even in the ~30 hours of work I recently dedicated, only ~1 hours work was required to ensure python 2 compatibility.

We have previously discussed the idea of Cython or C-compiled assistance for performance (I will open an issue shortly), and, continuing to support python 2 and 3.5, and, the operating systems and build tools that would accompany it, might be too burdensome to maintain at that time, but I would only worry about it then.

And for typing on the whole, #71, I am in no hurry to drop support for ~1 million downloads/month of "users", for the ~10 developer who might benefit from types, and even then I believe there is little to gain there as this library API is basic, obvious and the types are in the docstrings, it's hardly worth the attention!

On 'it seems not worth to maintain 2 different branches'

I propose to drop python 2/mark a release as "final python 2 release". With #91 and #97 now released for python 2.7, and, 'ucs-detect' tool and results mostly complete, and over time with open number of issues low, I can say with high confidence that wcwidth seems to be >99% accurate for all versions of unicode up to 15.0

And, as for Unicode updates, version 15 is able to accurately write a great majority of the world's languages. I expect only very minority and ancient languages, symbols/emojis, and rare punctuation additions in future releases.

And, that in the case of supporting legacy software that is stuck on python 2.7, I see little need to continue to support python 2.7 for Unicode versions 16+. They would not likely also be using the latest 2023 releases of terminal emulators, and also be in need of viewing characters exclusive to future releases like Unicode 16.

[vfazio]

The pypi stats don't give great insights on what python versions are pulling which versions AFAICT, so I ran a quick query on the public pypi dataset for the past 30 days

SELECT REGEXP_EXTRACT(details.python, r"[0-9]+\.[0-9]+") AS python_version, file.version, COUNT(*) AS num_downloads FROM `bigquery-public-data.pypi.file_downloads` WHERE file.project = 'wcwidth' AND 
  DATE(timestamp) BETWEEN DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY) AND CURRENT_DATE()
GROUP BY `python_version`, file.version
ORDER BY `python_version`, file.version

python_version	version	num_downloads
	0.0.1	24
	0.1.0	24
	0.1.1	22
	0.1.2	23
	0.1.3	21
	0.1.4	71
	0.1.5	80
	0.1.6	78
	0.1.7	34878
	0.1.8	18099
	0.1.9	12046
	0.2.0	153
	0.2.1	23
	0.2.10	563454
	0.2.11	75314
	0.2.12	80469
	0.2.2	99
	0.2.3	176
	0.2.4	2942
	0.2.5	608248
	0.2.6	2400764
	0.2.7	16711
	0.2.8	1433167
	0.2.9	1664501
2.7	0.1.4	6
2.7	0.1.5	212
2.7	0.1.6	202
2.7	0.1.7	46574
2.7	0.1.8	6904
2.7	0.1.9	19847
2.7	0.2.10	170671
2.7	0.2.11	29474
2.7	0.2.12	40155
2.7	0.2.2	53
2.7	0.2.3	71
2.7	0.2.4	426
2.7	0.2.5	77507
2.7	0.2.6	42646
2.7	0.2.7	133
2.7	0.2.8	199542
2.7	0.2.9	327602
3.1	0.1.1	2
3.1	0.1.3	36
3.1	0.1.4	1
3.1	0.1.5	4
3.1	0.1.6	2314
3.1	0.1.7	25206
3.1	0.1.8	10131
3.1	0.1.9	48818
3.1	0.2.0	138
3.1	0.2.10	2047626
3.1	0.2.11	252603
3.1	0.2.12	334320
3.1	0.2.2	132
3.1	0.2.3	203
3.1	0.2.4	823
3.1	0.2.5	330516
3.1	0.2.6	1083648
3.1	0.2.7	5479
3.1	0.2.8	2525359
3.1	0.2.9	4161309
3.11	0.1.4	5
3.11	0.1.5	1
3.11	0.1.6	3
3.11	0.1.7	20459
3.11	0.1.8	1376
3.11	0.1.9	25514
3.11	0.2.0	96
3.11	0.2.10	1112439
3.11	0.2.11	151078
3.11	0.2.12	178585
3.11	0.2.2	110
3.11	0.2.3	524
3.11	0.2.4	216
3.11	0.2.5	112013
3.11	0.2.6	418648
3.11	0.2.7	6204
3.11	0.2.8	1394119
3.11	0.2.9	2199217
3.12	0.0.1	1
3.12	0.1.0	1
3.12	0.1.1	1
3.12	0.1.2	1
3.12	0.1.3	1
3.12	0.1.4	6
3.12	0.1.5	4
3.12	0.1.6	3
3.12	0.1.7	291
3.12	0.1.8	106
3.12	0.1.9	1472
3.12	0.2.0	222
3.12	0.2.10	221023
3.12	0.2.11	32029
3.12	0.2.12	36664
3.12	0.2.2	3
3.12	0.2.3	4
3.12	0.2.4	9
3.12	0.2.5	6580
3.12	0.2.6	5668
3.12	0.2.7	46
3.12	0.2.8	204408
3.12	0.2.9	401904
3.13	0.1.8	2
3.13	0.1.9	7
3.13	0.2.10	199
3.13	0.2.11	18
3.13	0.2.12	44
3.13	0.2.5	24
3.13	0.2.6	18
3.13	0.2.8	385
3.13	0.2.9	728
3.3	0.1.7	2
3.3	0.2.10	7
3.3	0.2.11	4
3.3	0.2.12	4
3.3	0.2.5	2
3.3	0.2.6	2
3.3	0.2.8	3
3.3	0.2.9	9
3.4	0.1.6	1
3.4	0.1.7	281
3.4	0.2.10	388
3.4	0.2.11	48
3.4	0.2.12	60
3.4	0.2.5	3
3.4	0.2.6	3
3.4	0.2.8	481
3.4	0.2.9	862
3.5	0.1.5	12
3.5	0.1.6	1
3.5	0.1.7	3258
3.5	0.1.8	40
3.5	0.1.9	317
3.5	0.2.10	5887
3.5	0.2.11	828
3.5	0.2.12	1051
3.5	0.2.3	16
3.5	0.2.4	2698
3.5	0.2.5	5024
3.5	0.2.6	194
3.5	0.2.7	6
3.5	0.2.8	5840
3.5	0.2.9	9973
3.6	0.1.7	51450
3.6	0.1.8	6732
3.6	0.1.9	8665
3.6	0.2.0	84
3.6	0.2.10	684398
3.6	0.2.11	83264
3.6	0.2.12	99211
3.6	0.2.2	31
3.6	0.2.3	9
3.6	0.2.4	546
3.6	0.2.5	54253
3.6	0.2.6	37351
3.6	0.2.7	192
3.6	0.2.8	766426
3.6	0.2.9	1289951
3.7	0.1.4	1
3.7	0.1.5	54
3.7	0.1.6	116
3.7	0.1.7	181080
3.7	0.1.8	31894
3.7	0.1.9	108883
3.7	0.2.0	34
3.7	0.2.10	1587073
3.7	0.2.11	176225
3.7	0.2.12	222093
3.7	0.2.2	697
3.7	0.2.3	1601
3.7	0.2.4	14324
3.7	0.2.5	765784
3.7	0.2.6	201724
3.7	0.2.7	830
3.7	0.2.8	1877494
3.7	0.2.9	3064506
3.8	0.0.1	1
3.8	0.1.0	1
3.8	0.1.3	5
3.8	0.1.4	2
3.8	0.1.6	480
3.8	0.1.7	56617
3.8	0.1.8	120587
3.8	0.1.9	164427
3.8	0.2.0	277
3.8	0.2.10	2216282
3.8	0.2.11	259522
3.8	0.2.12	327542
3.8	0.2.2	289
3.8	0.2.3	547
3.8	0.2.4	4191
3.8	0.2.5	602905
3.8	0.2.6	429148
3.8	0.2.7	2981
3.8	0.2.8	2516766
3.8	0.2.9	4498352
3.9	0.0.1	2
3.9	0.1.4	3
3.9	0.1.5	2
3.9	0.1.6	6
3.9	0.1.7	36367
3.9	0.1.8	8682
3.9	0.1.9	49313
3.9	0.2.0	231
3.9	0.2.10	1369850
3.9	0.2.11	194189
3.9	0.2.12	226996
3.9	0.2.2	249
3.9	0.2.3	3838
3.9	0.2.4	3016
3.9	0.2.5	469177
3.9	0.2.6	1070897
3.9	0.2.7	5369
3.9	0.2.8	1658737
3.9	0.2.9	2633441

[hugovk]

Please see https://github.com/jquast/wcwidth/pull/117 to drop 2.7 and 3.5.

And for typing on the whole, #71, I am in no hurry to drop support for ~1 million downloads/month of "users", for the ~10 developer who might benefit from types, and even then I believe there is little to gain there as this library API is basic, obvious and the types are in the docstrings, it's hardly worth the attention!

You can add me as number 11 who would like types :) I also maintain PrettyTable which accounts for about 10 million downloads/month of wcwidth, around 20%.