Deprecate and remove support for non-UTF 8 source encodings

[AA-Turner]

UTF-8 is now ubiquitous as the default encoding for text, and specifically Unicode. Were Sphinx to be designed today, we would simply expect all source files to be encoded in UTF-8. The default source encoding has been UTF-8 since 2008.

This is a tracking issue for deprecation, and eventual removal, of support for all source encodings other than UTF-8 in Sphinx. If this would cause a significant issue for you, please add a comment explaining your situation. We are open to change in the scheduled plan if there are more people affected than we're aware of.

A current GitHub search shows ~50 affected projects. None of the top 15,000 PyPI packages are affected.

If source_encoding is not set in conf.py, or it is set to any of utf-8, utf-8-sig, or utf8, your project will not be affected by this change.

Plan

1. The next release of Sphinx will contain a warning if the source encoding is explicitly set and is not utf-8, utf8, or utf-8-sig.
2. Sphinx 10 will change this warning to an error.
3. Sphinx 12 or later will remove the source_encoding configuration variable.

A

[jfbu]

Although I expect this to be non relevant here, let me mention in passing that file sphinx/texinputs/LatinRules.xdy although a text file, can not be UTF-8 encoded. It serves as source for the xindy binary. It is never opened per se from Python code, but only copied over to LaTeX build directory.