SciTools
Further problems with locale
As reported here in the context of iris-esmf-regrid, the current save - modify - restore code for the locale setting it seems does not always work correctly. This was reported in #435, fixed by #436, introduced in v3.3.0
Not clear yet what is actually wrong, but for iris-esmf-regrid it seems to occur specifically in readthedocs builds. The case given should hopefully point to it.
Detail
ReadTheDocs log captured here from a ReadTheDocs build test (i.e. its error log) on commit https://github.com/SciTools/iris-esmf-regrid/pull/431/commits/b5edcd79023e21d067e43ddb343ee4523b06e960 , in https://github.com/SciTools/iris-esmf-regrid/pull/431. Since we can't expect that record to persist
Traceback (most recent call last):
File "/home/docs/checkouts/readthedocs.org/user_builds/iris-esmf-regrid/conda/431/lib/python3.10/site-packages/esmf_regrid-0.12.dev0-py3.10.egg/esmf_regrid/init.py", line 11, in
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/docs/checkouts/readthedocs.org/user_builds/iris-esmf-regrid/conda/431/lib/python3.10/site-packages/sphinx/config.py", line 529, in eval_config_file
exec(code, namespace) # NoQA: S102
File "/home/docs/checkouts/readthedocs.org/user_builds/iris-esmf-regrid/checkouts/431/docs/src/conf.py", line 34, in
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "/home/docs/checkouts/readthedocs.org/user_builds/iris-esmf-regrid/conda/431/lib/python3.10/site-packages/sphinx/cmd/build.py", line 496, in build_main app = Sphinx( File "/home/docs/checkouts/readthedocs.org/user_builds/iris-esmf-regrid/conda/431/lib/python3.10/site-packages/sphinx/application.py", line 238, in init self.config = Config.read(self.confdir, confoverrides or {}, self.tags) File "/home/docs/checkouts/readthedocs.org/user_builds/iris-esmf-regrid/conda/431/lib/python3.10/site-packages/sphinx/config.py", line 316, in read namespace = eval_config_file(filename, tags) File "/home/docs/checkouts/readthedocs.org/user_builds/iris-esmf-regrid/conda/431/lib/python3.10/site-packages/sphinx/config.py", line 542, in eval_config_file raise ConfigError(msg % traceback.format_exc()) from exc sphinx.errors.ConfigError: There is a programmable error in your configuration file:
Traceback (most recent call last):
File "/home/docs/checkouts/readthedocs.org/user_builds/iris-esmf-regrid/conda/431/lib/python3.10/site-packages/esmf_regrid-0.12.dev0-py3.10.egg/esmf_regrid/init.py", line 11, in
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/docs/checkouts/readthedocs.org/user_builds/iris-esmf-regrid/conda/431/lib/python3.10/site-packages/sphinx/config.py", line 529, in eval_config_file
exec(code, namespace) # NoQA: S102
File "/home/docs/checkouts/readthedocs.org/user_builds/iris-esmf-regrid/checkouts/431/docs/src/conf.py", line 34, in
Configuration error: There is a programmable error in your configuration file:
Traceback (most recent call last):
File "/home/docs/checkouts/readthedocs.org/user_builds/iris-esmf-regrid/conda/431/lib/python3.10/site-packages/esmf_regrid-0.12.dev0-py3.10.egg/esmf_regrid/init.py", line 11, in
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/docs/checkouts/readthedocs.org/user_builds/iris-esmf-regrid/conda/431/lib/python3.10/site-packages/sphinx/config.py", line 529, in eval_config_file
exec(code, namespace) # NoQA: S102
File "/home/docs/checkouts/readthedocs.org/user_builds/iris-esmf-regrid/checkouts/431/docs/src/conf.py", line 34, in
To understand what is going on, it would be useful to print the value that is causing the error. Do you think you could modify the build so it prints that out?
from @SciTools/peloton:
AFAIK: For locale, setting LC_ALL=C should set sensible defaults for all the local settings...(such as LC_NUMERIC)
@ESadek-MO I am actually doing that in here https://github.com/ESMValGroup/ESMValCore/pull/2706 and that doesn't really help much; note that we are seeing flakiness in this fail, it's about 80% fail 20% pass
also note these rather relevant Sphinx issues: https://github.com/sphinx-doc/sphinx/issues/13485 (very recent) and https://github.com/sphinx-doc/sphinx/issues/11739 (probably marginally related)
woop: actually setting LC_ALL=C may actually do the trick, two test passes out of two so far
EDIT: that IS the solution, consistent pass rate, no fails; do you have an idea why the locale gets altered so that it needs restting to C? I thought the C locale was the default for Linux machines
From @SciTools/peloton: We got this info from @ukmo-ccbunney who isn't available right now. Locale stuff is a bit of a mystery to us, but we're happy that this workaround "just works".
@stephenworsley it'd be rather useful uf we could fibd out if something in iris is changing the locale settings. My hunch is it's nothing in iris though, since a) it started happening (to us, at least) all of a sudden, and b) others are reporting similar issues to Sphinx.
it'd be rather useful uf we could fibd out if something in iris is changing the locale settings.
We know that it's not iris because we've observed this issue here in cf-units (and that's not dependent on iris)
Not sure how to progress or investigate this ...
If you could print out the value of lc_numeric in this code when the crash happens, that would be helpful:
https://github.com/SciTools/cf-units/blob/main/cf_units/init.py#L180-L191
Somehow locale.getlocale(locale.LC_NUMERIC) seems to return a locale that is invalid or not installed.
@bjlittle some info from https://github.com/ESMValGroup/ESMValCore/pull/2706#issuecomment-2862974538 - I tested with cf-units==3.3.0 and all is fine, unless one has iris>=3.11 and Python==3.13.3, there was that change in 3.13.3 that I quote in the comment; we don't see any issues with Python<3.13.3; there may be multiple problems playing up at the same time, but whatever it hits us in the docs build is not happening without iris>=3.11 and Python==3.13.3 🍺
also, as a kluge we found out that by resetting the locale in our sphinx conf.py (see https://github.com/ESMValGroup/ESMValCore/pull/2708/files) with:
import locale
locale.setlocale(locale.LC_ALL, 'C')
that fixes the problem, but the cause is yet to be identified
Some further debugging results of the recent issue: restoring the locale to the previous setting works fine with Python 3.13.2 but does not work anymore with Python 3.13.3. Possibly this is caused by the changes in https://github.com/python/cpython/pull/129647.
Python 3.13.3 example:
$ docker run -it python:3.13.3
Python 3.13.3 (main, May 9 2025, 23:49:05) [GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os, locale
>>> os.environ
environ({'PATH': '/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', 'HOSTNAME': '2e43b79a7135', 'TERM': 'xterm', 'GPG_KEY': '7169605F62C751356D054A26A821E680E5FA6305', 'PYTHON_VERSION': '3.13.3', 'PYTHON_SHA256': '40f868bcbdeb8149a3149580bb9bfd407b3321cd48f0be631af955ac92c0e041', 'HOME': '/root', 'LC_CTYPE': 'C.UTF-8'})
>>> os.environ["LC_ALL"] = "C.UTF-8"
>>> locale.setlocale(locale.LC_ALL, "")
'C.UTF-8'
>>> locale.getlocale(locale.LC_NUMERIC)
('en_US', 'UTF-8')
>>> locale.setlocale(locale.LC_NUMERIC, locale.getlocale(locale.LC_NUMERIC))
Traceback (most recent call last):
File "<python-input-5>", line 1, in <module>
locale.setlocale(locale.LC_NUMERIC, locale.getlocale(locale.LC_NUMERIC))
~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.13/locale.py", line 615, in setlocale
return _setlocale(category, locale)
locale.Error: unsupported locale setting
Python 3.13.2 example:
$ docker run -it python:3.13.2
Python 3.13.2 (main, Apr 8 2025, 04:27:11) [GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os, locale
>>> os.environ
environ({'PATH': '/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', 'HOSTNAME': '64800af765e4', 'TERM': 'xterm', 'GPG_KEY': '7169605F62C751356D054A26A821E680E5FA6305', 'PYTHON_VERSION': '3.13.2', 'PYTHON_SHA256': 'd984bcc57cd67caab26f7def42e523b1c015bbc5dc07836cf4f0b63fa159eb56', 'HOME': '/root', 'LC_CTYPE': 'C.UTF-8'})
>>> os.environ["LC_ALL"] = "C.UTF-8"
>>> locale.setlocale(locale.LC_ALL, "")
'C.UTF-8'
>>> locale.getlocale(locale.LC_NUMERIC)
('C', 'UTF-8')
>>> locale.setlocale(locale.LC_NUMERIC, locale.getlocale(locale.LC_NUMERIC))
'C.UTF-8'
Opened https://github.com/python/cpython/issues/133967 to ask if this is a bug or intended behaviour.
Opened python/cpython#133967 to ask if this is a bug or intended behaviour.
@bouweandela can you confirm this was fixed in Python 3.13.5?
https://docs.python.org/3/whatsnew/changelog.html#id3
Yes, it was indeed fixed, so https://github.com/python/cpython/issues/133967 is only present in Python 3.13.3 and 3.13.4. However, this issue pre-dates the release of Python 3.13.3, so I do not think we have gotten to the root of things here.