Recording and restoring the LC_NUMERIC locale setting with LC_ALL=C.UTF-8 no longer works with Python 3.13.3

Bug report

Bug description:

We have project where we record the current value of locale.LC_NUMERIC, change it to a different value to use a library that needs a particular locale setting, and then restore it to the original value. This stopped working with Python 3.13.3. I suspect this may be a bug, but please correct me if I misunderstood the functionality.

Example with Python 3.13.3:

$ docker run -it python:3.13.3 
Python 3.13.3 (main, May  9 2025, 23:49:05) [GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os, locale
>>> os.environ
environ({'PATH': '/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', 'HOSTNAME': '2e43b79a7135', 'TERM': 'xterm', 'GPG_KEY': '7169605F62C751356D054A26A821E680E5FA6305', 'PYTHON_VERSION': '3.13.3', 'PYTHON_SHA256': '40f868bcbdeb8149a3149580bb9bfd407b3321cd48f0be631af955ac92c0e041', 'HOME': '/root', 'LC_CTYPE': 'C.UTF-8'})
>>> os.environ["LC_ALL"] = "C.UTF-8"
>>> locale.setlocale(locale.LC_ALL, "")
'C.UTF-8'
>>> locale.getlocale(locale.LC_NUMERIC)
('en_US', 'UTF-8')
>>> locale.setlocale(locale.LC_NUMERIC, locale.getlocale(locale.LC_NUMERIC))
Traceback (most recent call last):
  File "<python-input-5>", line 1, in <module>
    locale.setlocale(locale.LC_NUMERIC, locale.getlocale(locale.LC_NUMERIC))
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/locale.py", line 615, in setlocale
    return _setlocale(category, locale)
locale.Error: unsupported locale setting

The same example works fine with earlier versions of Python, for example 3.13.2:

$ docker run -it python:3.13.2 
Python 3.13.2 (main, Apr  8 2025, 04:27:11) [GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os, locale
>>> os.environ
environ({'PATH': '/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', 'HOSTNAME': '64800af765e4', 'TERM': 'xterm', 'GPG_KEY': '7169605F62C751356D054A26A821E680E5FA6305', 'PYTHON_VERSION': '3.13.2', 'PYTHON_SHA256': 'd984bcc57cd67caab26f7def42e523b1c015bbc5dc07836cf4f0b63fa159eb56', 'HOME': '/root', 'LC_CTYPE': 'C.UTF-8'})
>>> os.environ["LC_ALL"] = "C.UTF-8"
>>> locale.setlocale(locale.LC_ALL, "")
'C.UTF-8'
>>> locale.getlocale(locale.LC_NUMERIC)
('C', 'UTF-8')
>>> locale.setlocale(locale.LC_NUMERIC, locale.getlocale(locale.LC_NUMERIC))
'C.UTF-8'

It looks like this may have been caused by the changes in #129647.

CPython versions tested on:

3.13

Operating systems tested on:

No response

Linked PRs

I'm at the CPython sprints right now, so I can take a look at this.

While I'm unable to reproduce your exact issue, I'm debugging a slightly similar issue that I'm having on my machine. Instead of the error, I'm getting this:

Type "help", "copyright", "credits" or "license" for more information.
>>> import os, locale
>>> os.environ
environ({'PATH': '/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', 'HOSTNAME': '64800af765e4', 'TERM': 'xterm', 'GPG_KEY': '7169605F62C751356D054A26A821E680E5FA6305', 'PYTHON_VERSION': '3.13.2', 'PYTHON_SHA256': 'd984bcc57cd67caab26f7def42e523b1c015bbc5dc07836cf4f0b63fa159eb56', 'HOME': '/root', 'LC_CTYPE': 'C.UTF-8'})
>>> os.environ["LC_ALL"] = "C.UTF-8"
>>> locale.setlocale(locale.LC_ALL, "")
'C.UTF-8'
>>> locale.getlocale(locale.LC_NUMERIC)
('en_US, UTF-8)
>>> locale.setlocale(locale.LC_NUMERIC, ('C', 'UTF-8'))
'en_US.UTF-8'

Thank you for taking the time to investigate! Are you using the Python containers from Dockerhub? In my tests with those containers, the problem only appears with Python 3.13.3, while for you, it seems to partially appear in 3.13.2, but without the crash when trying to set locale.LC_NUMERIC. Could you try if you can reproduce it with Python 3.13.3?

Ah, this is for 3.13.3!

@mvkashyap

many thanks @mvkashyap 🍺

@malemburg

This was caused by #129646.

The locale.aliases file from X.org distribution (https://cgit.freedesktop.org/xorg/lib/libX11/tree/nls/locale.alias.pre) maps locale C.utf8 to en_US.UTF-8. The problem is that en_US.UTF-8 may not exist on all platforms, especially on the old ones. This is a congenital defect in the locale module design -- that it uses a static platform-independent mapping.

cc @malemburg

I think the problem is different: the en_US locale files may not have been generated on the base system, whereas the C locale should always be present.

In any case, we should probably special case the C local mappings in the script Tools/i18n/makelocalealias.py to not have them alias to en_US (even though that is what they normally refer to).

BTW: I do wonder why the locale files don't exist in the official Python docker image.

Note that locale.getlocale()returns ('en_US', 'UTF-8') in the 3.13.3 image, but the /usr/share/locale is missing the en_US subdir. In the 3.12.3 image, it returns ('C', 'UTF-8'), which is present. This image is also missing the en_US subdir.

This suggests that the official images are somewhat broken... where can this be reported ?

Checking using locale -a, only the locales C, C.utf8 and POSIX are available in those images, so I guess they are based on a minimal installation of Debian.

pythongh-133967: Do not normalize locale name 'C.UTF-8' to 'en_US.UTF-8'

In any case, we should probably special case the C local mappings in the script Tools/i18n/makelocalealias.py to not have them alias to en_US (even though that is what they normally refer to).

This is what I came to. It is not clear what to do with the C.ISO8859-1 -> en_US.ISO8859-1 mapping which has existed for a long time, but at least it is not so common as C.UTF-8.

gh-133967: Do not normalize locale name 'C.UTF-8' to 'en_US.UTF-8' (#…

pythongh-133967: Do not normalize locale name 'C.UTF-8' to 'en_US.UTF…

It is not clear what to do with the C.ISO8859-1 -> en_US.ISO8859-1 mapping

Let's leave that as it is, since it hasn't caused any reported issues, AFAIK.

en_US.ISO8859-1 is much more likely to exist. It still makes me very uncomfortable (as mentioned on Discord) because we're changing the encoding in a patch release, but it's not entirely clear to me when and how this could affect people.

For some people changing the encoding was a bug fix, for others a regression. This is a problem with such approach -- there may be different default encodings for the same locale on different platforms, but the locale module will use the same mapping. There are even conflicts between glibc and X.org mappings, so this problem is not Python specific.

[3.14] gh-133967: Do not normalize locale name 'C.UTF-8' to 'en_US.UT…

[3.13] gh-133967: Do not normalize locale name 'C.UTF-8' to 'en_US.UT…

pythongh-133967: Do not normalize locale name 'C.UTF-8' to 'en_US.UTF…

Navigation Menu

Repository navigation

Recording and restoring the LC_NUMERIC locale setting with LC_ALL=C.UTF-8 no longer works with Python 3.13.3 #133967

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Linked PRs

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Recording and restoring the LC_NUMERIC locale setting with LC_ALL=C.UTF-8 no longer works with Python 3.13.3 #133967

Description

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Linked PRs

Activity

mvkashyap commented on May 19, 2025

mvkashyap commented on May 19, 2025

bouweandela commented on May 20, 2025

mvkashyap commented on May 20, 2025

valeriupredoi commented on May 22, 2025

serhiy-storchaka commented on Jun 10, 2025

malemburg commented on Jun 10, 2025

malemburg commented on Jun 10, 2025

malemburg commented on Jun 10, 2025

serhiy-storchaka commented on Jun 10, 2025

malemburg commented on Jun 10, 2025

Yhg1s commented on Jun 10, 2025

serhiy-storchaka commented on Jun 10, 2025

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions