Repository navigation
test_re fails on Alpine Linux (with musl) #131342
Description
Activity
- changed the title
[-]test_locale_caching and test_locale_compiled failing while installing python3.13[/-][+]test_re fails on Alpine Linux (with musl)[/+]on Mar 17, 2025 Do you have
en_US_iso88591installed? Check with:locale -a | grep en_USBoth of the tests depend on it.
I presume it is setting a locale that is not available for some reason.
for loc in 'en_US.iso88591', 'en_US.utf8': try: locale.setlocale(locale.LC_CTYPE, loc) except locale.Error: # Unsupported locale on this system self.skipTest('test needs %s locale' % loc)I am unable to reproduce after deleting locale:
test_locale_caching (test.test_re.ReTests.test_locale_caching) ... skipped 'test needs en_US.iso88591 locale' test_locale_compiled (test.test_re.ReTests.test_locale_compiled) ... skipped 'test needs en_US.iso88591 locale'@StanFromIreland: Do you link Python to the musl C library?
@vstinner: I should have specified, I did not. I am not sure whether it is even packaged for Fedora.
musl is package in Fedora, but I failed to build Python using musl-gcc wrapper. I built Python in Alpine to get musl.
Reacted by Stan UlbrychPlease add the following lines at the beginning of
test_locale_compiled()and show the output.print() print(f'LC_CTYPE: {locale.setlocale(locale.LC_CTYPE)}') print(f'LC_COLLATE: {locale.setlocale(locale.LC_COLLATE)}') print(f'LC_ALL: {locale.setlocale(locale.LC_ALL)}')
Please add the following lines at the beginning of test_locale_compiled() and show the output.
Here you have:
test_locale_compiled (test.test_re.ReTests.test_locale_compiled) ... LC_CTYPE: C.UTF-8 LC_COLLATE: C LC_ALL: C.UTF-8;C;C;C;C;C FAILUsing musl, setlocale() doesn't fail even if the locale isn't supported:
$ ./python Python 3.14.0a5+ (heads/main-dirty:fe186d76cfd, Mar 11 2025, 08:54:51) [GCC 14.2.0] on linux >>> import locale >>> locale.setlocale(locale.LC_CTYPE, 'en_US.iso88591') 'en_US.iso88591' >>> locale.getencoding() 'UTF-8'Another example:
>>> import locale >>> locale.setlocale(locale.LC_CTYPE, 'donotexist') 'donotexist' >>> locale.getencoding() 'UTF-8'Reacted by Stan UlbrychThe PR covers both the tests here already. I guess the PR can be assigned this issue?
I have roughly identified the reasons:
- Alpine does not include the
en_US_iso88591encoding by default. - The implementations of the
setlocalefunction in musl and glibc are inconsistent. For musl, passing any value will not cause an error. Therefore, even iflocale.setlocale(locale.LC_CTYPE, 'en_US.iso88591')does not throw an exception, it will not work properly.
When I run the code in two environments, the behavior is different
locale.setlocale(locale.LC_CTYPE, "xxx.xxx")
glibc:
locale.setlocale(locale.LC_CTYPE, "xxx.xxx") Traceback (most recent call last): File "/home/deepwzh/opensource/python_demo/test.py", line 9, in <module> locale.setlocale(locale.LC_CTYPE, "xxx.xxx") File "/home/deepwzh/miniconda3/lib/python3.12/locale.py", line 615, in setlocale return _setlocale(category, locale) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^musl:
No error is thrown.Further, you can refer to the musl source code here:
https://github.com/bminor/musl/blob/c47ad25ea3b484e10326f933e927c0bc8cded3da/src/locale/locale_map.c#L99
From this, it can be seen that if a non-existent locale is passed, it will actually default to UTF-8 encoding instead of throwing an error.Reacted by Victor Stinner- Alpine does not include the
Bug report
Bug description:
CPython versions tested on:
3.13
Operating systems tested on:
Linux