Repository navigation
OpenSSL with FIPS enabled causes test_hashlib failures #128071
Description
Bug report
Bug description:
We are using OpenSSL 3.3.1 with FIPS enabled. When compiling Python version 3.13.1 with --enable-optimizations, test_hashlib will fail with the following.
... fetching http://www.pythontest.net/hashlib/blake2b.txt ...
.. fetching http://www.pythontest.net/hashlib/blake2s.txt ...
.............ss................. fetching http://www.pythontest.net/hashlib/sha3_224.txt ...
.. fetching http://www.pythontest.net/hashlib/sha3_256.txt ...
.. fetching http://www.pythontest.net/hashlib/sha3_384.txt ...
.. fetching http://www.pythontest.net/hashlib/sha3_512.txt ...
..... fetching http://www.pythontest.net/hashlib/shake_128.txt ...
. fetching http://www.pythontest.net/hashlib/shake_256.txt ...
.......................EE...E
======================================================================
ERROR: test_usedforsecurity_false (__main__.HashLibTestCase.test_usedforsecurity_false)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/user/Python-3.13.1/Lib/test/test_hashlib.py", line 246, in test_usedforsecurity_false
self._hashlib.new("md5", usedforsecurity=False)
~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
_hashlib.UnsupportedDigestmodError: [digital envelope routines] unsupported
======================================================================
ERROR: test_usedforsecurity_true (__main__.HashLibTestCase.test_usedforsecurity_true)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/user/Python-3.13.1/Lib/test/test_hashlib.py", line 235, in test_usedforsecurity_true
self._hashlib.new("md5", usedforsecurity=True)
~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
_hashlib.UnsupportedDigestmodError: [digital envelope routines] unsupported
======================================================================
ERROR: test_scrypt (__main__.KDFTests.test_scrypt)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/user/Python-3.13.1/Lib/test/test_hashlib.py", line 1122, in test_scrypt
result = hashlib.scrypt(password, salt=salt, n=n, r=r, p=p)
ValueError: [digital envelope routines] unsupported
----------------------------------------------------------------------
Ran 78 tests in 24.957s
FAILED (errors=3, skipped=2)I believe there was a change with 3.12 where test failures no longer fail silently so it would make sense why our Python 3.11 build does not fail.
I did some digging and I found the following PRs and issue which I believe pertain to what I am seeing:
- gh-127298: When in FIPS mode ensure builtin hashes check for usedforsecurity=False #127301
- gh-127298: Refactor test_hashlib for better usedforsecurity & openssl fips mode env support. #127492
- Ensure builtin hashlib implementations honor usedforsecurity=True when _hashlib is in FIPS mode #127298
CPython versions tested on:
3.13
Operating systems tested on:
Linux
Activity
ghmj2417 commentedon Dec 30, 2024
Is there a path forward without waiting on a fix? For example, allowing the tests to fail silently like in 3.11? Or is this something OpenSSL users with FIPS will have to wait for the hashlib test to be updated? It's definitely not dire straights since we can continue to use 3.11, but we would like to move to 3.13.
picnixz commentedon Dec 31, 2024
I think we'll just skip the tests if the underlying hash functions are not here.
cc @gpshead who r4ecently added the decorator for skipping.
gpshead commentedon Dec 31, 2024
"OpenSSL with FIPS enabled" is not well defined and allows for arbitrarily unusable configurations - intentionally injecting dynamic changes into a process from outside that it was never written for or tested with. "It" (quoted because "OpenSSL with FIPS enabled" is undefined) is not a a PEP-11 tier supported platform. Look at #127298 and the related discuss thread for a better understanding of why.
Ignore the test suite failures you see there when running in such a mode today. They're unimportant. What matters is whether your own application testing works. Python is a programming language, not an application.
We'll probably have our own test suite better protected from what "it" is defined as at the moment in the future based on some ongoing work some of which you've linked to. No guarantees. But since "it" changes and nobody has signed up to support it beyond redistributors for their own repackaged builds of CPython, don't assume upstream CPython will pass in such an environment.
ghmj2417 commentedon Dec 31, 2024
That's the problem, failures are no longer ignored. These test failures were made somewhat important with 3.13. I stated this above. "I believe there was a change with 3.12 where test failures no longer fail silently so it would make sense why our Python 3.11 build does not fail." I was incorrect in saying it was 3.12. It looks like the changes were not backported. I believe this was the change, #110295.
As you said, what matters is how our application works, but we cannot get to that point if the compiling of 3.13 is erring out because of a failed test. Is there a way for us to tell the tests to fail silently like in 3.11?
gpshead commentedon Dec 31, 2024
Yep, sounds like you'll need to work around it on your end for now.
ghmj2417 commentedon Dec 31, 2024
I get it, changes to the hashlib test may or may not happen. I will follow the issues, discussions, and PRs posted in this ticket to see what happens. But what I am trying to understand, is there a workaround to skip tests or ignore test failures so we can compile 3.13 without failing the make script on the hashlib test? An argument passed during configure? An environmental variable?
I can certainly create my own patch to skip those tests, or add back
|| trueduring our build, or some other change. I just was hoping there might be something simpler (disabling FIPS mode is not an option).picnixz commentedon Dec 31, 2024
Those tests are needed (ok, not mandatory but helpful) for the PGO build but I think you can tweak
PROFILE_TASKin Makefile and replace-m test --pgoby-m test --pgo -i test_hashlib. The hashlib tests would be skipped. For that, I thinkmake PROFILE_TASK="-m test --pgo -i test_hashlib" -s -j4should be sufficient to compile without generating profile from test_hashlib; otherwise you'll need to specify the tests to run for creating the profile. I haven't test this though, (I don't know if the
-iflag can be combined with--pgo).You can also tweak the PGO tests here:
cpython/Lib/test/libregrtest/pgo.py
Line 27 in c5438fd
EDIT: replaced --pgo-extended with --pgo
ghmj2417 commentedon Dec 31, 2024
I was able to get a successful build using the following:
make PROFILE_TASK="-m test --pgo -i test_hashlib" -j16Thank you very very much for the tip.
gpshead commentedon Dec 31, 2024
On the cpython infra side if we update one of our "FIPS" config buildbot configs to use --enable-optimizations we'll notice and likely address trivia like this sooner.
ghmj2417 commentedon Jan 1, 2025
I'll go ahead and close this issue since we have a workaround. I'll follow the discussions, issues, and PRs regarding the potential test_hashlib changes.
fix(agent,tools): add fallback hashing for internal legacy digests