hashlib.new() allows MD5 in FIPS mode

[iwane-pl]

Bug report

Bug description:

On a system with FIPS mode enabled, hashlib.new() allows MD5 hash to be created even without using usedforsecurity flag. hashlib.md5() call fails, as expected.

(venv) [me@fipstest]$ python
Python 3.12.2 (main, Feb 21 2024, 07:05:18) [GCC 8.5.0 20210514 (Red Hat 8.5.0-20)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import hashlib
>>> hashlib._hashlib.get_fips_mode()
1
>>> md = hashlib.new('md5')   
>>> md.update(b'abcdef')
>>> md.digest()
b'\xe8\x0bP\x17\t\x89P\xfcX\xaa\xd8<\x8c\x14\x97\x8e'
>>> hashlib.md5(b'abcdefgh')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: [digital envelope routines: EVP_DigestInit_ex] disabled for FIPS
>>> hashlib.md5
<built-in function openssl_md5>
>>> md
<_md5.md5 object at 0x7f3f71d1d6f0>

[me@fipstest]$ cat /etc/os-release 
NAME="Rocky Linux"
VERSION="8.9 (Green Obsidian)"
...

[me@fipstest]$ dnf info openssl
Installed Packages
Name         : openssl
Epoch        : 1
Version      : 1.1.1k
Release      : 12.el8_9
Architecture : x86_64
Size         : 1.1 M
Source       : openssl-1.1.1k-12.el8_9.src.rpm
Repository   : @System
From repo    : baseos
Summary      : Utilities from the general purpose cryptography library with TLS implementation
URL          : http://www.openssl.org/
License      : OpenSSL and ASL 2.0
Description  : The OpenSSL toolkit provides support for secure communications between
             : machines. OpenSSL includes a certificate management tool and shared
             : libraries which provide various cryptographic algorithms and
             : protocols.

CPython versions tested on:

3.12

Operating systems tested on:

Linux

[stratakis]

This seems expected as hashlib.new falls back in this case to python's internal md5 implementation and not the openssl one. If you compile with --without-builtin-hashlib-hashes (or --with-builtin-hashlib-hashes=) this should be fixed. However I guess it can be improved to not even allow the fallback in case fips mode is enabled but then Python would have to start tracking the FIPS related restrictions and their updates and apply those to the hashlib module.

[iwane-pl]

Thanks for replying!

After reading the docs I was under impression that the fallback should happen when usedforsecurity=False in FIPS mode. Or maybe new() shouldn't have that parameter at all.

The way it is done now confused me with some XPASS tests ("Hey... Why do they pass if we use MD5 here?").

I'll try to compile Python without built-in hashes and see what fails then.

[stratakis]

The usedforsecurity=False would allow the openssl md5 digest to be created. You can observe the behaviour of the system python which should be what you want. Also just for reference here is the FIPS patch implementation for python3.12 on rhel8, some bits may be more upstreamable than others but I haven't moved it forward yet.

[iwane-pl]

The usedforsecurity=False would allow the openssl md5 digest to be created.

That's what I expected. The default value is True, though, so I expected both md5() and new('md5') to fail.

I tried compiling Python with --without-builtin-hashlib-hashes, however without patches it fails in some places due to missing algorithms. It seems that we'd need to patch it with the FIPS patch you linked to work. For now it's a rabbit hole I don't want to dive in unless someone complains... :)

I'll leave this issue open for the contributors to decide if it's worth any action on Python's side.

Thank you for help!

[stratakis]

cc'ing @tiran

[encukou]

CPython doesn't support FIPS mode. Supporting it would be a new feature. (Or some might call it a mis-feature, since all it does is limit what you can do.)
What would be the reason for supporting it?

You can query OpenSSL's EVP_default_properties_is_fips_enabled setting through the (private, internal, unstable, unsupported) _hashlib module, but that only tells you what OpenSSL will do; the built-in hash implementations ignore it.

Similarly, AFAIK the usedforsecurity argument is passed to OpenSSL if the OpenSSL backend is used. The built-in implementations accept it but ignore it.

[gpshead]

Feel free to join the broader discussion to the recently started https://discuss.python.org/t/python-and-openssl-fips-mode/51389 thread.

If you don't want non-OpenSSL hash implementations to be present so no fallbacks exist, I suggest building with --with-builtin-hashlib-hashes=.

[gpshead]

see also #118224.

[xnox]

If you don't want non-OpenSSL hash implementations to be present so no fallbacks exist, I suggest building with --with-builtin-hashlib-hashes=.

When configuring python in that way, lots of tracebacks are shown upon import hashlib even when one doesn't try to use md5.

I don't think it's ok, to close this issue (and also dissmiss my PR) when proposed "solutions" in python build configuration yield runtime tracebacks for users.

Also majority of users do not have access to an alternative build of python matching the openssl runtime configuration. Most of the time openssl-fips is not a separate openssl build, but a runtime configuration & plugin only. Thus existing python is expected to work correctly with it.

[gpshead]

Most of the time openssl-fips is not a separate openssl build, but a runtime configuration & plugin only. Thus existing python is expected to work correctly with it.

There's a disconnect in expectations here. CPython doesn't claim to work with OpenSSL FIPS mode itself. This is something some that re-packagers and distributors (like RedHat) have tried to make work themselves, for their own packages. The important gist of the discuss thread is that this configuration isn't supportable by CPython OSS contributors as is because there has been little interest in maintaining support for yet another unusual configuration, no CI or regression testing for it, and the specific goals are are not well defined.

I recommend using that discuss thread for further conversation.