[Core] MD5 not supported as is for FIPS enabled machines

[KPostOffice]

Description

FIPS (Federal Information Processing Standards) is a series of security standards required for most government workflows. FIPS compliance requires packages to adhere to a certain level of security and comes with requirements for cryptographic libraries. One of my coworkers (@jbusche) was running into issues on a FIPS enabled machine when trying to submit jobs to a Ray Cluster. The key part of the linked issue is this:

File /opt/app-root/lib64/python3.8/site-packages/ray/_private/runtime_env/packaging.py:152, in _hash_directory.<locals>.handler(path)
    151 def handler(path: Path):
--> 152     md5 = hashlib.md5()
    153     md5.update(str(path.relative_to(relative_path)).encode())

ValueError: [digital envelope routines: EVP_DigestInit_ex] disabled for FIPS

MD5 is not supported for security related applications on a FIPS enabled machine

If the community is open to addressing this issue I see two options:

• change hashlib.md5(...) to hashlib.md5(..., usedforsecurity=False) because, everywhere in the Ray library that I've found md5 being used it is being used only for calculating file hashes and not for security purposes
• use a SHA-X for computing hashes since it is FIPS compliant

I'm happy to open a PR to solve this issue.

Use case

Using Ray in a FIPS enabled OpenShift cluster

[jjyao]

@KPostOffice Contribution is very welcome!