ModuleNotFoundError: No module named '_cffi_backend'

[firdouseM]

mod_wsgi (pid=2135587): Failed to exec Python script file '/var/www/html/atc/atc/wsgi.py'.
mod_wsgi (pid=2135587): Exception occurred processing WSGI script '/var/www/html/atc/atc/wsgi.py'.
Traceback (most recent call last):
File "/var/www/html/atc/atc/wsgi.py", line 29, in
application = get_wsgi_application()
File "/var/www/html/venv_atc/lib/python3.7/site-packages/django/core/wsgi.py", line 13, in get_wsgi_application
return WSGIHandler()
File "/var/www/html/venv_atc/lib/python3.7/site-packages/django/core/handlers/wsgi.py", line 127, in init
self.load_middleware()
File "/var/www/html/venv_atc/lib/python3.7/site-packages/django/core/handlers/base.py", line 35, in load_middleware
middleware = import_string(middleware_path)
File "/var/www/html/venv_atc/lib/python3.7/site-packages/django/utils/module_loading.py", line 17, in import_string
module = import_module(module_path)
File "/usr/lib64/python3.6/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 994, in _gcd_import
File "", line 971, in _find_and_load
File "", line 955, in _find_and_load_unlocked
File "", line 665, in _load_unlocked
File "", line 678, in exec_module
File "", line 219, in _call_with_frames_removed
File "/var/www/html/venv_atc/lib/python3.7/site-packages/mozilla_django_oidc/middleware.py", line 12, in
from mozilla_django_oidc.auth import OIDCAuthenticationBackend
File "/var/www/html/venv_atc/lib/python3.7/site-packages/mozilla_django_oidc/auth.py", line 16, in
from josepy.b64 import b64decode
File "/var/www/html/venv_atc/lib/python3.7/site-packages/josepy/init.py", line 41, in
from josepy.interfaces import JSONDeSerializable
File "/var/www/html/venv_atc/lib/python3.7/site-packages/josepy/interfaces.py", line 6, in
from josepy import errors, util
File "/var/www/html/venv_atc/lib/python3.7/site-packages/josepy/util.py", line 4, in
import OpenSSL
File "/var/www/html/venv_atc/lib/python3.7/site-packages/OpenSSL/init.py", line 8, in
from OpenSSL import crypto, SSL
File "/var/www/html/venv_atc/lib/python3.7/site-packages/OpenSSL/crypto.py", line 17, in
from OpenSSL._util import (
File "/var/www/html/venv_atc/lib/python3.7/site-packages/OpenSSL/_util.py", line 6, in
from cryptography.hazmat.bindings.openssl.binding import Binding
File "/var/www/html/venv_atc/lib/python3.7/site-packages/cryptography/hazmat/bindings/openssl/binding.py", line 14, in
from cryptography.hazmat.bindings._openssl import ffi, lib
ModuleNotFoundError: No module named '_cffi_backend'

appdirs==1.4.4
asgiref==3.3.4
attrs==21.1.0
autoit==0.2.3
cached-property==1.5.2
certifi==2020.12.5
cffi==1.15.1
chardet==4.0.0
click==7.1.2
cryptography==3.4.7
dataclasses==0.6
defusedxml==0.7.1
distlib==0.3.9
Django==3.0.5
django-multiselectfield==0.1.12
djongo==1.3.4
elastic-transport==8.4.1
elasticsearch==7.12.0
filelock==3.12.2
greenlet==1.0.0
gunicorn==23.0.0
idna==2.10
importlib-metadata==6.6.0
isodate==0.6.0
joblib==1.0.1
josepy==1.8.0
lxml==4.6.3
mod-wsgi==4.9.4
mozilla-django-oidc==1.2.4
multi-key-dict==2.0.3
mysql-connector-python==8.0.23
nltk==3.5
numpy==1.20.2
packaging==24.0
pandas==1.2.3
pbr==5.5.1
Pillow==8.1.2
pipdeptree==2.9.6
platformdirs==4.0.0
protobuf==3.15.8
psutil==5.8.0
pyasn1==0.5.1
PyAutoIt==0.6.5
pycparser==2.21
pymongo==3.12.3
pymsteams==0.1.14
pyOpenSSL==20.0.1
python-dateutil==2.8.1
python-jenkins==1.7.0
pytz==2021.1
rauth==0.7.3
regex==2021.4.4
requests==2.25.1
requests-file==1.5.1
requests-toolbelt==0.9.1
selenium==3.141.0
six==1.16.0
SQLAlchemy==1.4.4
sqlparse==0.2.4
tqdm==4.60.0
typing_extensions==4.7.1
urllib3==1.26.2
virtualenv==20.26.6
zeep==4.0.0
zipp==3.4.1

Python 3.7 is the current one with RHEL 8 server.

[arigo]

Search for _cffi_backend*.so in /var/www/html/venv_atc/lib/python3.7/site-packages/. If it is not there, then your installation is broken and cffi==1.15.1 is not actually installed.

[Firdouse532]

(venv_atc) [root@ncerndatclnx01 ~]# ls -al /var/www/html/venv_atc/lib/python3.7/site-packages/ | grep -i _cffi_backend*
-rwxr-xr-x 1 apache apache 932624 Nov 6 06:48 _cffi_backend.cpython-37m-x86_64-linux-gnu.so

It is present

[arigo]

Sorry, I don't know. Maybe ask on other platforms like StackOverflow? It doesn't make much sense that most modules inside this directory can be imported (at least all the ones shown in the traceback) but this precise one cannot. Maybe try to do import _cffi_backend explicitly and see if that succeeds, or if not, if the error message is more detailed.

[mattip]

Are you actually running on a x86_64 instance? For instance what does this show?

python3.7 -c "import _ssl; print(_ssl)"

[Firdouse532]

(venv_atc) [root@ncerndatclnx01 html]# python3 -c "import _ssl; print(_ssl)"
<module '_ssl' from '/usr/local/lib/python3.7/lib-dynload/_ssl.cpython-37m-x86_64-linux-gnu.so'>
(venv_atc) [root@ncerndatclnx01 html]# venv_atc/bin/python -c "import _ssl; print(_ssl)"
<module '_ssl' from '/usr/local/lib/python3.7/lib-dynload/_ssl.cpython-37m-x86_64-linux-gnu.so'>

[mattip]

OK, that is promising, at least your CPython can import c-extension modules. If `python3 -c "import _cffi_backend" fails, does it print out any more information?

[Firdouse532]

(venv_atc) [root@ncerndatclnx01 html]# python3 -c "import _cffi_backend"
(venv_atc) [root@ncerndatclnx01 html]#

No error occurred!

[arigo]

@mattip Note that the paths printed by python3 -c "import _ssl; print(_ssl)" are different than the paths in the reported traceback. I still think that cffi is not actually correctly installed in the virtualenv or whatever is set up for mod_wsgi.

@Firdouse532 How did you get the list of installed modules you posted in the first message?

[Firdouse532]

I have performed pip freeze within the venv created for this project.

[mattip]

No error occurred!

That would indicate you are not using the same python in these checks and in mod_wsgi.

Can you try the steps I mentioned above again with the python at /var/www/html/venv_atc?

[Firdouse532]

(venv_atc) [root@ncerndatclnx01 html]# /var/www/html/venv_atc/bin/python -c "import _cffi_backend"
(venv_atc) [root@ncerndatclnx01 html]# /var/www/html/venv_atc/bin/python -c "import _ssl; print(_ssl)"
<module '_ssl' from '/usr/local/lib/python3.7/lib-dynload/_ssl.cpython-37m-x86_64-linux-gnu.so'>

(venv_atc) [root@ncerndatclnx01 html]# /var/www/html/venv_atc/bin/python3 -c "import _ssl; print(_ssl)"
<module '_ssl' from '/usr/local/lib/python3.7/lib-dynload/_ssl.cpython-37m-x86_64-linux-gnu.so'>
(venv_atc) [root@ncerndatclnx01 html]# /var/www/html/venv_atc/bin/python3 -c "import _cffi_backend"
(venv_atc) [root@ncerndatclnx01 html]# /var/www/html/venv_atc/bin/python3 -c "import _cffi_backend; print(_cffi_backend)"
<module '_cffi_backend' from '/var/www/html/venv_atc/lib/python3.7/site-packages/_cffi_backend.cpython-37m-x86_64-linux-gnu.so'>

[arigo]

And what about: /var/www/html/venv_atc/bin/python -c "import OpenSSL"

[mattip]

@arigo do you have an OpenSSL in your python3? (sorry I misunderstood)

@Firdouse532 I am out of ideas. I still claim you are running a different python from mod_wsgi and not the one in /var/www/html/venv_atc/bin/python, and that this is not the right place to debug this issue: try on a django or (I assume) apache forum/issue tracker. Line 14 in your original error log supports that line of thought, it is execing with python3.6

File "/usr/lib64/python3.6/importlib/init.py", line 126, in import_module

I know nothing about how this is supposed to work. I would start by modifying the script /var/www/html/atc/atc/wsgi.py to give more information about the environment: something is off and it is not cffi.

[mattip]

Note as per your comment that the venv in /var/www/html/venv_atc successfully imports cffi_backend, which strongly suggests this is not a cffi problem.

[Firdouse532]

Sure, Maybe I will check with different forums of django or Apache. Thanks for your help @mattip