Skip to content

GLIBC_2.27 not found on Xenial GPU Docker image #724

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
d00astro opened this issue May 7, 2021 · 2 comments
Open

GLIBC_2.27 not found on Xenial GPU Docker image #724

d00astro opened this issue May 7, 2021 · 2 comments
Assignees

Comments

@d00astro
Copy link

d00astro commented May 7, 2021

Describe the bug
"ERR could not load backend" when using the Docker image redislabs/redisai:latest-gpu-x64-xenial, and trying to load a TF model.
The image redislabs/redisai:1.0.3-gpu-x64-xenial has this issue too.

It seems to be specific to these images, as neither redislabs/redisai:latest-cpu-x64-xenial (Xenial CPU) nor redislabs/redisai:latest-gpu-x64-bionic (Bionic GPU) have this issue.

To Reproduce
Steps to reproduce the behavior:

  1. Download minimal RedisAI multiplication example TF model : graph.pb
  2. docker run -d --rm --gpus all --name redisai -p 6379:6379 redislabs/redisai:latest-gpu-x64-xenial
  3. cat graph.pb | redis-cli -x AI.MODELSET mymodel TF CPU INPUTS a b OUTPUTS c BLOB
    (error) ERR Could not load backend
  4. docker logs redisai
1:C 07 May 2021 00:20:44.039 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
1:C 07 May 2021 00:20:44.039 # Redis version=6.0.5, bits=64, commit=00000000, modified=0, pid=1, just started
1:C 07 May 2021 00:20:44.039 # Configuration loaded
1:M 07 May 2021 00:20:44.040 * Running mode=standalone, port=6379.
1:M 07 May 2021 00:20:44.040 # Server initialized
1:M 07 May 2021 00:20:44.040 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
1:M 07 May 2021 00:20:44.040 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
1:M 07 May 2021 00:20:44.040 * <ai> Redis version found by RedisAI: 6.0.5 - oss
1:M 07 May 2021 00:20:44.040 * <ai> RedisAI version 10003, git_sha=7f808a934dff121e188cb76fdfcc3eb1f9ec7cbf
1:M 07 May 2021 00:20:44.041 * Module 'ai' loaded from /usr/lib/redis/modules/redisai.so
1:M 07 May 2021 00:20:44.041 * Ready to accept connections
1:M 07 May 2021 00:20:49.613 # <ai> backend TF not loaded, will try loading default backend
1:M 07 May 2021 00:20:49.619 # <ai> Could not load TF backend from /usr/lib/redis/modules/backends/redisai_tensorflow/redisai_tensorflow.so: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.27' not found (required by /usr/lib/redis/modules/backends/redisai_tensorflow/lib/libtensorflow.so.1)

Expected behavior
Response OK, and:

redis-cli AI.MODELGET mymodel META
 1) "backend"
 2) "TF"
 3) "device"
 4) "CPU"
 5) "tag"
 6) ""
 7) "batchsize"
 8) (integer) 0
 9) "minbatchsize"
10) (integer) 0
11) "inputs"
12) 1) "a"
    2) "b"
13) "outputs"
14) 1) "c"

Screenshots
N/A

Environment (please complete the following information):

  • OS: Host: Ubuntu 20.04 + Docker image: 16.04
  • Version: 1.0.3 (?)
  • Platfrom: x86
  • Runtime: GPU (CC 6.1) / Cuda 10.1 / NCCL 2.8.3 / CuDNN 7.6.5

Additional context
Not critical for me as I can use Bionic just as well

@DvirDukhan
Copy link
Collaborator

@d00astro thanks for reporting!
we will check it out

@chayim
Copy link
Contributor

chayim commented May 9, 2021

@d00astro Thanks for reporting this. At first glance it looks like it could be the result of a dependency. Running strings against the shared object, I get the list below (note the lack of 2.2.27). I checked our shared objects, and we're similarly missing this version of glibc.

GLIBC_2.6
GLIBC_2.15
GLIBC_2.11
GLIBCXX_3.4.19
GLIBC_2.3.4
GLIBCXX_3.4.17
GLIBCXX_3.4.15
GLIBCXX_3.4.18
GLIBCXX_3.4
GLIBC_2.2.5
GLIBC_2.7
GLIBCXX_3.4.11
GLIBC_2.16
GLIBC_2.14
GLIBCXX_3.4.9
GLIBC_2.3.2
GLIBC_2.3
GLIBC_2.4
GLIBCXX_3.4.14
GLIBC_2.3.3
GLIBC_2.9
GLIBC_2.10
GLIBC_2.17
GLIBC_2.12

I'll check upstream in the future. For now, please keep using the docker you're currently content with.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants