Python Auto-instrumentation: Conflict dependencies versions

[ChouMeriam21]

Hello!

I encountered an issue while using the OpenTelemetry auto-instrumentation to instrument a Python application that relies on urllib3 library version 1.26.14. According to the documentation, this version is supposed to be supported by the OpenTelemetry operator (ghcr.io/open-telemetry/opentelemetry-operator/autoinstrumentation-python:0.39b0), which I used as the image for the Instrumentation resource in Kubernetes.

Unfortunately, during deployment, I encountered a conflict error that prevented the creation of my pod.

[srikanthccv]

Could you share the complete error/stack trace?

[ChouMeriam21]

Yes of course, here's the complete error trace:

[ChouMeriam21]

I had to change the versions of the following packages to make the auto instrumentation works :

• urllib3 (from 1.26.14 to 2.0.2)
• requests (from 2.31.0 to 2.82.2)
• charset-normaliser (from 2.1.1 to 3.1.0)
• packaging ( from 23.1 to 23.0)

Is there any other way ? Especially that there are some packages as requests and packaging that had to endure a downgrade which is not the desired behaviour.

[lusains]

autoinstrumentation-python:0.40b0，How to fix it~

[sy-be]

I'd like to add some more context here.

When an init container for autoinstrumentation runs, it installs all of the packages and their dependencies listed in the requirements.txt file. In case with urllib3 it installs the latest available and supported version (which atm is 2.1.0). However if the application code specifies a different version in the corresponding Python environment (e.g. 1.26.14), this unfortunately breaks the application as PYTHONPATH is set to:

PYTHONPATH="/otel-auto-instrumentation-python/opentelemetry/instrumentation/auto_instrumentation:/otel-auto-instrumentation-python"

which overrides the import path for the urllib3. This in turn breaks other dependencies which rely on functionality of urllib3 1.26.14 which is a different major version.

A simple solution would be to somehow prepend the PYTHONPATH which whichever environment is used by the application. Another, possibly a better way, would be to prevent installation of packages inside /otel-auto-instrumentation-python - only install the instrumentation libraries without any other dependencies.

[sy-be]

This is somewhat similar to #1884, however in my case, I do not set PYTHONPATH anywhere and when this env var is set - it overrides the path with opentelemetry-autoinstrumentation and the libraries there.

[kguelzau]

So the operator changes the PYTHONPATH and dependencies of the instrumentation get precedence over dependencies of the instrumented app, right? (same problem as in #2302?)
In our case we updated a dependency (weaviate) in our app which has a (transitive) dependency to a newer version of protobuf as used in instrumentation (0.48b0).
The app did not start anymore and the developers who should not care about the instrumentation had a hard time to figure out why.
I think the dependencies of instrumentaion and the app must not be mixed, but should be completely seperate.
With seperate I mean something like relocating the instrumentation dependencies like in java (https://gradleup.com/shadow/configuration/relocation/), since no dependency resolving mechanism has a chance to do it's job here.

It's a NO-GO that autoinstrumentation is preventing app startup and so we will add instrumentation directly to the app.
This way the developers are aware of dependency conflicts and have a chance to resolve them.