OTLP MetricExporter deadlock issue

[cijothomas]

https://github.com/open-telemetry/opentelemetry-rust/blob/main/opentelemetry-otlp/src/exporter/tonic/metrics.rs#L58

Tonic MetricExporter uses std::Mutex to protect inner client, and the lock is held across await points. This can cause deadlock. The fix would be to use tokio::Mutex, but then shutdown/cleanup cannot be easily done, like already faced by LogExporter. But that is better than having code that deadlocks.

Opening an issue to track fixing this.

[cijothomas]

Looks like same issue in HTTP also https://github.com/open-telemetry/opentelemetry-rust/blob/main/opentelemetry-otlp/src/exporter/http/mod.rs#L271

[fraillt]

I might be missing something, but current code doesn't hold the lock across the await point...
Given that inner is accessed in two places (export and shutdown), I pretty sure that code is correct and it's not possible to deadlock...
What do I miss here?

[cijothomas]

.and_then(|g| match &*g {
Some(client) => Ok(Arc::clone(client)),
_ => Err(OTelSdkError::AlreadyShutdown),
})?;

if the guard is dropped here itself, then yes, this looks correct. (I was incorrectly assuming the lock guard is not dropped until export() method ends..