Improve Integrations promotion, docs, and guidance

[cartersocha]

As OpenTelemetry continues to gain traction we need to ensure that services and platforms are able to easily integrate with OpenTelemetry, have straightforward guidance, and that the integrations are properly spotlighted. Given that many services are already choosing to adopt OpenTelemetry on their own we have a great opportunity to supercharge this movement.

The current integrations page is missing at least 4 current or upcoming substantial 1P integrations.

• Kong API Gateway Plugin
• VMWare PCF Default Collector
• Tyk API Gateway
• Nginx Web Server

There has also been general user frustration with the lack of documentation around adding OpenTelemetry into a library you own.

If we overhaul our integration experience to better serve 1P integrations, we can accelerate project adoption while reducing our need to develop and maintain bespoke components. With an eye towards the upcoming KubeCon (November 6th) I propose we:

• Produce clear guidance on the OTel site for an integration
• Spotlight the integrations page like we do on the site landing page or just increase visibility
• Co-write and co-promote blogs with vendors are they release integrations with initial releases at this KubeCon
• Potentially offer an entry point for library owners to reach out to the community to declare interest in integrating and state a desire for a little expert help

[svrnm]

Thanks for raising this @cartersocha

A few comments:

The current integrations page is missing at least 4 current or upcoming substantial 1P integrations.

Right now the page for integrations only holds oss integrations, so as it looks like only the recently open source nginx module would meet that criteria. I think we should open that up. Note that the whole vendor, integrations, adopters is still in flux and needs a major rework. I have an idea in my head how to make things better but I need some time to write this down and maybe make a proposal via a PR. Although, if nobody objects I think we can already add those 1P integrations to the current page.

Produce clear guidance on the OTel site for an integration.

There is guidance for that already:

https://opentelemetry.io/docs/concepts/instrumentation/libraries

We also started to improve the manual instrumentation guidelines which are crucial for that, see

• https://opentelemetry.io/docs/instrumentation/java/manual/
• https://opentelemetry.io/docs/instrumentation/js/manual/

We also have an explicit pointer via the "Devs" getting started:

https://opentelemetry.io/docs/getting-started/dev/

So the focus should be on the "clear" part, how can we make this more visible to people that want to do integrations? There is probably some cross linking we can do, to improve that. But key is to have per-language documentation for manual instrumentation improved as this is the crucial part for people wanting to do the actual work.

Spotlight the integrations page like we do on the site landing page or just increase visibility

I am happy to adopt ideas for that.

Co-write and co-promote blogs with vendors are they release integrations with initial releases at this KubeCon

Happy to take those blog posts in, although we still need to be cautious around commercial offerings. We have been very reluctant to accept anything that's non OSS for our blog.

Potentially offer an entry point for library owners to reach out to the community to declare interest in integrating and state a desire for a little expert help

We can create such an entrypoint, but at the end this is something that the language SIGs have to carry, so they need to provide the bandwidth to help with that.

[cartersocha]

Thanks for the detailed comments and context.

I do strongly believe we need to re-visit our open source / vendor split for integrations / promotion. I understand our default inclination to OSS but as OTel eats the world we'll be handicapping the project significantly by insisting on OSS only. If IBM creates a first party non-OSS integration for mainframes that's a significant milestone for them and the community that should be spotlighted / celebrated

[svrnm]

I do strongly believe we need to re-visit our open source / vendor split for integrations / promotion. I understand our default inclination to OSS but as OTel eats the world we'll be handicapping the project significantly by insisting on OSS only. If IBM creates a first party non-OSS integration for mainframes that's a significant milestone for them and the community that should be spotlighted / celebrated

I do not disagree, the way how it is today is just how things grew over time, of course we can change that. My bigger concern is that the way how we present the different adopters is sub-optimal, since what we have as adopters right now are "end users" (which is again confusing as a term), integrations is only some OSS projects that use OpenTelemetry and a vendor is an endless list of companies that may or may not provide an observability backend that is OTLP-compliant.

So the first step is re-thinking that representation of the OpenTelemetry ecosystem and create a more meaningful structure, for example:

• List entities that use OpenTelemetry as "end users", i.e. they do observability with it for their own purpose
• List entities that have integrated OpenTelemetry into their offering to provide observability for consumers of their offering (OSS projects, libraries etc...)
• List entities that provide offerings that consume OpenTelemetry data, do something with it and expose that data again (observability pipelines, etc.)
• List entitiies that provide data sinks / backends for OpenTelemetry data.
• (Consulting??)

Each one could be split in sub-categories again.

[AlexanderWert]

... we need to ensure that services and platforms are able to easily integrate with OpenTelemetry, have straightforward guidance ...

See a related comment: open-telemetry/opentelemetry-java-instrumentation#9337 (comment)

We are currently working on a how-to guide for native instrumentation (including all the best practices, things to consider and learnings we gained from instrumenting the Elasticsearch clients natively) and were planning to publish that either as an OTel blog post or as part of the OTel docs.

I think that would also help with the topic raised here. @cartersocha @svrnm WDYT about that?

... so as it looks like only the recently open source nginx module would meet that criteria. I think we should open that up

+1 on extending the integrations to non-open-source projects as well. Since the community would benefit from integrations in those projects as well.

cc @estolfo

[svrnm]

... we need to ensure that services and platforms are able to easily integrate with OpenTelemetry, have straightforward guidance ...

See a related comment: open-telemetry/opentelemetry-java-instrumentation#9337 (comment)

We are currently working on a how-to guide for native instrumentation (including all the best practices, things to consider and learnings we gained from instrumenting the Elasticsearch clients natively) and were planning to publish that either as an OTel blog post or as part of the OTel docs.

That would be great! We always prefer docs over blog posts, but starting with a blog post makes it easier to promote the topic and we still can work from there to turn it into docs. But we can make that decision when the content is ready :-)

[cartersocha]

Sounds great @AlexanderWert!

I spoke with Morgan from the GC who followed up with communications. We should be good to add non-OSS integrations and vendor integration blogs to the website with the normal community caveats

I’m planning on working with Nginx, Tyk, and VMWare to open blog PRs or other content additions

[svrnm]

@cartersocha thanks! Can you share some links to the integrations for Nginx, Tyk & VMware already?

Besides OSS vs non-OSS there is also something we should discuss if we want to change (see #3182 as well): right now we focus on "native integrations", aka opentelemtry is built-in to the library, application. Cerbos but maybe also nginx are good examples for first-part but non-native integrations since you need to load an additional module (=instrumentation library) to make them work. I think we should have a preference for native still and highlight them, but it also adds value if a first-party integration is accomplished by the party themselves.

[svrnm]

I’m planning on working with Nginx, Tyk, and VMWare to open blog PRs or other content additions

Also, we (Cisco) provide some first-party integrations with OpenTelemetry as well :-)

[cartersocha]

@svrnm I provided most of the links above already. All 3 are about to be released but aren't yet. Unfortunately Nginx and VMWare don't have an easy demo. But Tyk has this really nice docker up example - https://github.com/SonjaChevre/api-observability-opentelemetry

S/o @SonjaChevre for that 😎

[cartersocha]

I can add the relevant company contacts if necessary here too. Mostly coordinating on CNCF slack

[svrnm]

@svrnm I provided most of the links above already. All 3 are about to be released but aren't yet.

lol ... I totally missed that. Thanks.

[svrnm]

I can add the relevant company contacts if necessary here too. Mostly coordinating on CNCF slack

If there is a slack channel already where you discuss this please share the link here so folks can join in.

I think what we need is a clear plan on what we want to do exactly to promote those integrations: having them in the "Integrations" page is step 1 and easy to implement since we only need to update our list. If we want to do more (blog posts etc) we should figure out details first.

[cartersocha]

Is there a preferred community channel I can add them to? #otel-comms?

[svrnm]

#otel-comms should work just fine, yes

[svrnm]

please take a look at #3221

Note that I didn't include Tyk & VMWare, because both links indicate that OpenTelemetry is planned but not yet existing. Maybe that's a misunderstanding from my site, so please let me know.

I restructured the document in a why that it

• allows "non-OSS"
• allows "non-native, but first party" integrations (this addresses A JS library with native OpenTelemetry integration #3182, but also makes it easier to have nginx or kong which explicitly talk about a plugin that needs to be loaded)

[SonjaChevre]

Hi! I’m a product manager from Tyk. We released yesterday the latest version of our open source API Gateway (5.2) which includes native support for OpenTelemetry distributed tracing.

Doc link: https://tyk.io/docs/product-stack/tyk-gateway/advanced-configurations/distributed-tracing/open-telemetry/open-telemetry-overview/

happy to be included into the list of integration. Let me know if you have any questions!

[svrnm]

Thanks @SonjaChevre, I updated the PR to include Tyk API Gateway as well.

Since I skimmed over your docs, a minor comment on the "Common HTTP Span Attributes": the ones listed in your documentation are not up-to-date with some of the most recent spec changes (http.method => http.request.method, http.status_code => http.response.status_code, http.schema => url.scheme http.url => url.full)

[SonjaChevre]

Thanks @svrnm, really appreciate this feedback! I will pass this info to the team.

[svrnm]

I think we accomplished some/most of the topics. there is more work that needs to be done, but we should put that into new issues.