Support for LLM observalibility

[srinisubramanian]

Any ideas for providing LLM observalibility options from frameworks like LangChain?

A simple approach (given OpenTelemetry integration) could be to use openllmetry. More details on a SDK for itnegration is here: https://github.com/traceloop/openllmetry

[MikeShi42]

Definitely familiar with openllmetry - met with Nir a couple of times!

We should be supported already, though I'll want to test it out myself and submit a PR for their docs and ours if it works successfully. In theory you should be able to set them up and provide the following configs (assuming localhost, the base url would be https://in-otel.hyperdx.io for cloud):

TRACELOOP_BASE_URL=http://localhost:4318
TRACELOOP_HEADERS="authorization=<YOUR_HYPERDX_API_KEY_HERE>"

Any chance you already have openllmetry setup with HyperDX and want to give that a spin?

[srinisubramanian]

@MikeShi42 I havent tred this ,, but since you say it should work I will try and let you know for sure,

[MikeShi42]

@srinisubramanian Traceloop has published an integration page about us, we'll hopefully do the same on our end soon!

https://www.traceloop.com/docs/openllmetry/integrations/hyperdx