Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

examples/features/opentelemetry: demonstrate tracing using OpenTelemetry plugin #8056

Open
wants to merge 74 commits into
base: master
Choose a base branch
from

Conversation

janardhanvissa
Copy link
Contributor

@janardhanvissa janardhanvissa commented Jan 30, 2025

RELEASE NOTES:

  • examples/features/opentelemetry: modify example to demonstrate tracing using OpenTelemtry plugin.

Copy link

codecov bot commented Jan 30, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 82.13%. Comparing base (ce35fd4) to head (f5828d3).

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #8056      +/-   ##
==========================================
- Coverage   82.16%   82.13%   -0.03%     
==========================================
  Files         410      410              
  Lines       40248    40248              
==========================================
- Hits        33068    33059       -9     
- Misses       5830     5833       +3     
- Partials     1350     1356       +6     

see 23 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@purnesh42H purnesh42H changed the title opentelemetry: add open telemetry trace for client/server examples/features/opentelemetry: Add example to demonstrate OpenTelemetry Tracing Feb 19, 2025
Copy link
Contributor

@purnesh42H purnesh42H left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@janardhanvissa please use any of the trace exporter from here https://opentelemetry.io/docs/languages/go/exporters/. May be otlptracehttp or stdouttrace are good candidates

Also, please refer to https://opentelemetry.io/docs/languages/go/getting-started/ for writing otel trace exporting parts in client and server. We should try to avoid as much custom code as possible here to make it easy and fast for users to pick up.

@purnesh42H purnesh42H added this to the 1.72 Release milestone Mar 24, 2025
Copy link
Contributor

@purnesh42H purnesh42H left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@janardhanvissa i discussed with @dfawley and we are fine with using console exporter instead for the tracing example so that we don't have install extra stufff. Please modify the example using the console exporter

examples/go.mod Outdated
//
// For the OpenTelemetry Collector Core distribution specifically, see
// https://github.com/open-telemetry/opentelemetry-collector-releases/tree/main/distributions/otelcol

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: // Documentation:
// The builder manifests used for official OpenTelemetry Collector binaries
// can be found at the following repository:
// https://github.com/open-telemetry/opentelemetry-collector-releases
//
// For the OpenTelemetry Collector Core distribution specifically, refer to:
// https://github.com/open-telemetry/opentelemetry-collector-releases/tree/main/distributions/otelcol

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These docs are for the OpenTelemetry Collector, which we're not using; we're using stdouttrace instead. These things are already removed from the go.mod.

// the client and the server.
for {
r, err := c.UnaryEcho(ctx, &echo.EchoRequest{Message: "this is examples/opentelemetry"})
if err != nil {
log.Fatalf("UnaryEcho failed: %v", err)
}
fmt.Println(r)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: remove the extra space

@@ -51,14 +60,47 @@ func (s *echoServer) UnaryEcho(_ context.Context, req *pb.EchoRequest) (*pb.Echo
}

func main() {
flag.Parse()

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Remove the extra space and check the other places as well

OpenTelemetry is configured on both the client and the server, and exports to
the Prometheus exporter. The exporter exposes metrics on the Prometheus ports
described above.
OpenTelemetry is configured on both the client and the server, and exports metrics to the Prometheus exporter. The exporter exposes metrics on the Prometheus ports described above. OpenTelemetry also exports traces using the stdouttrace exporter. These traces are printed as structured data to the console output of both the client and the server.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
OpenTelemetry is configured on both the client and the server, and exports metrics to the Prometheus exporter. The exporter exposes metrics on the Prometheus ports described above. OpenTelemetry also exports traces using the stdouttrace exporter. These traces are printed as structured data to the console output of both the client and the server.
OpenTelemetry is configured on both the client and the server, and exports metrics to the Prometheus exporter. The exporter exposes metrics on the Prometheus ports described above.
OpenTelemetry exports traces using the stdouttrace exporter, which prints structured trace data to the console output of both the client and server. Each RPC call produces trace information that captures the execution flow and timing of operations.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Copy link
Contributor

@purnesh42H purnesh42H left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@janardhanvissa in README.md, please all sentences to 80 cols.

@@ -1,10 +1,13 @@
# OpenTelemetry

This example shows how to configure OpenTelemetry on a client and server, and shows
what type of telemetry data it can produce for certain RPC's.
This example shows how to configure OpenTelemetry on a client and server, and shows what type of telemetry data it can produce for certain RPC's.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please restrict the sentences to 80 cols. What is the change here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reached 80 cols, so moved to the next line.


## Try it

This section shows how to configure OpenTelemetry Metrics and Traces.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can remove this I think. The first sentence is enough

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done


## Try it

This section shows how to configure OpenTelemetry Metrics and Traces.

**1. Run the gRPC Applications:**
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same. No need of this

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@@ -13,20 +16,18 @@ go run server/main.go
go run client/main.go
```

**2. View Telemetry Data:**
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no need of this

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

The client continuously makes RPC's to a server. The client and server both
expose a prometheus exporter to listen and provide metrics. This defaults to
:9464 for the server and :9465 for the client.
The client continuously makes RPC's to a server. The client and server both expose a Prometheus exporter to listen and provide metrics. This defaults to :9464 for the server and :9465 for the client. The client and server are also configured to output traces directly to their standard output streams using stdouttrace.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wrap stdouttrace within backticks

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

)

func main() {
exporter, err := prometheus.New()
flag.Parse()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we need this flag.Parse()? or it can be removed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed

if err != nil {
log.Fatalf("Failed to start prometheus exporter: %v", err)
}
provider := metric.NewMeterProvider(metric.WithReader(exporter))
go http.ListenAndServe(*prometheusEndpoint, promhttp.Handler())
// Initialize MeterProvider with Prometheus exporter.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no need of this comment. Actually there is no change needed here. It can be reverted

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

)

var (
addr = flag.String("addr", ":50051", "the server address to connect to")
prometheusEndpoint = flag.String("prometheus_endpoint", ":9464", "the Prometheus exporter endpoint")
prometheusEndpoint = flag.String("prometheus_endpoint", ":9464", "the Prometheus exporter endpoint for metrics")
serviceName = "grpc-server"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same. Do we need this? or it can be removed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we need to set the service name as an attribute in the resource information attached to the traces for the server.

@@ -51,14 +57,28 @@ func (s *echoServer) UnaryEcho(_ context.Context, req *pb.EchoRequest) (*pb.Echo
}

func main() {
exporter, err := prometheus.New()
flag.Parse()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same. do we need this? or it can be removed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed

if err != nil {
log.Fatalf("Failed to start prometheus exporter: %v", err)
}
provider := metric.NewMeterProvider(metric.WithReader(exporter))
go http.ListenAndServe(*prometheusEndpoint, promhttp.Handler())
// Initialize MeterProvider with Prometheus exporter.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same. No need of any change here. Let it be how it was.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Copy link
Contributor

@purnesh42H purnesh42H left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

)

func main() {
exporter, err := prometheus.New()
if err != nil {
log.Fatalf("Failed to start prometheus exporter: %v", err)
}
provider := metric.NewMeterProvider(metric.WithReader(exporter))
go http.ListenAndServe(*prometheusEndpoint, promhttp.Handler())
provider := otelmetric.NewMeterProvider(otelmetric.WithReader(exporter))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: one comment above "Configure meter provider for metrics"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

if err != nil {
log.Fatalf("Failed to create stdouttrace exporter: %v", err)
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nix new line

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

log.Fatalf("Failed to create stdouttrace exporter: %v", err)
}

// Configure the tracer provider
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can remove this

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done


// Configure the tracer provider
tp := sdktrace.NewTracerProvider(sdktrace.WithBatcher(traceExporter), sdktrace.WithResource(otelresource.NewWithAttributes(semconv.SchemaURL, semconv.ServiceName(serviceName))))
textMapPropagator := otelpropagation.TraceContext{} // Using W3C Trace Context Propagator.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
textMapPropagator := otelpropagation.TraceContext{} // Using W3C Trace Context Propagator.
// Configure W3C Trace Context Propagator for traces
textMapPropagator := otelpropagation.TraceContext{} // Using W3C Trace Context Propagator.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done


ctx := context.Background()
do := opentelemetry.DialOption(opentelemetry.Options{MetricsOptions: opentelemetry.MetricsOptions{MeterProvider: provider}})
// Initialize stdouttrace exporter for traces
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

replace comment with "Configure exporter for traces"

Suggested change
// Initialize stdouttrace exporter for traces
// Configure exporter for traces

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@purnesh42H purnesh42H requested a review from dfawley April 3, 2025 16:14
@purnesh42H purnesh42H assigned dfawley and unassigned janardhanvissa Apr 3, 2025
@purnesh42H
Copy link
Contributor

@janardhanvissa could you add the screenshots of traces on clients and server please?

@janardhanvissa
Copy link
Contributor Author

@janardhanvissa could you add the screenshots of traces on clients and server please?

Server

Screenshot 2025-04-03 10 42 48 PM
Screenshot 2025-04-03 10 44 34 PM
Screenshot 2025-04-03 10 44 50 PM
Screenshot 2025-04-03 10 45 37 PM

Client

Screenshot 2025-04-03 10 45 55 PM
Screenshot 2025-04-03 10 47 42 PM
Screenshot 2025-04-03 10 47 53 PM
Screenshot 2025-04-03 10 48 07 PM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area: Observability Includes Stats, Tracing, Channelz, Healthz, Binlog, Reflection, Admin, GCP Observability Type: Documentation Documentation or examples
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants