fix(weave): Swallow GeneratorExit exceptions when consumer abandons the generator

[chance-wnb]

suppress GeneratorExit errors in DataDog for streaming generator methods

Problem

Every call to the /call/read endpoint (and any other endpoint that partially consumes a streaming generator) was producing error spans in DataDog with error.type: builtins.GeneratorExit, despite the HTTP response being a
successful 200. This generates unnecessary noises.

Exampes:
DD Trace for POST /traces/call/read

Root cause

ClickHouseTraceServer.call_read fetches only the first row from calls_query_stream and then explicitly closes the generator in its finally
block:

# clickhouse_trace_server_batched.py
try:
    call = next(res)
except StopIteration:
    call = None
finally:
    res.close()  # ← always fires GeneratorExit into calls_query_stream

res.close() injects GeneratorExit into the partially-consumed generator chain. This is the correct and expected Python pattern for releasing resources — the generator was intentionally closed early, not due to any error.

However, two layers of DataDog tracing were incorrectly treating GeneratorExit as an application error:

Layer 1 — TracingTraceServer (tracing_trace_server.py)

def calls_query_stream(self, req):
    with tracer.trace("calls_query_stream"):
        yield from super().calls_query_stream(req)  # GeneratorExit propagates out
                                                    # → ddtrace marks span as error

Layer 2 — @ddtrace.tracer.wrap on generator methods (clickhouse_trace_server_batched.py)

ddtrace.tracer.wrap internally uses yield from to wrap generator functions, so GeneratorExit propagates through its context manager and is recorded as an error on the calls_query_stream and _query_stream spans.

This produced three error spans in DataDog on every single /call/read request:

calls_query_stream              ← tracing_trace_server.py
  clickhouse.calls_query_stream ← @ddtrace.tracer.wrap
    clickhouse._query_stream    ← @ddtrace.tracer.wrap

Fix

1. tracing_trace_server.py — catch GeneratorExit in all streaming methods

Added except GeneratorExit: pass inside the with tracer.trace(...) block for all generator methods. GeneratorExit is a BaseException, not an Exception, so it bypasses normal error handling — this is the minimal targeted change.

Applies to: calls_query_stream, completions_create_stream, annotation_queues_query_stream.

2. datadog.py — new generator_trace decorator

Introduced generator_trace(span_name) as a drop-in replacement for @ddtrace.tracer.wrap on generator functions. It creates a span that covers the full iteration lifetime and catches GeneratorExit cleanly.

3. clickhouse_trace_server_batched.py — replace @ddtrace.tracer.wrap on generators

Replaced @ddtrace.tracer.wrap with @generator_trace on calls_query_stream
and _query_stream.

Notice that catching these error at the HTTP endpoint root isn't enough: we don't want to see unexpected errors on any DD spans.

Verification

Without the fix, I got this DD trace:
https://us5.datadoghq.com/apm/trace/69bc788b000000004ed551844f71907b?graphType=waterfall&shouldShowLegend=true&spanID=9924611229400349424&timeHint=1773959307425.397&trace=AwAAAZ0INuMVPCpShwAAABhBWjBJTnZOWEFBQmVQZTN3c1ZFZlpkVk8AAAAkZjE5ZDA4MzctMmU3NS00OTkzLTk3MmMtOWZmZmU3ZjkxNzE0AAAAJg&traceQuery=

With the fix, I got this:
https://us5.datadoghq.com/apm/trace/69bc7d4c0000000057bf66bf13c0e7f1?spanID=7177803520423907467&timeHint=1773960524370.98&trace=AwAAAZ0ISXTmvZOdpgAAABhBWjBJU1g3ZEFBQm5NNTJuTnYxNnpyOVgAAAAkZjE5ZDA4NGEtMmRjNS00YjNiLThmNDYtN2E5ZTA0NWE2NmY5AAAACQ

Is there any valid case that we should surface up GeneratorExit exception from calls_query_stream ?

No. GeneratorExit is by definition a signal from the consumer saying "stop, I'm done with you" — it is never an application error. It carries no information about what went wrong; it only means the generator was closed early.

[chance-wnb]

• fix(weave): Swallow GeneratorExit exceptions when consumer abandons the generator #6397 👈 (View in Graphite)
• master

This stack of pull requests is managed by Graphite. Learn more about stacking.

[wandbot-3000]

Preview this PR with FeatureBee: https://beta.wandb.ai/?betaVersion=79540aa9fb2eef13c9717e6b9b88ab4745979bda

[gtarpenning]

amazing, that was annoying me

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!