Description
Is your feature request related to a problem? Please describe.
Hello team,
First of all, as this is my first post here, I would like to say thanks for the amazing work on the Java agent.
If not anything else, just thank you for this awesome project.
With that said, I would like to reach out with an enhancement request.
As of today, with Spark 4 and Java agent2, it seems it is not possible to have traces for Spark structured streaming Kafka consumer tracing.
By that I mean: suppose a Kafka consumer, written in Go, Python, Java, or else, which puts in the Kafka header a valid W3C trace ID.
Using Spark and the option to include headers, the Spark streaming service is able to consume the message and even see the valid trace.
However, even with the latest Java agent version 2, there is no way to have the traces propagated.
Describe the solution you'd like
As of today, the only solution possible is for the user to perform manual work on the Spark side to continue and propagate the trace.
Instead of having to ask each and every Spark structured streaming Kafka consumer job in the world to perform its own implementation, it would be great if the Java agent could help with the tracing.
Describe alternatives you've considered
No response
Additional context
No response