jaeger agent stop working

[YMingZeng]

using this command to start the agent:
./jaeger-agent --reporter.grpc.host-port=xxxx --processor.jaeger-binary.server-queue-size=2000 --processor.jaeger-compact.server-queue-size=2000

when I start the agent in tchannel way , it's ok
./jaeger-agent --collector.host-port=XXX:14267 --processor.jaeger-binary.server-queue-size=2000 --processor.jaeger-compact.server-queue-size=2000

the version of jaeger-agent is v1.13.0

I know it's old ,but for some reason, I can't update it.
I don't know whether the latest version of jaeger-agent has the some problem.
Is it a bug ?

[dosubot[bot]]

Hey @YMingZeng! I'm here to help you with any bugs, questions, or contributions you have regarding this repository. Let's tackle this together!

In Jaeger Agent v1.13.0, the gRPC protocol is the recommended and stable protocol for communication between Agents and Collectors, while the TChannel protocol is deprecated and will be removed in future versions ^[1].

Given that the Jaeger Agent works with TChannel but not with gRPC, here are a few steps to diagnose and potentially resolve the issue:

1. Check Logs: Increase the logging level to debug to get more detailed information about what might be going wrong. You can do this by adding the --log-level=debug flag when starting the agent. Look for any errors or warnings related to gRPC communication.

2. Metrics: Use the /metrics endpoint to check for any anomalies or errors in the metrics related to gRPC communication. Key metrics to look at include:

   • jaeger_agent_reporter_batches_failures{format="jaeger",protocol="grpc"}
   • jaeger_agent_reporter_spans_failures{format="jaeger",protocol="grpc"}

3. Configuration Options: Ensure that the --reporter.grpc.host-port is correctly configured and reachable from the agent. Also, verify other gRPC-related configuration options such as reporter.grpc.retry.max and reporter.grpc.discovery.min-peers ^[2].

4. Queue Size: The default queue size for --processor.jaeger-binary.server-queue-size and --processor.jaeger-compact.server-queue-size is 1000. You have set it to 2000, which should be fine, but ensure that the system has enough resources to handle this increased queue size ^[3].

5. Known Issues: There are no known issues or bugs related to the gRPC reporter in Jaeger Agent v1.13.0 documented in the release notes or issue tracker ^[4].

6. Tenant Header: Ensure that the gRPC request includes the necessary tenant header if your setup requires it. Missing tenant headers can cause permission denied errors ^[5].

By following these steps, you should be able to gather more information about why the gRPC reporter is not functioning as expected and potentially resolve the issue.

To continue talking to Dosu, mention @dosu.