Agent not reporting HTTP operations from spring app

[bradooles]

Expected behavior

Using the application agent, HTTP operations / transactions should appear in azure application insights

Actual behavior

HTTP Operations / transactions do not appear in application insights - other data does such as instance roles, utilisation, dependencies, trace logs etc so I don't believe it is a connection problem. Have tested with versions 3.4.16 and 3.4.18 and both have same issue. Interestingly in the debug logs I can see the http transactions being picked up by the agent (getTransaction and our health checks), they just don't appear in azure application insights.

I've attached some of the debug logs for the agent - let me know if there are specific logs you want us to try and find (or if you need TRACE logging)

System information

Please provide the following information:

• SDK Version: 3.4.18, 3.4.16
• OS type and version: Alpine Linux 3.18 on Kubernetes
• Using spring-boot? Yes - 2.7.14
• Additional relevant libraries (with version, if applicable): Java 17

Thanks for your time

agent_logs_debug.txt

[heyams]

@bradooles I noticed the following:

read environment variable: APPLICATIONINSIGHTS_CONFIGURATION_FILE=**************\
2023-11-29 17:09:13.723+11:00 DEBUG c.m.a.a.i.c.ConfigurationBuilder - configuration: \{"connectionString":"InstrumentationKey=********\'94,\'94selfDiagnostics":\{"destination":"console","level":"DEBUG"\}\}\

can you make sure the connectionString in the log matches what you have in your APPLICATIONINSIGHTS_CONFIGURATION_FILE file?
Logs implied that data did flow to Application Insights. I believe it's a matter of whether you're looking at the right Application Insights resource.

[bradooles]

Hi @heyams,

Thanks for responding.

I've double checked and we should be looking at the correct resource. I've attached some screens (with some masking) to show that the connection string is the same in the config file as the application insights we are viewing. I've also attached some screens which show our pods are indeed connecting and sending utilisation and dependency data.

There is an additional screen which shows the Operation screen which is empty - I can see the traffic in the logs when I send in a request (as well as regular health checks) but it is not reflected here.

[trask]

hi @bradooles, I don't see any "Request" telemetry being sent in the debug log you attached. Is your Spring App handling HTTP requests? Does it use Servlets, Netty, or something else for handling HTTP requests? Thanks.

[bradooles]

Hi @trask the service is setup to use servlet for handling HTTP requests - the 'getTransaction' in the logs is the HTTP GET request for the service. The rest controller is quite basic with just a single GET and POST mappings. You think it's not detecting the request events?
In the logs I could see it find a 'getTransaction' event, so is it a case of it not counting it towards request data?

Let me know if you need any other logging, I can run with TRACE if you need

[trask]

Let me know if you need any other logging, I can run with TRACE if you need

yes this would be helpful, thx

[bradooles]

TRACE logs attached.
I can see in the logs:
{"baseType":"MetricData","baseData":{"ver":2,"metrics":[{"name":"http_server_requests","value":1522.108629,"count":1,"max":1522.108629}],"properties":{"exception":"None","method":"GET","uri":"/getTransaction","outcome":"SUCCESS","status":"200"}}}}
which is the GET request.

agent_with_trace.txt

[trask]

thanks @bradooles, I'm not seeing anything unusual in the logs (except for the Request telemetry being missing). can you create a repro that we can use to troubleshoot this?