Skip to content

JMX metrics not being logged #4218

Open
@s21-himesh

Description

@s21-himesh

Expected behavior

JMX metrics to be logged into application insights if specified in applicationinsights.json

Actual behavior

No logs captured for the JMX metrics specified.

Jmx Metrics

"jmxMetrics": [
    {
      "Name": "Menu Locks Active Locks",
      "ObjectName": "com.xxx.p2j:name=menu.locks",
      "Attribute": "ActiveLocks" 
    },
    {
      "Name": "Menu Locks Deletes From VST",
      "ObjectName": "com.xxx.p2j:name=menu.locks",
      "Attribute": "DeletesFromVST" 
    },
    {
      "Name": "Menu Locks Lock Events",
      "ObjectName": "com.xxx.p2j:name=menu.locks",
      "Attribute": "LockEvents" 
    },
    {
      "Name": "Menu Locks Persist Calls",
      "ObjectName": "com.xxx.p2j:name=menu.locks",
      "Attribute": "PersistCalls" 
    }
  ]

System information

Please provide the following information:

  • SDK Version: 3.7.2
  • OS type and version: Ubuntu 22.02
  • Application Server type and version (if applicable):
  • Using spring-boot?
  • Additional relevant libraries (with version, if applicable):

Details

Java Process

$ ps aux | grep java
fwd         1228  9.5 22.8 12629524 3729156 ?    Sl   11:09   5:16 java -XX:GCTimeRatio=19 -XX:MaxTenuringThreshold=7 -XX:StringTableSize=1000003 -Xmx8192m -server -Dcom.sun.management.jmxremote -Dc                                    om.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.port=22100 -Dcom.sun.management.jmxremote.rmi.port=22100 -XX:+PrintGCDetails -v                                    erbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:server_garbage_collection_stats.log -javaagent:/opt/counteract/lib/applicationinsights-agent.jar

Jstat confirming its monitoring

$ jstat -gc 1228
    S0C         S1C         S0U         S1U          EC           EU           OC           OU          MC         MU       CCSC      CCSU     YGC     YGCT     FGC    FGCT     CGC    CGCT       GCT                                        
    67072.0     67584.0         0.0     35103.4     222720.0      12281.4    2516480.0    2016039.3   121984.0   117666.9   14464.0   13423.7    227    18.764    18    41.705     -         -    60.4                                       69

Info on MBean

$>info
#mbean = com.xxx.p2j:name=menu.locks
#class name = com.goldencode.p2j.jmx.LockTableUpdaterStat
# attributes
  %0   - ActiveLocks (long, r)
  %1   - DeletesFromVST (long, r)
  %2   - LockEvents (long, r)
  %3   - PersistCalls (long, r)
  %4   - PersistedLocks (long, r)
# operations
  %0   - void flush()

Logs


2025-05-27 13:39:58.145+01:00 DEBUG c.m.a.a.i.c.ConfigurationBuilder - configuration: {"selfDiagnostics":{"level":"debug"},"preview":{"generalExportQueueCapacity":4096},"connectionString":"InstrumentationKey=************************;IngestionEndpoint=https://uksouth-1.in.applicationinsights.azure.com/;LiveEndpoint=https://uksouth.livediagnostics.monitor.azure.com/;ApplicationId=****************************************"}
2025-05-27 13:39:58.166+01:00 INFO  c.m.a.a.i.c.ConfigurationBuilder - Some telemetry may be sampled out because a default sampling configuration was added in version 3.4.0 to reduce the default billing cost. You can set the sampling configuration explicitly: https://learn.microsoft.com/azure/azure-monitor/app/java-standalone-config#sampling
2025-05-27 13:39:58.181+01:00 DEBUG c.m.applicationinsights.agent - Input arguments: [-XX:GCTimeRatio=19, -XX:MaxTenuringThreshold=7, -XX:StringTableSize=1000003, -Xmx8192m, -javaagent:/opt/counteract/lib/applicationinsights-agent.jar, -Djava.locale.providers=SPI,JRE, -Djava.ext.dirs=/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext:/usr/java/packages/lib/ext:../lib/spi]
2025-05-27 13:39:58.181+01:00 DEBUG c.m.applicationinsights.agent - _JAVA_OPTIONS: null
2025-05-27 13:39:58.181+01:00 DEBUG c.m.applicationinsights.agent - JAVA_TOOL_OPTIONS: null
2025-05-27 13:40:00.040+01:00 INFO  c.a.c.h.n.i.NettyUtility - {"az.sdk.message":"The following Netty versions were found on the classpath and have a mismatch with the versions used by azure-core-http-netty. If your application runs without issue this message can be ignored, otherwise please align the Netty versions used in your application. For more information, see https://aka.ms/azsdk/java/dependency/troubleshoot.","azure-netty-version":"4.1.115.Final","azure-netty-native-version":"2.0.69.Final","classpath-netty-version-io.netty:netty-common":"unknown (not found and is required)","classpath-netty-version-io.netty:netty-handler":"unknown (not found and is required)","classpath-netty-version-io.netty:netty-handler-proxy":"4.1.118.Final","classpath-netty-version-io.netty:netty-buffer":"unknown (not found and is required)","classpath-netty-version-io.netty:netty-codec":"unknown (not found and is required)","classpath-netty-version-io.netty:netty-codec-http":"unknown (not found and is required)","classpath-netty-version-io.netty:netty-codec-http2":"unknown (not found and is required)"}
2025-05-27 13:40:00.478+01:00 INFO  c.a.c.i.ReflectionUtilsMethodHandle - Unable to create MethodHandles to use Java 9+ MethodHandles.privateLookupIn. Will attempt to fallback to using the package-private constructor.
2025-05-27 13:40:00.526+01:00 DEBUG c.m.applicationinsights.agent - connection string is not null, start HeartbeatExporter
2025-05-27 13:40:00.720+01:00 WARN  i.o.s.a.ResourceConfiguration - Found reference to io.opentelemetry.sdk.autoconfigure.internal.EnvironmentResourceProvider in otel.java.enabled.resource.providers. Please update to io.opentelemetry.sdk.autoconfigure.EnvironmentResourceProvider. Support for the old provider name will be removed after 1.49.0.
2025-05-27 13:40:03.995+01:00 DEBUG n.b.agent.builder.AgentBuilderUtil - Could not decompose matcher failSafe(try(AnnotationInstrumentationModule#WithSpanInstrumentation((not(name(startsWith(kotlin.coroutines.))) and declaresMethods(whereOne((declaresAnnotations(whereOne(ofAnnotationType(name(equals(io.opentelemetry.instrumentation.annotations.WithSpan))))) and returns(erasure(is(class java.lang.Object))) and io.opentelemetry.javaagent.instrumentation.instrumentationannotations.KotlinCoroutineUtil$$Lambda$293/822359325@108531c2 and not(false))))))) or false)
2025-05-27 13:40:03.996+01:00 DEBUG n.b.agent.builder.AgentBuilderUtil - Could not decompose matcher failSafe(try(AnnotationInstrumentationModule#WithSpanInstrumentation(declaresMethods(whereOne(declaresAnnotations(whereOne(ofAnnotationType(name(equals(io.opentelemetry.instrumentation.annotations.WithSpan))))))))) or false)
2025-05-27 13:40:03.996+01:00 DEBUG n.b.agent.builder.AgentBuilderUtil - Could not decompose matcher failSafe(try(AnnotationInstrumentationModule#AddingSpanAttributesInstrumentation(declaresMethods(whereOne((declaresAnnotations(whereOne(ofAnnotationType(name(equals(io.opentelemetry.instrumentation.annotations.AddingSpanAttributes))))) and not(declaresAnnotations(whereOne(ofAnnotationType(name(equals(io.opentelemetry.instrumentation.annotations.WithSpan))))))))))) or false)
2025-05-27 13:40:03.998+01:00 DEBUG n.b.agent.builder.AgentBuilderUtil - Could not decompose matcher failSafe(try(WithSpanInstrumentationModule#WithSpanInstrumentation(declaresMethods(whereOne(declaresAnnotations(whereOne(ofAnnotationType(name(equals(io.opentelemetry.extension.annotations.WithSpan))))))))) or false)
2025-05-27 13:40:03.999+01:00 DEBUG n.b.agent.builder.AgentBuilderUtil - Could not decompose matcher failSafe(try(JaxrsInstrumentationModule#JaxrsAnnotationsInstrumentation(safeHasSuperType(safeErasure((declaresAnnotations(whereOne(ofAnnotationType(name(equals(javax.ws.rs.Path))))) or declaresMethods(whereOne(declaresAnnotations(whereOne(ofAnnotationType(name(equals(javax.ws.rs.Path)))))))))))) or false)
2025-05-27 13:40:03.999+01:00 DEBUG n.b.agent.builder.AgentBuilderUtil - Could not decompose matcher failSafe(try(JaxrsAnnotationsInstrumentationModule#JaxrsAnnotationsInstrumentation(safeHasSuperType(safeErasure((declaresAnnotations(whereOne(ofAnnotationType(name(equals(javax.ws.rs.Path))))) or declaresMethods(whereOne(declaresAnnotations(whereOne(ofAnnotationType(name(equals(javax.ws.rs.Path)))))))))))) or false)
2025-05-27 13:40:04.000+01:00 DEBUG n.b.agent.builder.AgentBuilderUtil - Could not decompose matcher failSafe(try(JaxrsAnnotationsInstrumentationModule#JaxrsAnnotationsInstrumentation(safeHasSuperType(safeErasure((declaresAnnotations(whereOne(ofAnnotationType(name(equals(jakarta.ws.rs.Path))))) or declaresMethods(whereOne(declaresAnnotations(whereOne(ofAnnotationType(name(equals(jakarta.ws.rs.Path)))))))))))) or false)
2025-05-27 13:40:04.000+01:00 DEBUG n.b.agent.builder.AgentBuilderUtil - Could not decompose matcher failSafe(try(JwsInstrumentationModule#JwsAnnotationsInstrumentation((safeHasSuperType(safeErasure(declaresAnnotations(whereOne(ofAnnotationType(name(equals(javax.jws.WebService))))))) or declaresAnnotations(whereOne(ofAnnotationType(name(equals(javax.jws.WebService)))))))) or false)
2025-05-27 13:40:05.676+01:00 DEBUG c.a.m.o.a.i.quickpulse.QuickPulse - Initializing QuickPulse with instrumentation key: **************, URL https://uksouth.livediagnostics.monitor.azure.com/, rolename null, role instance c-act, sdk version 3.7.1
2025-05-27 13:40:06.035+01:00 DEBUG c.a.m.o.a.i.q.f.FilteringConfiguration - Initializing an empty live metrics filtering configuration - did not yet receive a configuration from ping or post.
2025-05-27 13:40:06.141+01:00 DEBUG c.a.m.o.a.i.q.QuickPulseCoordinator - QuickPulseDataSender initialized with endpointUrl: https://uksouth.livediagnostics.monitor.azure.com/, instrumentationKey: ******************************
2025-05-27 13:40:06.149+01:00 DEBUG c.a.m.o.a.i.q.QuickPulsePingSender - About to ping quickpulse with the endpoint prefix: https://uksouth.livediagnostics.monitor.azure.com/
2025-05-27 13:40:22.034+01:00 INFO  c.m.applicationinsights.agent - Application Insights Java Agent 3.7.1 started successfully (PID 1219, JVM running for 41.218 s)
2025-05-27 13:40:22.034+01:00 INFO  c.m.applicationinsights.agent - Java version: 1.8.0_442, vendor: Private Build, home: /usr/lib/jvm/java-8-openjdk-amd64/jre
2025-05-27 13:40:28.670+01:00 DEBUG c.a.m.o.a.i.q.QuickPulsePingSender - About to ping quickpulse with the endpoint prefix: https://uksouth.livediagnostics.monitor.azure.com/

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions