Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to easily suppress dependency exception using java agent #3144

Open
petergphillips opened this issue Jun 20, 2023 · 4 comments
Open
Assignees

Comments

@petergphillips
Copy link

Expected behavior

I was hoping to suppress an exception using exception.message or exception.stacktrace that is flooding our logs - 900,000 times a day.

Actual behavior

I only appear to be able to suppress the exception using exception.type.

To Reproduce

I started off followed the steps detailed in Span Attributes Available for Sampling to look for the exception. Whilst I can see all the dependencies exported that way, it appears that the exceptions are exported differently and don't appear with text exporting span. In my case I can see the exception as json in the log, for example

{
  "ver": 1,
  "name": "Exception",
  "time": "2023-06-19T11:41:42.152Z",
  "iKey": "b44af8fc-1647-443d-8cea-55c5cfbd4518",
  "tags": {
    "ai.internal.sdkVersion": "java:3.4.13",
  },
  "data": {
    "baseType": "ExceptionData",
    "baseData": {
      "ver": 2,
      "exceptions": [
        {
          "typeName": "java.sql.SQLException",
          "message": "ORA-06550: line 1, column 28:",
          "stack": "java.sql.SQLException: ORA-06550: line 1, column 28:\nPLS-00302: component 'GETMAXIDENTIFIERLENGTH' must be declared..."
        }
      ]
    }
  }

I can get the suppression working with configuration such as

          "telemetryType": "exception",
          "attributes": [
            {
              "key": "exception.type",
              "value": "java.sql.SQLException",
              "matchType": "strict"
            }
          ],
          "percentage": 0

but would ideally like to match on the message or stacktrace instead. According to the opentelemetry exception documentation I think I should be matching on exception.message or exception.stacktrace, but can't get either to work. For example I think:

          "telemetryType": "exception",
          "attributes": [
            {
              "key": "exception.stacktrace",
              "value": ".*GETDBCOMPATMAJORVERSION.*",
              "matchType": "regexp"
            }
          ],
          "percentage": 0

should do the trick, but the exception still appears in the logs. I've also tried suppressing the dependency, which works fine with the above regexp but doesn't suppress the corresponding exception.

Please could you let me know the key that I need therefore to match on the stacktrace and / or message to suppress the exception. Looking at the Oracle code that throws the exception it appears to be caught and doesn't then affect the behaviour of the application, hence the reason for the suppression.

System information

Please provide the following information:

  • SDK Version: java:3.4.13
  • OS type and version: Based off docker image eclipse-temurin:19-jre-jammy so Java 19
  • Application Server type and version (if applicable): Spring Boot 3.1.0
  • Using spring-boot? Yes
@heyams heyams self-assigned this Jun 20, 2023
@heyams
Copy link
Contributor

heyams commented Jun 20, 2023

@petergphillips
Copy link
Author

hi @heyams yes, that is the same document that I followed. I saw all the dependencies that were exported with the exporting span, but couldn't see any exceptions with that text. I have attached some of the log for you to confirm
events.log

@heyams
Copy link
Contributor

heyams commented Jun 21, 2023

@petergphillips i have checked your log. there were 4 exceptions and none of them had attributes.
that's why sampling override didn't work.

can you submit a repro app here?
I want to understand why those attributes weren't generated.

{
  "ver": 1,
  "name": "Exception",
  "time": "2023-06-19T11:41:42.152Z",
  "iKey": "<REDACTED>",
  "tags": {
    "ai.internal.sdkVersion": "java:3.4.13",
  },
  "data": {
    "baseType": "ExceptionData",
    "baseData": {
      "ver": 2,
      "exceptions": [
        {
          "typeName": "java.sql.SQLException",
          "message": "ORA-06550: line 1, column 28:",
          "stack": "java.sql.SQLException: ORA-06550: line 1, column 28:\nPLS-00302: component 'GETMAXIDENTIFIERLENGTH' must be declared..."
        }
      ]
    }
  }

this is the payload we send to Application Insights after the sampling overrides have been applied.

@petergphillips
Copy link
Author

Sorry, I won't be able to create a repo that reproduces the issue. It requires an Oracle 11g database with Oracle adque installed. We are connecting to the queue using an newer version of the Oracle adque library which causes an incompatibility and thus a stacktrace is generated deep within the Oracle library.

What I find odd here is that for the exception we don't get any exporting span events at all. The exporting span events only seem to be generated for dependencies in this example.

Note that we are able to suppress the exception by targeting exception.type so the suppressions are working, just not in the way that we would like as suppressing all sql exceptions is a bit too broad.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants