Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

receiver/awsxray not allowing id to be missing in segment/cause/exception #37414

Open
Hampus-postkod opened this issue Jan 22, 2025 · 1 comment
Labels
bug Something isn't working needs triage New item requiring triage receiver/awsxray

Comments

@Hampus-postkod
Copy link

Component(s)

receiver/awsxray

What happened?

Description

There is a bug in receiver/awsxray that make the otel collector crash when receiving a trace containing a segment/cause/exception without an id attribute.

Sending the following trace:

{
  "Id": "1-678fbf94-446a5c2a65badb2f6b1a16c2",
  "Duration": 0.213,
  "LimitExceeded": false,
  "Segments": [
    {
      "Id": "1de48a95158d36ca",
      "Document": {
        "id": "1de48a95158d36ca",
        "name": "api_authorizer",
        "start_time": 1737473940.3929505,
        "trace_id": "1-678fbf94-446a5c2a65badb2f6b1a16c2",
        "end_time": 1737473940.601357,
        "parent_id": "35763592299be764",
        "error": true,
        "aws": {
          "account_id": "465143222094",
          "function_arn": "arn:aws:lambda:eu-west-1:465143222094:function:api_authorizer",
          "cloudwatch_logs": [
            {
              "log_group": "/aws/lambda/api_authorizer"
            }
          ],
          "resource_names": [
            "api_authorizer"
          ]
        },
        "origin": "AWS::Lambda::Function",
        "cause": {
          "working_directory": "/var/task",
          "paths": [
            "/src/authorizer/AuthorizerLambda.ts",
            "/src/authorizer/index.ts",
            "/node_modules/@middy/core/index.js"
          ],
          "exceptions": [
            {
              "message": "Unauthorized",
              "type": "UnauthorizedError",
              "stack": [
                {
                  "path": "/src/authorizer/AuthorizerLambda.ts",
                  "line": 34,
                  "label": "qs.handler"
                },
                {
                  "path": "/src/authorizer/index.ts",
                  "line": 199,
                  "label": "lambdaHandler"
                },
                {
                  "path": "/node_modules/@middy/core/index.js",
                  "line": 121,
                  "label": "yXe"
                }
              ]
            }
          ]
        }
      }
    }
  ]
}

Result in the following error:

1737537832989,panic: runtime error: invalid memory address or nil pointer dereference
1737537832989,[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xedbdb8]
1737537832989,goroutine 53 [running]:
1737537832989,"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/awsxrayreceiver/internal/translator.addCause(0x40010bdcc8?, {0x40012a20e0?, 0x4001097c14?})"
1737537832989,	github.com/open-telemetry/opentelemetry-collector-contrib/receiver/[email protected]/internal/translator/cause.go:61 +0x398
1737537832989,"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/awsxrayreceiver/internal/translator.populateSpan(0x40010bdcc8, 0x40012a05c0, 0x0, {0x40012a20e0?, 0x4001097c14?})"
1737537832989,	github.com/open-telemetry/opentelemetry-collector-contrib/receiver/[email protected]/internal/translator/translator.go:172 +0x320
1737537832989,"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/awsxrayreceiver/internal/translator.segToSpans({0x40012a05b0, 0x40012a05a0, 0x40010979a8, 0x0, 0x40012a0610, 0x0, 0x0, {0x0, 0x0, 0x0}, ...}, ...)"
1737537832989,	github.com/open-telemetry/opentelemetry-collector-contrib/receiver/[email protected]/internal/translator/translator.go:80 +0x11c
1737537832989,"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/awsxrayreceiver/internal/translator.ToTraces({0x40012a4000, 0xabe, 0xabe}, {0xffff40f62e40, 0x245f520})"
1737537832989,	github.com/open-telemetry/opentelemetry-collector-contrib/receiver/[email protected]/internal/translator/translator.go:68 +0x4c0
1737537832989,github.com/open-telemetry/opentelemetry-collector-contrib/receiver/awsxrayreceiver.(*xrayReceiver).start(0x4000206a80)
1737537832989,	github.com/open-telemetry/opentelemetry-collector-contrib/receiver/[email protected]/receiver.go:110 +0x114
1737537832989,created by github.com/open-telemetry/opentelemetry-collector-contrib/receiver/awsxrayreceiver.(*xrayReceiver).Start in goroutine 1
1737537832989,	github.com/open-telemetry/opentelemetry-collector-contrib/receiver/[email protected]/receiver.go:86 +0x78

Due to that the code in https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/awsxrayreceiver/internal/translator/cause.go consider the exception id to be mandatory:

	case awsxray.CauseTypeObject:
		evts := span.Events()
		// not sure whether there are existing events, so
		// append new empty events instead
		exceptionEventStartIndex := evts.Len()
		evts.EnsureCapacity(exceptionEventStartIndex + len(seg.Cause.Exceptions))

		for _, excp := range seg.Cause.Exceptions {
			evt := evts.AppendEmpty()
			evt.SetName(ExceptionEventName)
			attrs := evt.Attributes()
			attrs.EnsureCapacity(8)

			// ID is a required field
			attrs.PutStr(awsxray.AWSXrayExceptionIDAttribute, *excp.ID)
			addString(excp.Message, conventions.AttributeExceptionMessage, attrs)
			addString(excp.Type, conventions.AttributeExceptionType, attrs)
			addBool(excp.Remote, awsxray.AWSXrayExceptionRemoteAttribute, attrs)
			addInt64(excp.Truncated, awsxray.AWSXrayExceptionTruncatedAttribute, attrs)
			addInt64(excp.Skipped, awsxray.AWSXrayExceptionSkippedAttribute, attrs)
			addString(excp.Cause, awsxray.AWSXrayExceptionCauseAttribute, attrs)

			if len(excp.Stack) > 0 {
				stackTrace := convertStackFramesToStackTraceStr(excp)
				attrs.PutStr(conventions.AttributeExceptionStacktrace, stackTrace)
			}
		}
	}

Even though it is not according to AWS documentation

Steps to Reproduce

Send a trace with segment/casuse/exception without id attribute to otel-collector with reciever/awsxray.

Expected Result

Should accept request and keep running

Actual Result

Crashes with null pointer exception.

Collector version

v0.117.0

Environment information

Environment

OS: (e.g., "Ubuntu 20.04")
Compiler(if manually compiled): (e.g., "go 14.2")

OpenTelemetry Collector configuration

receivers:
  awsxray:
    transport: udp
    endpoint: 0.0.0.0:2000

processors:
  batch:

exporters:
  debug:
    verbosity: detailed

service:
  pipelines:
    traces:
      receivers:
        - awsxray
      processors:
        - batch
      exporters:
        - debug

Log output

1737537832989,panic: runtime error: invalid memory address or nil pointer dereference
1737537832989,[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xedbdb8]
1737537832989,goroutine 53 [running]:
1737537832989,"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/awsxrayreceiver/internal/translator.addCause(0x40010bdcc8?, {0x40012a20e0?, 0x4001097c14?})"
1737537832989,	github.com/open-telemetry/opentelemetry-collector-contrib/receiver/[email protected]/internal/translator/cause.go:61 +0x398
1737537832989,"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/awsxrayreceiver/internal/translator.populateSpan(0x40010bdcc8, 0x40012a05c0, 0x0, {0x40012a20e0?, 0x4001097c14?})"
1737537832989,	github.com/open-telemetry/opentelemetry-collector-contrib/receiver/[email protected]/internal/translator/translator.go:172 +0x320
1737537832989,"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/awsxrayreceiver/internal/translator.segToSpans({0x40012a05b0, 0x40012a05a0, 0x40010979a8, 0x0, 0x40012a0610, 0x0, 0x0, {0x0, 0x0, 0x0}, ...}, ...)"
1737537832989,	github.com/open-telemetry/opentelemetry-collector-contrib/receiver/[email protected]/internal/translator/translator.go:80 +0x11c
1737537832989,"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/awsxrayreceiver/internal/translator.ToTraces({0x40012a4000, 0xabe, 0xabe}, {0xffff40f62e40, 0x245f520})"
1737537832989,	github.com/open-telemetry/opentelemetry-collector-contrib/receiver/[email protected]/internal/translator/translator.go:68 +0x4c0
1737537832989,github.com/open-telemetry/opentelemetry-collector-contrib/receiver/awsxrayreceiver.(*xrayReceiver).start(0x4000206a80)
1737537832989,	github.com/open-telemetry/opentelemetry-collector-contrib/receiver/[email protected]/receiver.go:110 +0x114
1737537832989,created by github.com/open-telemetry/opentelemetry-collector-contrib/receiver/awsxrayreceiver.(*xrayReceiver).Start in goroutine 1
1737537832989,	github.com/open-telemetry/opentelemetry-collector-contrib/receiver/[email protected]/receiver.go:86 +0x78

Additional context

No response

@Hampus-postkod Hampus-postkod added bug Something isn't working needs triage New item requiring triage labels Jan 22, 2025
Copy link
Contributor

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working needs triage New item requiring triage receiver/awsxray
Projects
None yet
Development

No branches or pull requests

1 participant