tetragon v1.5 pod crash due to "concurrent map read and map write" #4057
Description
What happened? How can we reproduce this?
-
in the prod env
-
this is the tetragon-config : debug: "false"
enable-k8s-api: "true"
enable-pod-info: "false"
enable-policy-filter: "true"
enable-process-cred: "false"
enable-process-ns: "false"
enable-tracing-policy-crd: "true"
export-allowlist: '{"event_set":["PROCESS_KPROBE","PROCESS_TRACEPOINT"]}'
export-denylist: '{"namespace":["monitoring"]}'
export-file-compress: ""
export-file-max-backups: ""
export-file-max-size-mb: ""
export-file-perm: ""
export-filename: ""
export-rate-limit: "-1"
field-filters: ""
gops-address: localhost:8118
health-server-address: :6789
health-server-interval: "10"
metrics-label-filter: namespace,workload,pod,binary
metrics-server: 0.0.0.0:2112
process-cache-size: "65536"
procfs: /procRoot
redaction-filters: ""
server-address: 127.0.0.1:54321
username-metadata: unix -
our tetragon pod with about 10% will be restart after i upgrade to v1.5 --config-dir=/etc/tetragon/tetragon.conf.d/
State: Running
Started: Thu, 28 Aug 2025 00:18:12 +0800
Last State: Terminated
Reason: Error
Message: ang.org/x/net/http2.(*pipe).Read(0xc001e4c630, {0xc004e04418, 0x4, 0x4})
/go/src/github.com/cilium/tetragon/vendor/golang.org/x/net/http2/pipe.go:76 +0xd6 fp=0xc000db3c10 sp=0xc000db3ba8 pc=0x8b2c96
golang.org/x/net/http2.transportResponseBody.Read({0x2?}, {0xc004e04418?, 0xc000db3cd0?, 0x419d45?})
/go/src/github.com/cilium/tetragon/vendor/golang.org/x/net/http2/transport.go:2560 +0x59 fp=0xc000db3c98 sp=0xc000db3c10 pc=0x8bffd9
io.ReadAtLeast({0x7ddc4599c2f0, 0xc001e4c600}, {0xc004e04418, 0x4, 0x4}, 0x4)
/usr/local/go/src/io/io.go:335 +0x91 fp=0xc000db3ce0 sp=0xc000db3c98 pc=0x496371
k8s.io/apimachinery/pkg/util/framer.(*lengthDelimitedFrameReader).Read(0xc00178ab28, {0xc002db6000, 0x4000, 0x5000})
/go/src/github.com/cilium/tetragon/vendor/k8s.io/apimachinery/pkg/util/framer/framer.go:76 +0x7f fp=0xc000db3d40 sp=0xc000db3ce0 pc=0xbeeabf
k8s.io/apimachinery/pkg/runtime/serializer/streaming.(*decoder).Decode(0xc00270ac30, 0x0, {0x286f050, 0xc002be4040})
/go/src/github.com/cilium/tetragon/vendor/k8s.io/apimachinery/pkg/runtime/serializer/streaming/streaming.go:77 +0xa3 fp=0xc000db3df0 sp=0xc000db3d40 pc=0xc1e903
k8s.io/client-go/rest/watch.(*Decoder).Decode(0xc0034ccb80)
/go/src/github.com/cilium/tetragon/vendor/k8s.io/client-go/rest/watch/decoder.go:49 +0x4b fp=0xc000db3e50 sp=0xc000db3df0 pc=0xc1ec4b
k8s.io/apimachinery/pkg/watch.(*StreamWatcher).receive(0xc00270ad70)
/go/src/github.com/cilium/tetragon/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:114 +0xf8 fp=0xc000db3fc8 sp=0xc000db3e50 pc=0xb6d038
k8s.io/apimachinery/pkg/watch.NewStreamWatcherWithLogger.gowrap1()
/go/src/github.com/cilium/tetragon/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:85 +0x25 fp=0xc000db3fe0 sp=0xc000db3fc8 pc=0xb6cda5
runtime.goexit({})
/usr/local/go/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000db3fe8 sp=0xc000db3fe0 pc=0x481c01
created by k8s.io/apimachinery/pkg/watch.NewStreamWatcherWithLogger in goroutine 344
/go/src/github.com/cilium/tetragon/vendor/k8s.io/apimachinery/pkg/watch/streamwatcher.go:85 +0x149Exit Code: 2
Started: Wed, 27 Aug 2025 19:59:22 +0800
Finished: Thu, 28 Aug 2025 00:18:12 +0800
Ready: True
Restart Count: 5
Tetragon Version
1.5
Kernel Version
kernel version 6.8.12
Kubernetes Version
Kubernetes 1.32
Bugtool
Relevant log output
1756269215107 /usr/local/go/src/runtime/mgc.go:1339 +0x105
1756269215107 created by runtime.gcBgMarkStartWorkers in goroutine 1
1756269215107 /usr/local/go/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc00021cfe8 sp=0xc00021cfe0 pc=0x481c01
1756269215107 runtime.goexit({})
1756269215107 /usr/local/go/src/runtime/mgc.go:1339 +0x25 fp=0xc00021cfe0 sp=0xc00021cfc8 pc=0x4230e5
1756269215107 runtime.gcBgMarkStartWorkers.gowrap1()
1756269215107 /usr/local/go/src/runtime/mgc.go:1423 +0xe9 fp=0xc00021cfc8 sp=0xc00021cf38 pc=0x423209
1756269215107 runtime.gcBgMarkWorker(0xc0001d2070)
1756269215107 /usr/local/go/src/runtime/proc.go:435 +0xce fp=0xc00021cf38 sp=0xc00021cf18 pc=0x479c6e
1756269215107 runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
1756269215107 goroutine 23 gp=0xc00028ca80 m=nil [GC worker (idle), 258 minutes]:
1756269215107
1756269215107 /usr/local/go/src/runtime/mgc.go:1339 +0x105
1756269215107 created by runtime.gcBgMarkStartWorkers in goroutine 1
1756269215107 /usr/local/go/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc00021c7e8 sp=0xc00021c7e0 pc=0x481c01
1756269215107 runtime.goexit({})
1756269215107 /usr/local/go/src/runtime/mgc.go:1339 +0x25 fp=0xc00021c7e0 sp=0xc00021c7c8 pc=0x4230e5
1756269215107 runtime.gcBgMarkStartWorkers.gowrap1()
1756269215107 /usr/local/go/src/runtime/mgc.go:1423 +0xe9 fp=0xc00021c7c8 sp=0xc00021c738 pc=0x423209
1756269215107 runtime.gcBgMarkWorker(0xc0001d2070)
1756269215107 /usr/local/go/src/runtime/proc.go:435 +0xce fp=0xc00021c738 sp=0xc00021c718 pc=0x479c6e
1756269215107 runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
1756269215107 goroutine 22 gp=0xc00028c8c0 m=nil [GC worker (idle), 258 minutes]:
1756269215107
1756269215107 /usr/local/go/src/runtime/mgc.go:1339 +0x105
1756269215107 created by runtime.gcBgMarkStartWorkers in goroutine 1
1756269215107 /usr/local/go/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc00021bfe8 sp=0xc00021bfe0 pc=0x481c01
1756269215107 runtime.goexit({})
1756269215107 /usr/local/go/src/runtime/mgc.go:1339 +0x25 fp=0xc00021bfe0 sp=0xc00021bfc8 pc=0x4230e5
1756269215107 runtime.gcBgMarkStartWorkers.gowrap1()
1756269215107 /usr/local/go/src/runtime/mgc.go:1423 +0xe9 fp=0xc00021bfc8 sp=0xc00021bf38 pc=0x423209
1756269215107 runtime.gcBgMarkWorker(0xc0001d2070)
1756269215107 /usr/local/go/src/runtime/proc.go:435 +0xce fp=0xc00021bf38 sp=0xc00021bf18 pc=0x479c6e
1756269215107 runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
1756269215107 goroutine 21 gp=0xc00028c700 m=nil [GC worker (idle), 258 minutes]:
1756269215107
1756269215107 /usr/local/go/src/runtime/mgc.go:1339 +0x105
1756269215106 created by runtime.gcBgMarkStartWorkers in goroutine 1
1756269215106 /usr/local/go/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc00021b7e8 sp=0xc00021b7e0 pc=0x481c01
1756269215106 runtime.goexit({})
1756269215106 /usr/local/go/src/runtime/mgc.go:1339 +0x25 fp=0xc00021b7e0 sp=0xc00021b7c8 pc=0x4230e5
1756269215106 runtime.gcBgMarkStartWorkers.gowrap1()
1756269215106 /usr/local/go/src/runtime/mgc.go:1423 +0xe9 fp=0xc00021b7c8 sp=0xc00021b738 pc=0x423209
1756269215106 runtime.gcBgMarkWorker(0xc0001d2070)
1756269215106 /usr/local/go/src/runtime/proc.go:435 +0xce fp=0xc00021b738 sp=0xc00021b718 pc=0x479c6e
1756269215106 runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
1756269215106 goroutine 20 gp=0xc00028c540 m=nil [GC worker (idle), 258 minutes]:
1756269215106
1756269215106 /usr/local/go/src/runtime/mgc.go:1339 +0x105
1756269215106 created by runtime.gcBgMarkStartWorkers in goroutine 1
1756269215106 /usr/local/go/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc00021afe8 sp=0xc00021afe0 pc=0x481c01
1756269215106 runtime.goexit({})
1756269215106 /usr/local/go/src/runtime/mgc.go:1339 +0x25 fp=0xc00021afe0 sp=0xc00021afc8 pc=0x4230e5
1756269215106 runtime.gcBgMarkStartWorkers.gowrap1()
1756269215106 /usr/local/go/src/runtime/mgc.go:1423 +0xe9 fp=0xc00021afc8 sp=0xc00021af38 pc=0x423209
1756269215106 runtime.gcBgMarkWorker(0xc0001d2070)
1756269215106 /usr/local/go/src/runtime/proc.go:435 +0xce fp=0xc00021af38 sp=0xc00021af18 pc=0x479c6e
1756269215106 runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
1756269215106 goroutine 19 gp=0xc00028c380 m=nil [GC worker (idle), 258 minutes]:
1756269215106
1756269215106 /usr/local/go/src/runtime/mgc.go:1339 +0x105
1756269215106 created by runtime.gcBgMarkStartWorkers in goroutine 1
1756269215106 /usr/local/go/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc00021a7e8 sp=0xc00021a7e0 pc=0x481c01
1756269215106 runtime.goexit({})
1756269215106 /usr/local/go/src/runtime/mgc.go:1339 +0x25 fp=0xc00021a7e0 sp=0xc00021a7c8 pc=0x4230e5
1756269215106 runtime.gcBgMarkStartWorkers.gowrap1()
1756269215106 /usr/local/go/src/runtime/mgc.go:1423 +0xe9 fp=0xc00021a7c8 sp=0xc00021a738 pc=0x423209
1756269215106 runtime.gcBgMarkWorker(0xc0001d2070)
1756269215106 /usr/local/go/src/runtime/proc.go:435 +0xce fp=0xc00021a738 sp=0xc00021a718 pc=0x479c6e
1756269215106 runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
1756269215106 goroutine 18 gp=0xc00028c1c0 m=nil [GC worker (idle), 258 minutes]:
1756269215106
1756269215106 /usr/local/go/src/runtime/mgc.go:1339 +0x105
1756269215106 created by runtime.gcBgMarkStartWorkers in goroutine 1
1756269215106 /usr/local/go/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc0001a2fe8 sp=0xc0001a2fe0 pc=0x481c01
1756269215106 runtime.goexit({})
1756269215106 /usr/local/go/src/runtime/mgc.go:1339 +0x25 fp=0xc0001a2fe0 sp=0xc0001a2fc8 pc=0x4230e5
1756269215105 runtime.gcBgMarkStartWorkers.gowrap1()
1756269215105 /usr/local/go/src/runtime/mgc.go:1423 +0xe9 fp=0xc0001a2fc8 sp=0xc0001a2f38 pc=0x423209
1756269215105 runtime.gcBgMarkWorker(0xc0001d2070)
1756269215105 /usr/local/go/src/runtime/proc.go:435 +0xce fp=0xc0001a2f38 sp=0xc0001a2f18 pc=0x479c6e
1756269215105 runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
1756269215105 goroutine 10 gp=0xc000003dc0 m=nil [GC worker (idle), 258 minutes]:
1756269215105
1756269215105 /usr/local/go/src/runtime/mgc.go:1339 +0x105
1756269215105 created by runtime.gcBgMarkStartWorkers in goroutine 1
1756269215105 /usr/local/go/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc0002207e8 sp=0xc0002207e0 pc=0x481c01
1756269215105 runtime.goexit({})
1756269215105 /usr/local/go/src/runtime/mgc.go:1339 +0x25 fp=0xc0002207e0 sp=0xc0002207c8 pc=0x4230e5
1756269215105 runtime.gcBgMarkStartWorkers.gowrap1()
1756269215105 /usr/local/go/src/runtime/mgc.go:1423 +0xe9 fp=0xc0002207c8 sp=0xc000220738 pc=0x423209
1756269215105 runtime.gcBgMarkWorker(0xc0001d2070)
1756269215105 /usr/local/go/src/runtime/proc.go:435 +0xce fp=0xc000220738 sp=0xc000220718 pc=0x479c6e
1756269215105 runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
1756269215105 goroutine 37 gp=0xc000202a80 m=nil [GC worker (idle), 258 minutes]:
1756269215105
1756269215105 /usr/local/go/src/runtime/mgc.go:1339 +0x105
1756269215105 created by runtime.gcBgMarkStartWorkers in goroutine 1
1756269215105 /usr/local/go/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc00021ffe8 sp=0xc00021ffe0 pc=0x481c01
1756269215105 runtime.goexit({})
1756269215105 /usr/local/go/src/runtime/mgc.go:1339 +0x25 fp=0xc00021ffe0 sp=0xc00021ffc8 pc=0x4230e5
1756269215105 runtime.gcBgMarkStartWorkers.gowrap1()
1756269215105 /usr/local/go/src/runtime/mgc.go:1423 +0xe9 fp=0xc00021ffc8 sp=0xc00021ff38 pc=0x423209
1756269215105 runtime.gcBgMarkWorker(0xc0001d2070)
1756269215105 /usr/local/go/src/runtime/proc.go:435 +0xce fp=0xc00021ff38 sp=0xc00021ff18 pc=0x479c6e
1756269215105 runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
1756269215105 goroutine 36 gp=0xc0002028c0 m=nil [GC worker (idle), 258 minutes]:
1756269215105
1756269215105 /usr/local/go/src/runtime/mgc.go:1339 +0x105
1756269215105 created by runtime.gcBgMarkStartWorkers in goroutine 1
1756269215105 /usr/local/go/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc00021f7e8 sp=0xc00021f7e0 pc=0x481c01
1756269215105 runtime.goexit({})
1756269215105 /usr/local/go/src/runtime/mgc.go:1339 +0x25 fp=0xc00021f7e0 sp=0xc00021f7c8 pc=0x4230e5
1756269215105 runtime.gcBgMarkStartWorkers.gowrap1()
1756269215105 /usr/local/go/src/runtime/mgc.go:1423 +0xe9 fp=0xc00021f7c8 sp=0xc00021f738 pc=0x423209
1756269215105 runtime.gcBgMarkWorker(0xc0001d2070)
1756269215105 /usr/local/go/src/runtime/proc.go:435 +0xce fp=0xc00021f738 sp=0xc00021f718 pc=0x479c6e
1756269215104 runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
1756269215104 goroutine 35 gp=0xc000202700 m=nil [GC worker (idle), 258 minutes]:
1756269215104
1756269215104 /usr/local/go/src/runtime/mgc.go:1339 +0x105
1756269215104 created by runtime.gcBgMarkStartWorkers in goroutine 1
1756269215104 /usr/local/go/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc00021efe8 sp=0xc00021efe0 pc=0x481c01
1756269215104 runtime.goexit({})
1756269215104 /usr/local/go/src/runtime/mgc.go:1339 +0x25 fp=0xc00021efe0 sp=0xc00021efc8 pc=0x4230e5
1756269215104 runtime.gcBgMarkStartWorkers.gowrap1()
1756269215104 /usr/local/go/src/runtime/mgc.go:1423 +0xe9 fp=0xc00021efc8 sp=0xc00021ef38 pc=0x423209
1756269215104 runtime.gcBgMarkWorker(0xc0001d2070)
1756269215104 /usr/local/go/src/runtime/proc.go:435 +0xce fp=0xc00021ef38 sp=0xc00021ef18 pc=0x479c6e
1756269215104 runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
1756269215104 goroutine 34 gp=0xc000202540 m=nil [GC worker (idle), 258 minutes]:
1756269215104
1756269215104 /usr/local/go/src/runtime/mgc.go:1339 +0x105
1756269215104 created by runtime.gcBgMarkStartWorkers in goroutine 1
1756269215104 /usr/local/go/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc00021e7e8 sp=0xc00021e7e0 pc=0x481c01
1756269215104 runtime.goexit({})
1756269215104 /usr/local/go/src/runtime/mgc.go:1339 +0x25 fp=0xc00021e7e0 sp=0xc00021e7c8 pc=0x4230e5
1756269215104 runtime.gcBgMarkStartWorkers.gowrap1()
1756269215104 /usr/local/go/src/runtime/mgc.go:1423 +0xe9 fp=0xc00021e7c8 sp=0xc00021e738 pc=0x423209
1756269215104 runtime.gcBgMarkWorker(0xc0001d2070)
1756269215104 /usr/local/go/src/runtime/proc.go:435 +0xce fp=0xc00021e738 sp=0xc00021e718 pc=0x479c6e
1756269215104 runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
1756269215104 goroutine 33 gp=0xc000202380 m=nil [GC worker (idle), 258 minutes]:
1756269215104
1756269215104 /usr/local/go/src/runtime/mgc.go:1339 +0x105
1756269215104 created by runtime.gcBgMarkStartWorkers in goroutine 1
1756269215104 /usr/local/go/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc0001a27e8 sp=0xc0001a27e0 pc=0x481c01
1756269215104 runtime.goexit({})
1756269215104 /usr/local/go/src/runtime/mgc.go:1339 +0x25 fp=0xc0001a27e0 sp=0xc0001a27c8 pc=0x4230e5
1756269215104 runtime.gcBgMarkStartWorkers.gowrap1()
1756269215104 /usr/local/go/src/runtime/mgc.go:1423 +0xe9 fp=0xc0001a27c8 sp=0xc0001a2738 pc=0x423209
1756269215104 runtime.gcBgMarkWorker(0xc0001d2070)
1756269215104 /usr/local/go/src/runtime/proc.go:435 +0xce fp=0xc0001a2738 sp=0xc0001a2718 pc=0x479c6e
1756269215104 runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
1756269215104 goroutine 9 gp=0xc000003c00 m=nil [GC worker (idle), 258 minutes]:
1756269215104
1756269215104 /usr/local/go/src/runtime/mgc.go:1339 +0x105
1756269215104 created by runtime.gcBgMarkStartWorkers in goroutine 1
1756269215104 /usr/local/go/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc0001a67e8 sp=0xc0001a67e0 pc=0x481c01
1756269215103 runtime.goexit({})
1756269215103 /usr/local/go/src/runtime/mgc.go:1339 +0x25 fp=0xc0001a67e0 sp=0xc0001a67c8 pc=0x4230e5
1756269215103 runtime.gcBgMarkStartWorkers.gowrap1()
1756269215103 /usr/local/go/src/runtime/mgc.go:1423 +0xe9 fp=0xc0001a67c8 sp=0xc0001a6738 pc=0x423209
1756269215103 runtime.gcBgMarkWorker(0xc0001d2070)
1756269215103 /usr/local/go/src/runtime/proc.go:435 +0xce fp=0xc0001a6738 sp=0xc0001a6718 pc=0x479c6e
1756269215103 runtime.gopark(0xc00019e080?, 0xc0001a67d0?, 0xd3?, 0x48?, 0x3beca60?)
1756269215103 goroutine 17 gp=0xc00028c000 m=nil [GC worker (idle), 258 minutes]:
1756269215103
1756269215103 /usr/local/go/src/runtime/mgc.go:1339 +0x105
1756269215103 created by runtime.gcBgMarkStartWorkers in goroutine 1
1756269215103 /usr/local/go/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc0001a9fe8 sp=0xc0001a9fe0 pc=0x481c01
1756269215103 runtime.goexit({})
1756269215103 /usr/local/go/src/runtime/mgc.go:1339 +0x25 fp=0xc0001a9fe0 sp=0xc0001a9fc8 pc=0x4230e5
1756269215103 runtime.gcBgMarkStartWorkers.gowrap1()
1756269215103 /usr/local/go/src/runtime/mgc.go:1423 +0xe9 fp=0xc0001a9fc8 sp=0xc0001a9f38 pc=0x423209
1756269215103 runtime.gcBgMarkWorker(0xc0001d2070)
1756269215103 /usr/local/go/src/runtime/proc.go:435 +0xce fp=0xc0001a9f38 sp=0xc0001a9f18 pc=0x479c6e
1756269215103 runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
1756269215103 goroutine 8 gp=0xc000003a40 m=nil [GC worker (idle), 258 minutes]:
1756269215103
1756269215103 /usr/local/go/src/runtime/mgc.go:1339 +0x105
1756269215103 created by runtime.gcBgMarkStartWorkers in goroutine 1
1756269215103 /usr/local/go/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc0001a97e8 sp=0xc0001a97e0 pc=0x481c01
1756269215103 runtime.goexit({})
1756269215103 /usr/local/go/src/runtime/mgc.go:1339 +0x25 fp=0xc0001a97e0 sp=0xc0001a97c8 pc=0x4230e5
1756269215103 runtime.gcBgMarkStartWorkers.gowrap1()
1756269215103 /usr/local/go/src/runtime/mgc.go:1423 +0xe9 fp=0xc0001a97c8 sp=0xc0001a9738 pc=0x423209
1756269215103 runtime.gcBgMarkWorker(0xc0001d2070)
1756269215103 /usr/local/go/src/runtime/proc.go:435 +0xce fp=0xc0001a9738 sp=0xc0001a9718 pc=0x479c6e
1756269215103 runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
1756269215103 goroutine 7 gp=0xc000003880 m=nil [GC worker (idle), 258 minutes]:
1756269215103
1756269215103 /usr/local/go/src/runtime/mgc.go:1339 +0x105
1756269215103 created by runtime.gcBgMarkStartWorkers in goroutine 1
1756269215103 /usr/local/go/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc0001a8fe8 sp=0xc0001a8fe0 pc=0x481c01
1756269215103 runtime.goexit({})
1756269215103 /usr/local/go/src/runtime/mgc.go:1339 +0x25 fp=0xc0001a8fe0 sp=0xc0001a8fc8 pc=0x4230e5
1756269215103 runtime.gcBgMarkStartWorkers.gowrap1()
1756269215103 /usr/local/go/src/runtime/mgc.go:1423 +0xe9 fp=0xc0001a8fc8 sp=0xc0001a8f38 pc=0x423209
1756269215102 runtime.gcBgMarkWorker(0xc0001d2070)
1756269215102 /usr/local/go/src/runtime/proc.go:435 +0xce fp=0xc0001a8f38 sp=0xc0001a8f18 pc=0x479c6e
1756269215102 runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
1756269215102 goroutine 6 gp=0xc0000036c0 m=nil [GC worker (idle), 258 minutes]:
1756269215102
1756269215102 /usr/local/go/src/runtime/mgc.go:1339 +0x105
1756269215102 created by runtime.gcBgMarkStartWorkers in goroutine 1
1756269215102 /usr/local/go/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc0001a87e8 sp=0xc0001a87e0 pc=0x481c01
1756269215102 runtime.goexit({})
1756269215102 /usr/local/go/src/runtime/mgc.go:1339 +0x25 fp=0xc0001a87e0 sp=0xc0001a87c8 pc=0x4230e5
1756269215102 runtime.gcBgMarkStartWorkers.gowrap1()
1756269215102 /usr/local/go/src/runtime/mgc.go:1423 +0xe9 fp=0xc0001a87c8 sp=0xc0001a8738 pc=0x423209
1756269215102 runtime.gcBgMarkWorker(0xc0001d2070)
1756269215102 /usr/local/go/src/runtime/proc.go:435 +0xce fp=0xc0001a8738 sp=0xc0001a8718 pc=0x479c6e
1756269215102 runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
1756269215102 goroutine 5 gp=0xc000003500 m=nil [GC worker (idle), 258 minutes]:
1756269215102
1756269215102 /usr/local/go/src/runtime/mgc.go:205 +0xa5
1756269215102 created by runtime.gcenable in goroutine 1
1756269215102 /usr/local/go/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc0001a7fe8 sp=0xc0001a7fe0 pc=0x481c01
1756269215102 runtime.goexit({})
1756269215102 /usr/local/go/src/runtime/mgc.go:205 +0x25 fp=0xc0001a7fe0 sp=0xc0001a7fc8 pc=0x420d45
1756269215102 runtime.gcenable.gowrap2()
1756269215102 /usr/local/go/src/runtime/mgcscavenge.go:658 +0x59 fp=0xc0001a7fc8 sp=0xc0001a7fa8 pc=0x42a9b9
1756269215102 runtime.bgscavenge(0xc00019e080)
1756269215102 /usr/local/go/src/runtime/mgcscavenge.go:425 +0x49 fp=0xc0001a7fa8 sp=0xc0001a7f78 pc=0x42a429
1756269215102 runtime.(*scavengerState).park(0x3d31ce0)
1756269215102 /usr/local/go/src/runtime/proc.go:441
1756269215102 runtime.goparkunlock(...)
1756269215102 /usr/local/go/src/runtime/proc.go:435 +0xce fp=0xc0001a7f78 sp=0xc0001a7f58 pc=0x479c6e
1756269215102 runtime.gopark(0x108f3c?, 0x5dccd?, 0x0?, 0x0?, 0x0?)
1756269215102 goroutine 4 gp=0xc000002fc0 m=nil [GC scavenge wait]:
1756269215102
1756269215102 /usr/local/go/src/runtime/mgc.go:204 +0x66
1756269215102 created by runtime.gcenable in goroutine 1
1756269215102 /usr/local/go/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc0001a77e8 sp=0xc0001a77e0 pc=0x481c01
1756269215102 runtime.goexit({})
1756269215102 /usr/local/go/src/runtime/mgc.go:204 +0x25 fp=0xc0001a77e0 sp=0xc0001a77c8 pc=0x420da5
1756269215102 runtime.gcenable.gowrap1()
1756269215102 /usr/local/go/src/runtime/mgcsweep.go:316 +0xdf fp=0xc0001a77c8 sp=0xc0001a7780 pc=0x42c9df
1756269215102 runtime.bgsweep(0xc00019e080)
1756269215102 /usr/local/go/src/runtime/proc.go:441
1756269215102 runtime.goparkunlock(...)
1756269215101 /usr/local/go/src/runtime/proc.go:435 +0xce fp=0xc0001a7780 sp=0xc0001a7760 pc=0x479c6e
1756269215101 runtime.gopark(0x3d24e01?, 0x0?, 0x0?, 0x0?, 0x0?)
1756269215101 goroutine 3 gp=0xc000002e00 m=nil [GC sweep wait]:
1756269215101
1756269215101 /usr/local/go/src/runtime/proc.go:336 +0x1a
1756269215101 created by runtime.init.7 in goroutine 1
1756269215101 /usr/local/go/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc0001a6fe8 sp=0xc0001a6fe0 pc=0x481c01
1756269215101 runtime.goexit({})
1756269215101 /usr/local/go/src/runtime/proc.go:348 +0xb3 fp=0xc0001a6fe0 sp=0xc0001a6fa8 pc=0x444d33
1756269215101 runtime.forcegchelper()
1756269215101 /usr/local/go/src/runtime/proc.go:441
1756269215101 runtime.goparkunlock(...)
1756269215101 /usr/local/go/src/runtime/proc.go:435 +0xce fp=0xc0001a6fa8 sp=0xc0001a6f88 pc=0x479c6e
1756269215101 runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
1756269215101 goroutine 2 gp=0xc0000028c0 m=nil [force gc (idle), 258 minutes]:
1756269215101
1756269215101 /usr/local/go/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc001e9dfe8 sp=0xc001e9dfe0 pc=0x481c01
1756269215101 runtime.goexit({})
1756269215101 /usr/local/go/src/runtime/proc.go:283 +0x28b fp=0xc001e9dfe0 sp=0xc001e9df50 pc=0x4449eb
1756269215101 runtime.main()
1756269215101 /go/src/github.com/cilium/tetragon/cmd/tetragon/tetragon.go:12 +0x13 fp=0xc001e9df50 sp=0xc001e9df08 pc=0x1e6bcb3
1756269215101 main.main()
1756269215101 /go/src/github.com/cilium/tetragon/cmd/tetragon/main.go:827 +0x16f fp=0xc001e9df08 sp=0xc001e9dea0 pc=0x1e6b9ef
1756269215101 main.execute()
1756269215101 /go/src/github.com/cilium/tetragon/vendor/github.com/spf13/cobra/command.go:1071
1756269215101 github.com/spf13/cobra.(*Command).Execute(...)
1756269215101 /go/src/github.com/cilium/tetragon/vendor/github.com/spf13/cobra/command.go:1148 +0x46f fp=0xc001e9dea0 sp=0xc001e9ddb0 pc=0x1425b2f
1756269215101 github.com/spf13/cobra.(*Command).ExecuteC(0xc000360c08)
1756269215101 /go/src/github.com/cilium/tetragon/vendor/github.com/spf13/cobra/command.go:1019 +0xa91 fp=0xc001e9ddb0 sp=0xc001e9dc30 pc=0x14251d1
1756269215101 github.com/spf13/cobra.(*Command).execute(0xc000360c08, {0xc000244150, 0x1, 0x1})
1756269215101 /go/src/github.com/cilium/tetragon/cmd/tetragon/main.go:814 +0x245 fp=0xc001e9dc30 sp=0xc001e9dba8 pc=0x1e6bf65
1756269215101 main.execute.func1(0xc000360c08, {0x24a230d?, 0x4?, 0x24a2311?})
1756269215101 /go/src/github.com/cilium/tetragon/cmd/tetragon/main.go:205 +0x4d fp=0xc001e9dba8 sp=0xc001e9db58 pc=0x1e66fcd
1756269215101 main.tetragonExecute()
1756269215101 /go/src/github.com/cilium/tetragon/cmd/tetragon/main.go:544 +0x1c10 fp=0xc001e9db58 sp=0xc001e9d630 pc=0x1e68c30
1756269215101 main.tetragonExecuteCtx({0x2889930, 0xc0008a6460}, 0xc00099e340, 0x25fc848)
1756269215101 /go/src/github.com/cilium/tetragon/pkg/observer/observer.go:195 +0x6b fp=0xc001e9d630 sp=0xc001e9d5e8 pc=0x1bc6bcb
1756269215101 github.com/cilium/tetragon/pkg/observer.(*Observer).StartReady(0xc000a7f020, {0x2889930, 0xc00031c5f0}, 0x25fc848)
1756269215101 /go/src/github.com/cilium/tetragon/pkg/observer/observer_linux.go:145 +0x425 fp=0xc001e9d5e8 sp=0xc001e9d4f8 pc=0x1bc7e65
1756269215100 github.com/cilium/tetragon/pkg/observer.(*Observer).RunEvents(0xc000a7f020, {0x2889930, 0xc00031c5f0}, 0x25fc848)
1756269215100 /usr/local/go/src/runtime/chan.go:506 +0x12 fp=0xc001e9d4f8 sp=0xc001e9d4d0 pc=0x4118b2
1756269215100 runtime.chanrecv1(0xc000a7f020?, 0x24c9884?)
1756269215100 /usr/local/go/src/runtime/chan.go:664 +0x445 fp=0xc001e9d4d0 sp=0xc001e9d458 pc=0x411d25
1756269215100 runtime.chanrecv(0xc0048d8070, 0x0, 0x1)
1756269215100 /usr/local/go/src/runtime/proc.go:435 +0xce fp=0xc001e9d458 sp=0xc001e9d438 pc=0x479c6e
1756269215100 runtime.gopark(0x0?, 0x500000000000000?, 0x28?, 0x3f?, 0x726d80a1e5c0?)
1756269215100 goroutine 1 gp=0xc000002380 m=nil [chan receive, 258 minutes]:
1756269215100
1756269215100 /go/src/github.com/cilium/tetragon/pkg/observer/observer_linux.go:123 +0x3f7
1756269215100 created by github.com/cilium/tetragon/pkg/observer.(*Observer).RunEvents in goroutine 1
1756269215100 /usr/local/go/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc00178ffe8 sp=0xc00178ffe0 pc=0x481c01
1756269215100 runtime.goexit({})
1756269215100 /go/src/github.com/cilium/tetragon/pkg/observer/observer_linux.go:128 +0x9d fp=0xc00178ffe0 sp=0xc00178ff10 pc=0x1bc7f9d
1756269215100 github.com/cilium/tetragon/pkg/observer.(*Observer).RunEvents.func2()
1756269215100 /go/src/github.com/cilium/tetragon/pkg/observer/observer.go:129 +0x22b fp=0xc00178ff10 sp=0xc00178fe58 pc=0x1bc670b
1756269215100 github.com/cilium/tetragon/pkg/observer.(*Observer).receiveEvent(0xc000a7f020, {0xc00373c000?, 0x3c?, 0x0?})
1756269215100 /go/src/github.com/cilium/tetragon/pkg/observer/observer.go:45 +0xa2 fp=0xc00178fe58 sp=0xc00178fda8 pc=0x1bc5f82
1756269215100 github.com/cilium/tetragon/pkg/observer.(*Observer).observerListeners(0xc000a7f020, {0x288d2a0, 0xc001d6a2f0})
1756269215100 /go/src/github.com/cilium/tetragon/pkg/grpc/process_manager.go:55 +0x26 fp=0xc00178fda8 sp=0xc00178fd78 pc=0x1bd1d26
1756269215100 github.com/cilium/tetragon/pkg/grpc.(*ProcessManager).Notify(0xc001d01b18, {0x288d2a0, 0xc001d6a2f0})
1756269215100 /go/src/github.com/cilium/tetragon/pkg/grpc/exec/exec.go:283 +0x2b fp=0xc00178fd78 sp=0xc00178fd40 pc=0x1d0246b
1756269215100 github.com/cilium/tetragon/pkg/grpc/exec.(*MsgExecveEventUnix).HandleMessage(0xc001d6a2f0)
1756269215100 /go/src/github.com/cilium/tetragon/pkg/grpc/exec/exec.go:47 +0x34 fp=0xc00178fd40 sp=0xc00178fb00 pc=0x1d00894
1756269215100 github.com/cilium/tetragon/pkg/grpc/exec.GetProcessExec(0xc001d6a2f0, 0x1)
1756269215100 /go/src/github.com/cilium/tetragon/pkg/process/process.go:520 +0x3e fp=0xc00178fb00 sp=0xc00178fad0 pc=0x12b68be
1756269215100 github.com/cilium/tetragon/pkg/process.AddExecEvent(0xc002f58f00?)
1756269215100 /go/src/github.com/cilium/tetragon/pkg/process/process.go:286 +0x170 fp=0xc00178fad0 sp=0xc00178f730 pc=0x12b46d0
1756269215100 github.com/cilium/tetragon/pkg/process.initProcessInternalExec(0xc00185a000, {0x12270, 0x0, 0xa9d86ded3e962})
1756269215100 /go/src/github.com/cilium/tetragon/pkg/process/process.go:462
1756269215100 github.com/cilium/tetragon/pkg/process.GetPodInfo(...)
1756269215100 /go/src/github.com/cilium/tetragon/pkg/process/podinfo.go:99 +0x496 fp=0xc00178f730 sp=0xc00178f460 pc=0x12b36b6
1756269215100 github.com/cilium/tetragon/pkg/process.getPodInfo({0x2875148?, 0xc000d1dce0?}, {0xc002f594af, 0x1f}, {0xc004a842a0, 0xd}, {0xc004a843b0, 0x7}, 0x24)
1756269215099 /go/src/github.com/cilium/tetragon/pkg/podhelpers/workload.go:54 +0x211 fp=0xc00178f460 sp=0xc00178f300 pc=0x1039b91
1756269215099 github.com/cilium/tetragon/pkg/podhelpers.GetWorkloadMetaFromPod(0xc0016d5208)
1756269215099 /usr/local/go/src/internal/runtime/maps/runtime_faststr_swiss.go:115 +0x110 fp=0xc00178f300 sp=0xc00178f280 pc=0x40ce90
1756269215099 runtime.mapaccess1_faststr(0x1?, 0x0?, {0x24bba1d?, 0xc0044b8080?})
1756269215099 /usr/local/go/src/runtime/panic.go:1058 +0x18 fp=0xc00178f280 sp=0xc00178f260 pc=0x479a58
1756269215099 internal/runtime/maps.fatal({0x24e437c?, 0xc00178f450?})
1756269215099 /usr/local/go/src/runtime/panic.go:1123 +0x5c fp=0xc00178f260 sp=0xc00178f230 pc=0x4413fc
1756269215099 runtime.fatal({0x24e437c, 0x21})
1756269215099 goroutine 384 gp=0xc004c82700 m=66 mp=0xc001281808 [running]:
1756269215099
1756269215096 fatal error: concurrent map read and map writeAnything else?
No response