Skip to content

feat(metrics): instrument pipeline channels #4836

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

NDStrahilevitz
Copy link
Collaborator

@NDStrahilevitz NDStrahilevitz commented Jul 17, 2025

1. Explain what the PR does

c826cb3 feat(metrics): instrument pipeline channels

c826cb3 feat(metrics): instrument pipeline channels

Add a new ChannelMetrics type to track the size of the channels in the
pipeline. This can be used to detect bottlenecks in the pipeline. The
channels are registered by name in the handleEvents function. When
tracee is healthy the gauges should mostly be 0.

2. Explain how to test it

Start tracee with the --metrics flag
Go to the metrics url (usually localhost:3366/metrics if server is local)
Confirm that the pipeline channels show up at the bottom
Induce high throughput
Refresh a few times and you should occasionally see one of the stages with a number above 0 (under healthy function this is what should happen)

3. Other comments

Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR instruments pipeline channels by adding a generic ChannelMetrics type to track their sizes, registering those metrics with Prometheus, and populating the stats in the event handling pipeline.

  • Introduce ChannelMetrics in pkg/metrics for both Prometheus registration and JSON output
  • Wire up Stats.Channels in pkg/ebpf/events_pipeline.go for each pipeline stage
  • Register channel gauges in Stats.RegisterPrometheus

Reviewed Changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
pkg/metrics/stats.go Added Channels to Stats, initialized in NewStats, and registered in RegisterPrometheus
pkg/metrics/channels.go Implemented ChannelMetrics type with RegisterChannels and MarshalJSON
pkg/ebpf/events_pipeline.go Populated stats.Channels[...] for each pipeline stage
pkg/containers/containers.go Refactored EnrichCgroupInfo for consistent variable naming and map updates
.github/workflows/pr-sync.yaml Installed git, configured safe.directory, and added error checks in PR sync workflow
Comments suppressed due to low confidence (1)

pkg/metrics/channels.go:10

  • Add unit tests for ChannelMetrics.MarshalJSON and RegisterChannels to verify correct JSON serialization and Prometheus metric registration for each channel.
type ChannelMetrics[T any] map[string]<-chan T

@NDStrahilevitz NDStrahilevitz force-pushed the instrumented_channels2 branch 2 times, most recently from e65eb34 to 43912a9 Compare July 17, 2025 13:52
@NDStrahilevitz NDStrahilevitz changed the title Instrumented channels2 feat(metrics): instrument pipeline channels Jul 17, 2025
Add a new ChannelMetrics type to track the size of the channels in the
pipeline. This can be used to detect bottlenecks in the pipeline. The
channels are registered by name in the handleEvents function. When
tracee is healthy the gauges should mostly be 0.
@NDStrahilevitz NDStrahilevitz force-pushed the instrumented_channels2 branch from 43912a9 to c826cb3 Compare July 17, 2025 14:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant