-
Notifications
You must be signed in to change notification settings - Fork 357
feat: Propagate the uprobe link into sample type #824
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
feat: Propagate the uprobe link into sample type #824
Conversation
With open-telemetry#651 it is possible to attach uprobes, but they currently are merged together. This PR introduces the use of cookies (available from kernel 5.15+) to keep track of the links that lead to the events being generated. It now uses the symbol part of the uprobe link and propages it into the sample type. That is something that we currently don't do well enough. So let's look at this example: ``` --uprobe-link /nix/store/r7pnxs3cfl3qxwacj38iakpm5v1ch6lz-glibc-2.40-66/lib/libc.so.6:malloc --uprobe-link /nix/store/r7pnxs3cfl3qxwacj38iakpm5v1ch6lz-glibc-2.40-66/lib/libc.so.6:open --uprobe-link /proc/4501/root/usr/lib/aarch64-linux-gnu/libc.so.6:open --uprobe-link /proc/4501/root/usr/lib/aarch64-linux-gnu/libc.so.6:malloc ``` There would be resulting two sample types - `uprobe_malloc_events:count` - `uprobe_open_events:count`
aeb3d34 to
49c080c
Compare
| SEC("uprobe/generic") | ||
| int uprobe__generic(void *ctx) | ||
| { | ||
| u64 cookie = bpf_get_attach_cookie(ctx); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The minimal supported version is currently 5.4 - see https://github.com/open-telemetry/opentelemetry-ebpf-profiler?tab=readme-ov-file#supported-linux-kernel-version.
This bpf helper function is not available with this kernel version.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
CI will fail, once #826 is merged and this change is rebased.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not super experienced with the bpf tooling, but is should be possible to either target uprobes only for kernels 5.15+ or have a program each with/without cookie helper?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
so far, the complexity is avoided to differentiate such cases. a work around could be to define a global variable, that is loaded at load time of the eBPF program and sets the value to differentiate different attach points.
| u64 ts = bpf_ktime_get_ns(); | ||
|
|
||
| return collect_trace(ctx, TRACE_UPROBE, pid, tid, ts, 0); | ||
| return collect_trace(ctx, TRACE_UPROBE, pid, tid, ts, cookie); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If this goes ahead I would say we need to rename off_cpu_time to off_cpu_time_or_cookie or add an extra parameter.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To follow along this thought: What is the expected field in the OTel Profiling signal to communicate this?
| if origin.ProbeLinkName != "" { | ||
| st.SetTypeStrindex(stringSet.Add(fmt.Sprintf("uprobe_%s_events", origin.ProbeLinkName))) | ||
| } else { | ||
| st.SetTypeStrindex(stringSet.Add("uprobe_events")) | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The TypeStr across the Origins are not consistent right now, I think we lack to follow the spec here:
This feels closer to the spec.
With #651 it is possible to attach uprobes, but their results are currently merged together. This PR introduces the use of cookies (available from kernel 5.15+) to keep track of the links that lead to the events being generated.
It now uses the symbol part of the uprobe link and propages it into the sample type. That is something that we currently don't do well enough.
So let's look at this example:
There would be resulting two sample types
uprobe_malloc_events:countuprobe_open_events:count