-
Notifications
You must be signed in to change notification settings - Fork 1.6k
[service] Share log sampler core allocations with reflect magic #13107
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[service] Share log sampler core allocations with reflect magic #13107
Conversation
Codecov ReportAttention: Patch coverage is
❌ Your patch status has failed because the patch coverage (80.85%) is below the target coverage (95.00%). You can increase the patch coverage or adjust the target coverage. Additional details and impacted files@@ Coverage Diff @@
## main #13107 +/- ##
==========================================
- Coverage 91.48% 91.28% -0.21%
==========================================
Files 506 511 +5
Lines 28555 28802 +247
==========================================
+ Hits 26123 26291 +168
- Misses 1917 1994 +77
- Partials 515 517 +2 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
My comment from today's 1.0 meeting: I think we want to avoid the following situations:
|
|
// https://github.com/uber-go/zap/blob/fcf8ee58669e358bbd6460bef5c2ee7a53c0803a/zapcore/sampler.go#L168 | ||
// We need to create a new Zap sampler core with the same settings but with a new inner core, | ||
// while reusing the very RAM-intensive `counters` data structure. | ||
// The `With` method does something similar, but it's not quite what we want, so we use `reflect`. | ||
// This hack can be removed once Zap supports this use case. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the detailed comment.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jade-guiton-dd would be good if we can file an issue for this on their side and link it here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I filed an issue a while back, added it to the comment.
4ddf2cc
Context
PR #12617, which implemented the injection of component-identifying attributes into the
zap.Logger
provided to components, introduced significant additional memory use when the Collector's pipelines contain many components (#13014). This was because we would callzapcore.NewSamplerWithOptions
to wrap the specialized logger core of each Collector component, which allocates half a megabyte's worth of sampling counters.This problem was mitigated in #13015 by moving the sampling layer to a different location in the logger core hierarchy. This meant that Collector users that do not export their logs through OTLP and only use stdout-based logs no longer saw the memory increase.
Description
This PR aims to provide a better solution to this issue, by using the
reflect
library to clone zap's sampler core and set a new inner core, while reusing the counter allocation.(This may also be "more correct" from a sampling point of view, ie. we only have one global instance of the counters instead of one for console logs and one for each component's OTLP-exported logs, but I'm not sure if anyone noticed the difference anyway).
Link to tracking issue
Fixes #13014
Testing
A new test was added which checks that the log counters are shared between two sampler cores with different attributes.