Logging: Add interface with default glog implementation#4085
Logging: Add interface with default glog implementation#4085bsardo merged 45 commits intoprebid:masterfrom
Conversation
|
I think we want to add something about the compile time usage of this feature to the README. Mention the default, and how to compile the alternate logger in. |
|
@bsardo mentioned that a similar thing is done with the time package - will take a look |
SyntaxNode
left a comment
There was a problem hiding this comment.
@zhongshixi This is very similar to an idea you wanted to implement. Could you please review this PR instead and share your views?
.semgrep/adapter/package-import.go
Outdated
| "github.com/mitchellh/copystructure" | ||
| // ruleid: package-import-check | ||
| "github.com/golang/glog" | ||
| "github.com/prebid/prebid-server/v3/di" |
There was a problem hiding this comment.
This code tests the semgrep rule. Please review the rule to check if any changes to the semgrep definition is required,
There was a problem hiding this comment.
the semgrep rule seems to be checking that either of glog or copystructure packages is not used in the adapters, so I reverted this change to the package-import.go.
di/di.go
Outdated
| @@ -0,0 +1,8 @@ | |||
| package di | |||
There was a problem hiding this comment.
I prefer if this is called the "logger" package, which is more intuitive. I don't know if we'd want to put all global state items in one package and currently there is only one. For simplicity, I'd like to see the interfaces and providers packages collapsed up to this level under the same "logger" package.
di/interfaces/log.go
Outdated
| @@ -0,0 +1,18 @@ | |||
| package interfaces | |||
|
|
|||
| type ILogger interface { | |||
There was a problem hiding this comment.
Nitpick: The Java / C# naming convention does not apply to Go. It is most appropriate to name this "Logger".
di/providers/log/default.go
Outdated
| "github.com/prebid/prebid-server/v3/di/interfaces" | ||
| ) | ||
|
|
||
| type GlogWrapper struct { |
There was a problem hiding this comment.
Nitpick: GlogLogger would be more direct name.
di/providers/log/alternative.go
Outdated
| @@ -0,0 +1,74 @@ | |||
| //go:build custom_logger | |||
There was a problem hiding this comment.
This project does not currently use build directives to choose features. Instead, we offer a configuration system.
I like the idea of having a global log variable to override. I've seen the same in other Go projects, and it provides support for #3961. I propose to keep this ability, while also providing a config option to switch to slog from glog.
There was a problem hiding this comment.
This build tag is actually not required, it's just one potential mechanism that could be used to swap a dependency in compile time, so can probably removed. Configuration would require shipping both implementations and then have them swapped in runtime depending on the configuration, while compile time would only ship a single dependency. Just to be clear, by configuration - do you mean a host-level config file pbs.yaml / pbs.json right?
|
@justadreamer we'll try to revisit this in a few weeks. Sorry for the delay. |
|
This was discussed at the last engineering subcommittee meeting. The suggestion was to move from a compile time config to a runtime config. |
…tructure of the logger: encapsulate the logger object, call only external functions
# Conflicts: # modules/fiftyonedegrees/devicedetection/device_info_extractor.go
linux019
left a comment
There was a problem hiding this comment.
Consider to use build flags to include only one logging lib instead of bringing up more libs to the project codebase
logger/alternative.go
Outdated
| } | ||
|
|
||
| func (logger *SlogWrapper) Info(args ...any) { | ||
| msg := fmt.Sprint(args...) |
There was a problem hiding this comment.
The issue with this logger that glog or fmt.Sprintf use sync.Pool with buffer reuse but this logger always allocates a string to store message and pass it to the underlying function. At least this package should use sync.Pool to store formatted msg to bypass GC
buf := new(bytes.Buffer) // get it from sync.Pool
msg := fmt.Fprintf(buf, args...) //it takes io.Writer
slog.Info(buf.String()) // better to pass io.Reader instead of the string
// put buf back to pool
There was a problem hiding this comment.
this file will be removed as @bsardo mentioned on the call. build tags were used originally and were discarded by the committee in favor of runtime configuration.
config/config.go
Outdated
| return nil, err | ||
| } | ||
|
|
||
| logger.New(c.Logger.Type, c.Logger.Depth) |
There was a problem hiding this comment.
We should validate Type is a valid string value
There was a problem hiding this comment.
Type validation added
README.md
Outdated
| This can be done compile-time in the `logger` package. | ||
| It contains `Logger` interface definition and `default` and `alternative` implementation. | ||
| The `default` logger implementation based on `github.com/golang/glog` package. | ||
| The `alternative` logger implementation based on `log/slog` package. |
There was a problem hiding this comment.
Delete line since we are proposing removing the alternative logger implementation for now.
config/config.go
Outdated
| type Logger struct { | ||
| // the type of logger: default or alternative | ||
| Type string `mapstructure:"type"` | ||
| Depth *int `mapstructure:"depth"` |
There was a problem hiding this comment.
I'm curious why this is a pointer? I imagine you want to detect presence for some reason. Is it because you want to know whether to apply some default depth if depth is not specified? What should the default be? Do you think it will dependent on the implementation?
I see you're detecting presence via a nil check and have declared a default depth of 1 in logger.go.
Should we verify that depth is > 0 or >= 0? What happens if depth is some large number? Perhaps validation is needed for the upper bound as well?
There was a problem hiding this comment.
Should we verify that depth is > 0 or >= 0? What happens if depth is some large number? Perhaps validation is needed for the upper bound as well?
Fun fact: glog has no depth validation within itself:
https://github.com/golang/glog/blob/master/glog.go#L432
Re-check source code will show the same.
// If Depth is present, this function calls log from a different depth in the call stack.
// This enables a callee to emit logs that use the callsite information of its caller
// or any other callers in the stack. When depth == 0, the original callee's line
// information is emitted. When depth > 0, depth frames are skipped in the call stack
// and the final frame is treated like the original callee to Info.
I've tested with different values.
What I've got:
- -1000
glog.ErrorDepth(-1000, "Hello World")
Log:
extern.go:304] Hello World
- 0
glog.ErrorDepth(0, "Hello World")
Log:
main.go:8] Hello World
- 1000
glog.ErrorDepth(0, "Hello World")
Log:
???:1] Hello World
There was a problem hiding this comment.
I don't know in what range to validate the value, because glog doesn't have such validation, and the current implementation doesn't have any alternatives for it now.
logger/default.go
Outdated
| glog.FatalDepthf(logger.depth, format, args...) | ||
| } | ||
|
|
||
| func ProvideDefaultLogger(depth int) Logger { |
There was a problem hiding this comment.
Nitpick: maybe call this NewDefaultLogger to follow patterns throughout the code base?
logger/interface.go
Outdated
| Warning(args ...any) | ||
| Warningf(format string, args ...any) | ||
| Warningln(args ...any) |
There was a problem hiding this comment.
Do we need Warningln? The default implementation is the same as Warning.
logger/alternative.go
Outdated
There was a problem hiding this comment.
Let's delete this file. We can add an slog implementation in the future. For now, simply defining the logger interface and the glog implementation suffices.
In the future we will expand the interface to include functions that are more suitable for structured logging .
# Conflicts: # stored_requests/backends/http_fetcher/fetcher.go # stored_requests/config/config.go
…with thread-safe operations; introduced extensive unit tests for the new logger structure.
config/structlog.go
Outdated
| logInternal.Error("LogStruct called on type %s, whuch is not a struct!", v.Type().String()) | ||
| logger("LogStruct called on type %s, whuch is not a struct!", v.Type().String()) | ||
| os.Exit(1) |
There was a problem hiding this comment.
Let's add a Fatalf instead of calling os.Exit(1) here.
…ated the interface implementation; Updated logger calls with a new signature
bsardo
left a comment
There was a problem hiding this comment.
This is looking good, just a couple of quick comments.
config/structlog.go
Outdated
| func logStructWithLogger(v reflect.Value, prefix string, logger logMsg) { | ||
| if v.Kind() != reflect.Struct { | ||
| glog.Fatalf("LogStruct called on type %s, whuch is not a struct!", v.Type().String()) | ||
| logger("LogStruct called on type %s, whuch is not a struct!", v.Type().String()) |
There was a problem hiding this comment.
Is this line needed? Did you determine this was just missing so you decided to add it?
There was a problem hiding this comment.
I replaced the glog call with our internal logger
There was a problem hiding this comment.
Yes but I see two logger calls in sequence now
logger("LogStruct called on type %s, whuch is not a struct!", v.Type().String())
logInternal.Fatalf("LogStruct called on type %s, whuch is not a struct!", v.Type().String())
I think we can delete logger("LogStruct called on type %s, whuch is not a struct!", v.Type().String()) and just have the fatal call.
config/structlog.go
Outdated
| func logMapWithLogger(v reflect.Value, prefix string, logger logMsg) { | ||
| if v.Kind() != reflect.Map { | ||
| glog.Fatalf("LogMap called on type %s, whuch is not a map!", v.Type().String()) | ||
| logger("LogMap called on type %s, whuch is not a map!", v.Type().String()) |
There was a problem hiding this comment.
Is this line needed? Did you determine this was just missing so you decided to add it?
There was a problem hiding this comment.
I replaced the glog call with our internal logger
main.go
Outdated
|
|
||
| func main() { | ||
| flag.Parse() // required for glog flags and testing package flags | ||
| flag.Parse() // required for di.Log flags and testing package flags |
There was a problem hiding this comment.
What is di? Is this left over from a previous iteration of this PR?
There was a problem hiding this comment.
Yes, you are right, this is a leftover from the previous implementation and is no longer relevant. Removed.
| }, "Debug with args should not panic") | ||
| } | ||
|
|
||
| func TestGlogLogger_Info(t *testing.T) { |
There was a problem hiding this comment.
Can we add a test for Fatal?
There was a problem hiding this comment.
that can be tricky - but I've seen/written similar in other libraries that exit with a setting for whether fatal uses os.Exit or panic. If set to panic for testing, that can be easily caught and verified.
There was a problem hiding this comment.
Fatal test added
| assert.True(t, ok, "Default logger should be *GlogLogger") | ||
| } | ||
|
|
||
| func TestDebug(t *testing.T) { |
There was a problem hiding this comment.
Fatal test added
|
|
||
| func NewGlogLogger() Logger { | ||
| return &GlogLogger{ | ||
| depth: 1, |
There was a problem hiding this comment.
question (not-blocking), should there be a way to set the depth? Would that be an interface on the logger or in the New method itself… Would we want to consider the Functional Options Pattern or a single config object that could be passed that knows to choose default values for any "zero" (uninitialized) fields?
There was a problem hiding this comment.
Judging by the implementation phases, in the current phase 1
PHASE 1:
interface with just the nonstructured logging functions (call it Error, Warning, Info, etc)
swap out all calls to glog library with the nonstructured logging function calls from the glog concrete implementation
we are not implementing the logger settings at this stage.
Judging by the phase 2 description
PHASE 2 (should span multiple smaller PRs):
host config option to choose logger type (just glog or slog, glog is the default)
expand interface to include structured logging functions (ErrorS, WarningS, InfoS etc)
add slog concrete implementation that implements the interface structured logging functions
add shim so nonstructured calls when slog is enabled are converted to structured logging and vice versa
slog conversion to text just sprintf msg
glog conversion to structured (TBD)
going forward, authors can either call the glog or the slog function
the settings will be implemented during the implementation of phase 2
in fact, in phase 2 we can decide which parameters we can specify and which methods will be added for them.
But you are absolutely right, this method will be needed in the future for full configuration of the logger.
| }, "Debug with args should not panic") | ||
| } | ||
|
|
||
| func TestGlogLogger_Info(t *testing.T) { |
There was a problem hiding this comment.
that can be tricky - but I've seen/written similar in other libraries that exit with a setting for whether fatal uses os.Exit or panic. If set to panic for testing, that can be easily caught and verified.
logger/logger_test.go
Outdated
| <-done | ||
| } | ||
|
|
||
| assert.Equal(t, 10, len(mock.infoCalls), "Should have 10 info calls from concurrent execution") |
There was a problem hiding this comment.
Is this test flaky or is something wrong? The test is failing in the the validate github action workflow.
There was a problem hiding this comment.
I deleted this test because it runs correctly locally.
|
The action also crashes due to problems with the publishonly.yml file. |
| } | ||
|
|
||
| func main() { | ||
| flag.Parse() // required for glog flags and testing package flags |
There was a problem hiding this comment.
This is still needed so other flags may be passed in at startup.
There was a problem hiding this comment.
Reverted the changes back
…ithLogger, replacing them with Fatalf for stricter error handling
Co-authored-by: postindustria-code <oss@postindustria.com>
* Dianomi: Update user syncs to send gdpr_consent (prebid#4345) * MobileFuse: Remove tagid_src and pub_id params (prebid#4303) * Remove tagid_src ext from MobileFuse Adapter * remove pub_id query param * removed comment * Simplify Endpoint assignment in MobileFuseAdapter * New Adapter: Flatads (prebid#4243) Co-authored-by: wuzhijian <wuch1k1n@qq.com> * Rubicon: Add bid meta seat (prebid#4348) * Gothamads: Add Intenze alias (prebid#4319) * GumGum: Collect ad unit name for reporting (prebid#4302) * Seedtag: Fix required content-type header on http calls (prebid#4299) * Kobler: Remove sensitive device and user data (prebid#4295) * New Adapter: Nexx360 (prebid#4286) Co-authored-by: Gabriel Chicoye <gabriel@macbookrogab24g.lan> * HTTP Fetcher: Implement RFC 3986 compatibility as optional (prebid#4272) * Targeting: Add configurable targeting prefix (prebid#4355) * Pubmatic: Set bid.meta.mediaType=video when bid.ext.ibv=true (prebid#4189) * Rules Engine Module: Phase 1 (prebid#4407) Co-authored-by: Veronika Solovei <kalypsonika@gmail.com> Co-authored-by: guscarreon <guscarreon@gmail.com> * New Adapter: Sparteo (prebid#4275) * PubMatic: Fix missing bid type in bid response (prebid#4417) * Fix: Remove module per request JSON marshal and add custom viper decoder (prebid#4422) * RichAudience: change endpoint to https (prebid#4384) Co-authored-by: IAN <ian.musachs@exte.com> * Missena: Pass full openrtb request (prebid#4394) * New Adapter: Rediads (prebid#4233) * Zeta Global SSP: Declare OpenRTB 2.6 and GPP support (prebid#4389) * Record metric unsuccessful for PBC non-200 response status code or error (prebid#4341) Co-authored-by: oleksandr <oleksandr@assertive.ai> * New Adapter: Akcelo (prebid#4237) * New Adapter: Zentotem (prebid#4053) * New Adapter: Exco (prebid#4250) * Floors: Fix panic while getting adunit code for signaling (prebid#4424) * Stroeercore: Add adomain to bids (prebid#4392) * VIS.X: Relay bid currency from bid response (prebid#4381) * TMax: Add default host config (prebid#4430) * Fix the package (iterators) to agree with the dir (iterutil). (prebid#4447) * TheTradeDesk: Throw error for malformed endpoint url (prebid#4419) * Smartadserver : Send multi-impression requests without flattening (prebid#4402) Co-authored-by: nlesommer <nlesommer@equativ.com> Co-authored-by: gtodeschi <gregoire.todeschi@gmail.com> * RTB House: Remove PAAPI signals from imp[].ext (prebid#4399) * GumGum: Enable Opt-In change notification (prebid#4390) * Tappx: Add GPID support (prebid#4438) Co-authored-by: Jordi Arnau <jarnau@tappx.com> * New Adapter: progx - Vidazoo alias (prebid#4428) * New Adapter: Kuantyx - Aso alias (prebid#4420) * MinuteMedia: Add test endpoint (prebid#4425) * Adagio: Add site bidder param and web inventory tests (prebid#4403) Co-authored-by: Godefroi Roussel <groussel@adagio.io> * Modularity: Add hook code to module invocation context (prebid#4036) * Rubicon: Pass ext.prebid.multibid[].maxbids to XAPI (prebid#4412) * OpenX: Add gpp params to user sync (prebid#4445) Co-authored-by: gmiedlar-ox <gabriela.miedlar@openx.com> * New Adapter: Blis (prebid#4304) Co-authored-by: Tomas Koutny <tomas@blis.com> * Ogury: Support gpp for cookie sync (prebid#4406) * TheTradeDesk: Resolve AUCTION_PRICE macro (prebid#4448) * Fix: Set account http fetcher defaults enabling env vars (prebid#4460) * Bidder throttling code to increase network stability. (prebid#4415) * Adding request throttling to bidder requests * Add latency health * Adds new config options, metrics, and fixes * Set default window to 1000 and fix error type typo * Addresses PR comments * More PR feedback * Proper handling of the atomic uint in tests * Yieldlab: Forward ADomain to OpenRTB bid (prebid#4404) * Yandex: Add video support (prebid#4344) * Request: Add hb_env=amp targeting to AMP responses (prebid#4414) * Pubmatic: Forward skadn object in bid request (prebid#4453) * Adds missing defaults to network throttling code (prebid#4477) * New Adapter: Tagoras - Vidazoo alias (prebid#4329) Co-authored-by: anna-y-perion <annay@perion.com> * New Adapter: Omnidex - Vidazoo alias (prebid#4441) * Define An Adapter As White Label Only (prebid#4461) * Rules Engine Module: Result functions fix (prebid#4451) * Account: Support bid rounding options (prebid#4470) * Sovrn: Fix passing through imp.ext.gpid (prebid#4413) * Smartadserver: Add second endpoint for programmatic guaranteed (prebid#4467) Co-authored-by: Guillaume Laubier <glaubier@equativ.com> * sspBC: Updates moving operations to backend service (prebid#4351) * New Adapter: Adipolo - Xeworks alias (prebid#4350) Co-authored-by: And-Rud <andrii@xe.works> Co-authored-by: rbstdev <devgit204@xe.works> * Modules: Exitpoint Stage (prebid#4435) * New Adapter: Afront (prebid#4380) * Harden HTTP Response Body Handling (prebid#4489) * New Adapter: TeqBlaze (prebid#4480) * Onetag: imp.ext read generalization (prebid#4446) Co-authored-by: lorenzob <l.baglini@onetag.com> * Rules Engine Module: Configurable cache update frequency (prebid#4423) Co-authored-by: VeronikaSolovei9 <vsolovei@microsoft.com> * New Adapter: RocketLab (prebid#4383) * change zmaticoo.yaml endpoint (prebid#4471) Co-authored-by: Quentin.zuo <quentin.zuo@eclicktech.com.cn> * Metrics: Record GVL fetch count (prebid#4500) * TheTradeDesk: Resolve burl if not resolved (prebid#4481) * New Adapter: pinkLion (prebid#4376) * Metrics: Record HTTP connections wanted and obtained (prebid#4518) * Metrics: Record incoming request sizes (prebid#4495) * Modules: Execute mutations in the order they are listed (prebid#4279) * 4278 Execute mutations in the sequence they are listed. * Ensure that the rejected channel is closed (once) on reject * Use stack var. * Add comments. * Ensure rejected is closed. * Just return explicitly in all 3 cases. * Keep the same style - of passing the params to the go routine. * Revert "Modules: Execute mutations in the order they are listed (prebid#4279)" (prebid#4529) This reverts commit 93c17c2. * Revert "Metrics: Record HTTP connections wanted and obtained (prebid#4518)" (prebid#4538) * New Adapter: Nativery (prebid#4321) Co-authored-by: Andrea Fassina <fasenderos@gmail.com> Co-authored-by: nicoladellamorte <nicola.dellamorte@nativery.com> * E-Planning: Add support for adomain (prebid#4472) * SmileWanted: Append zoneId to endpoint path (prebid#4468) Co-authored-by: QuentinGallard <quentin.gallard@digitalnolimit.com> * Add configurable dialer timeouts to HTTP clients (prebid#4511) * New Adapter: 360playvid (prebid#4395) * New Adapter: MediaYo (prebid#4391) * 33across: Make zoneId the preferred option (prebid#4531) * 33across: Remove deprecated video.placement (prebid#4530) * New Adapter: RobustApps - Xeworks alias (prebid#4434) Co-authored-by: And-Rud <andrii@xe.works> Co-authored-by: rbstdev <devgit204@xe.works> * Adagio: Remove max length on params (prebid#4524) Co-authored-by: Godefroi Roussel <groussel@adagio.io> * fwssp: Add iframe usersync (prebid#4487) Co-authored-by: wenyuanzhang <wenyuanzhang@freewheel.tv> * Module: Scope3 Real-Time Data (prebid#4397) * Add Scope3 Real-Time Data (RTD) module This module integrates Scope3's Real-Time Data API to provide audience segments for targeting in Prebid Server auctions. Features: - Fetches real-time audience segments from Scope3 API - Adds targeting data to bid requests via hooks system - Thread-safe segment storage during auction lifecycle - Configurable timeout and endpoint settings - Graceful error handling that doesn't fail auctions The module implements three hook stages: - Entrypoint: Initialize module context - Raw Auction Request: Fetch segments from Scope3 API - Processed Auction Request: Add segments to targeting data 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com> * Add LiveRamp ATS integration and execution order documentation - Document proper execution order when using with LiveRamp ATS - Add user identifier detection for RampID integration - Include configuration examples for sequential module execution - Enhance API requests with available user identifiers - Add comprehensive documentation for Yahoo deployment scenario 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com> * Add LiveRamp ATS envelope support for publishers without sidecar - Support forwarding encrypted ATS envelopes directly to Scope3 API - Check multiple envelope locations: user.ext.liveramp_idl, user.ext.ats_envelope, ext.liveramp_idl - Prioritize sidecar RampID over envelope when both available - Document both sidecar and envelope integration patterns - Add note about Scope3 needing LiveRamp partner authorization This enables publishers without LiveRamp sidecar to still benefit from LiveRamp ATS user signals via encrypted envelope forwarding. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com> * Clean up debug code and finalize production-ready Scope3 RTD module - Remove all debug logging statements - Streamline segment storage and retrieval between hooks - Finalize request-level targeting for GAM integration - Production-ready code with proper error handling - Complete documentation with configuration examples The module is now ready for production deployment with: - Successful Scope3 API integration - LiveRamp ATS compatibility (sidecar and envelope) - GAM targeting data output - Thread-safe segment management 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com> * Simplify targeting output to single GAM-compatible format - Remove duplicate data.scope3_segments array format - Keep only targeting.hb_scope3_segments as comma-separated string - Follows standard header bidding targeting key conventions - Optimized for GAM key-value targeting integration 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com> * Add unit tests for Scope3 RTD module - Add basic unit tests for module builder and hook functions - Test invalid config handling and error cases - Test entrypoint hook initialization - Test processed auction hook with no segments - Satisfy CI requirements for test coverage 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com> * Fix Go formatting issues - Remove trailing whitespace in module.go - Add missing newline at end of module_test.go - Satisfy gofmt validation requirements 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com> * Address PR feedback: Add caching, improve LiveRamp integration, enhance configurability Key improvements based on reviewer feedback: - Add intelligent caching with configurable TTL to handle repeated requests - Set 60-second default cache TTL for frequency cap compatibility - Improve LiveRamp identifier detection across multiple locations - Remove unsubstantiated partnership claims and improve documentation - Add cache_ttl_seconds and bid_meta_data configuration options - Implement MD5-based cache keys from user IDs and site context - Add comprehensive test coverage for new caching functionality - Update documentation to explain targeting vs bid.meta approach - Change default timeout to 1000ms for better API compatibility Addresses concerns about: - Performance with hundreds of identical requests per user session - Flexibility in targeting data output (bid.meta future enhancement noted) - Accurate LiveRamp integration documentation - Proper hook implementation code naming 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com> * Complete PR feedback implementation: Response-level segments with GAM targeting Major changes to address all PR reviewer feedback: **Response Format Changes:** - Move from request targeting to auction response data per reviewer feedback - Change hook stage from processed_auction_request to auction_response - Add segments to response.ext.scope3.segments for publisher control - Add individual GAM targeting keys when add_to_targeting=true (e.g., gmp_eligible=true) **Configuration Updates:** - Rename bid_meta_data to add_to_targeting for clarity - Add comprehensive GAM integration with individual segment keys - Remove incorrect LiveRamp RTD adapter references from README - Update hook configuration examples to use auction_response stage **API Integration Fixes:** - Correct segment parsing to exclude destination field (triplelift.com) - Extract only actual segments from imp[].ext.scope3.segments[] - Maintain working authentication and caching functionality **Enhanced Testing:** - Add comprehensive mock API integration tests - Test both response formats (scope3 + targeting sections) - Test error handling with mock server responses - Apply gofmt formatting to all code **Publisher Benefits:** - Full control over segment usage via response.ext.scope3.segments - Optional automated GAM integration via individual targeting keys - Flexible configuration for different use cases - Maintains caching for high-frequency scenarios Addresses all PR reviewer concerns while providing maximum publisher flexibility. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com> * Address PR feedback: Add optimized HTTP Transport for high-frequency API calls Per reviewer feedback (@gravelg): 'if we're going to be make a lot of calls, we should use a Transport with better defaults' - MaxIdleConns: 100 (increased connection pool) - MaxIdleConnsPerHost: 10 (multiple connections per host) - IdleConnTimeout: 90s (longer connection reuse) - ForceAttemptHTTP2: true (HTTP/2 for better performance) - DisableCompression: false (bandwidth optimization) 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com> * run gofmt * update default endpoint * fix test * address comments * more review comments * address more comments * remove unused enhanceRequestWithUserIDs method * jsonutil, add more tests * Add privacy field masking for Scope3 RTD module Implement comprehensive privacy protection by masking sensitive user data before sending bid requests to the Scope3 API while preserving essential targeting capabilities. Features: - Configurable field masking with privacy-first defaults - Geographic data truncation with configurable precision (default: 2 decimals ~1.1km) - Identity provider filtering with allowlist for preserved EIDs - Always removes: IP addresses, user IDs, demographics, first-party data - Always preserves: device characteristics, country/region, site/app context - Comprehensive test coverage (92.3%) with edge case handling - All linting checks pass with zero issues 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com> * Fix cache key generation for proper per-user caching - Use SHA-256 hashed user.id as fallback when privacy-safe identifiers are unavailable - Maintains per-user cache segmentation for performance while protecting privacy - Privacy-safe identifiers (RampID, LiverampIDL) take priority over hashed user.id - Prevents accidental data leakage by returning nil on masking failures - Add configuration validation for geo precision (max 4 decimal places) - Add comprehensive tests for cache key behavior and configuration validation 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com> * Fix gofmt formatting in module_test.go 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com> * review fixes * more review comments * make it async * address more comments --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Gabriel Gravel <ggravel@scope3.com> * Metrics: Add connection dial metrics by adapter (prebid#4528) * KueezRTB: Remove additionalProperties from schema (prebid#4490) * Kargo: Add email to codepath notification workflow (prebid#4484) Co-authored-by: Nick Llerandi <nick.llerandi@kargo.com> * New Adapter: Contxtful (prebid#4443) Co-authored-by: rufiange <sebastien@rufiange.com> * Modules: Exitpoint Hook typo fix (prebid#4542) * Consumable: Declare geoscope (prebid#4496) * Connatix: Copy entire imp[i].ext (prebid#4521) * Fix: Preload Prometheus adapter connection dial metric labels when enabled (prebid#4543) Co-authored-by: oleksandr <oleksandr@assertive.ai> * Host Option To Use OS Certs (prebid#4536) * CPMStar: imp.ext passthrough (prebid#4450) * MetaX: Add redirect user sync (prebid#4550) Signed-off-by: Kehan Pan <kehan.pan@metaxsoft.com> * New Adapter: Xapads - AdKernel alias (prebid#4534) * Connatix: Add GPP macros (prebid#4525) * OpenX: Return buyer exts in response (prebid#4507) Co-authored-by: Rafal Sieczka <rafal.sieczka@openx.com> * Yandex: Set referer and target-ref to site.page (prebid#4488) * Nextmillennium: New fields and adapter version update (prebid#4486) * Accounts: Add coop cookiesync priority groups (prebid#4561) * Mobkoi: Remove url endpoint override (prebid#4555) * GDPR: Move enforcement processing upstream (prebid#4567) * Actions: Add publish to docker hub workflow (prebid#4558) * Omnidex: Add GVL ID (prebid#4568) * Adagio: Fix typo (prebid#4570) * Improve Digital: Switch to HTTPS (prebid#4559) * Mobkoi: Update endpoint (prebid#4523) * Remove flaky agma test (prebid#4578) * New Adapter: appStockSSP (prebid#4502) Co-authored-by: Kanceliarenko <kostiantyn.k@teqblaze.com> * Rules Engine Module: Dynamic ruleset from YAML geoscopes (prebid#4509) * In order to protect users of prebid-server that don't use CGO, add build flag (prebid#4058) * In order to protect users of prebid-server that don't use CGO, add build flag. * Conditionally configure fiftyonedegrees. * Because the module file is auto-generated, let it include fiftyonedegrees but just explain that it can't be enabled. * include used packages. * fix builder. * Remove unused file. * Add a test for the Builder when CGO is not enabled. * New Adapter: Gravite (prebid#4547) * New Adapter: Goldbach (prebid#4476) * New Adapter: Showheroes (prebid#4533) * New Adapter: Performist - Limelight Alias (prebid#4283) * Connatix: Support GZIP (prebid#4575) * RTBHouse: PMP removal, publisherId parameter extraction (prebid#4564) * Adnuntius: Pass targeting to ad server (prebid#4545) * New Adapter: Boldwin Rapid (prebid#4478) * Scope3: Various fixes to RTD module (prebid#4544) * PubMatic: Pass alias bidder name through (prebid#4588) * BidTheatre: Update endpoint and add redirect user sync (prebid#4562) * CWire: Add userSync (prebid#4516) * New Adapter: Clydo (prebid#4535) * New Adapter: Microsoft (msft) (prebid#4592) * Syncer: Allow Duplicate Syncer Key For Identical Config (prebid#4622) * Adagio: Enable site capability (prebid#4590) * Adagio: Remove hardcoded seat name (prebid#4596) * Warn in auction response for bidders blocked by privacy settings (prebid#4537) Co-authored-by: oleksandr <oleksandr@assertive.ai> * Logging: Add interface with default glog implementation (prebid#4085) Co-authored-by: postindustria-code <oss@postindustria.com> * msp 25-12 --------- Signed-off-by: Kehan Pan <kehan.pan@metaxsoft.com> Co-authored-by: Michael Stevens <michael.stevens@dianomi.com> Co-authored-by: tomaszbmf <142428312+tomaszbmf@users.noreply.github.com> Co-authored-by: FlatAds <83808812+flatads@users.noreply.github.com> Co-authored-by: wuzhijian <wuch1k1n@qq.com> Co-authored-by: Anton Babak <76536883+AntoxaAntoxic@users.noreply.github.com> Co-authored-by: support-gotham-ads <support@gothamads.com> Co-authored-by: ShayanK16GumGum <shayan.khan@ic.gumgum.com> Co-authored-by: sangarbe <sangarbe@gmail.com> Co-authored-by: Tommy Pettersen <42890605+TommyHPettersen@users.noreply.github.com> Co-authored-by: Gabriel Chicoye <gabriel.chicoye@gmail.com> Co-authored-by: Gabriel Chicoye <gabriel@macbookrogab24g.lan> Co-authored-by: Sigma Software <prebid.opensource@sigma.software> Co-authored-by: pm-priyanka-bagade <156899734+pm-priyanka-bagade@users.noreply.github.com> Co-authored-by: Brian Sardo <1168933+bsardo@users.noreply.github.com> Co-authored-by: Veronika Solovei <kalypsonika@gmail.com> Co-authored-by: guscarreon <guscarreon@gmail.com> Co-authored-by: t-sormonte <t.sormonte@sparteo.com> Co-authored-by: Isha Bharti <isha.bharti@pubmatic.com> Co-authored-by: Rich Audience <sergi.gimenez@richaudience.com> Co-authored-by: IAN <ian.musachs@exte.com> Co-authored-by: Jean-Sébastien Ney <jeansebastien.ney@gmail.com> Co-authored-by: rediads <123890182+rediads@users.noreply.github.com> Co-authored-by: abermanov-zeta <95416296+abermanov-zeta@users.noreply.github.com> Co-authored-by: linux019 <anode.dev@gmail.com> Co-authored-by: oleksandr <oleksandr@assertive.ai> Co-authored-by: Roger <104763658+rogerDyl@users.noreply.github.com> Co-authored-by: zentotem <programmatic@zentotem.net> Co-authored-by: Pete <petropo@ex.co> Co-authored-by: Nikhil Vaidya <102963966+pm-nikhil-vaidya@users.noreply.github.com> Co-authored-by: Philip Watson <philip.watson@adscale.co.nz> Co-authored-by: Vladimir Fedoseev <vl.fedoseev@gmail.com> Co-authored-by: Sheridan C Rawlins <41922797+scr-oath@users.noreply.github.com> Co-authored-by: andre-gielow-ttd <124626380+andre-gielow-ttd@users.noreply.github.com> Co-authored-by: Nathan Le Sommer <lesommer.nathan@gmail.com> Co-authored-by: nlesommer <nlesommer@equativ.com> Co-authored-by: gtodeschi <gregoire.todeschi@gmail.com> Co-authored-by: Piotr Jaworski <109736938+piotrj-rtbh@users.noreply.github.com> Co-authored-by: sindhuja-sridharan <148382298+sindhuja-sridharan@users.noreply.github.com> Co-authored-by: jordi-tappx <jarnau@techsoulogy.com> Co-authored-by: Jordi Arnau <jarnau@tappx.com> Co-authored-by: Anna Yablonsky <annay@perion.com> Co-authored-by: Adserver.Online <61009237+adserver-online@users.noreply.github.com> Co-authored-by: inna <innayare@gmail.com> Co-authored-by: GodefroiRoussel <GodefroiRoussel@users.noreply.github.com> Co-authored-by: Godefroi Roussel <groussel@adagio.io> Co-authored-by: Kacper Fus <kacper.fus@openx.com> Co-authored-by: gmiedlar-ox <gabriela.miedlar@openx.com> Co-authored-by: tomaskoutny-blis <60094686+tomaskoutny-blis@users.noreply.github.com> Co-authored-by: Tomas Koutny <tomas@blis.com> Co-authored-by: Vincent Bachelier <vincent@ogury.co> Co-authored-by: hhhjort <31041505+hhhjort@users.noreply.github.com> Co-authored-by: yuu.t <tongyu24@hotmail.com> Co-authored-by: Dmitry Ermakov <skfyann@gmail.com> Co-authored-by: Saar Amrani <saar120@gmail.com> Co-authored-by: Scott Kay <noreply@syntaxnode.com> Co-authored-by: Sandy Janicki <sejanick@users.noreply.github.com> Co-authored-by: Guillaume Laubier <guillaume.laubier@hotmail.fr> Co-authored-by: Guillaume Laubier <glaubier@equativ.com> Co-authored-by: mabielinski-wpm <155963187+mabielinski-wpm@users.noreply.github.com> Co-authored-by: Andrii Rudyk <96918101+And-Rud@users.noreply.github.com> Co-authored-by: And-Rud <andrii@xe.works> Co-authored-by: rbstdev <devgit204@xe.works> Co-authored-by: Afrontio <support@afront.io> Co-authored-by: MaksymTeqBlaze <maksym.pavliv@teqblaze.com> Co-authored-by: esimonelli@onetag <136568337+EmanueleSimonelli@users.noreply.github.com> Co-authored-by: lorenzob <l.baglini@onetag.com> Co-authored-by: VeronikaSolovei9 <vsolovei@microsoft.com> Co-authored-by: Leandro Marty <leandro.marty@rocketlab.ai> Co-authored-by: g-coder-baiai <77920802+g-coder-baiai@users.noreply.github.com> Co-authored-by: Quentin.zuo <quentin.zuo@eclicktech.com.cn> Co-authored-by: prebidPinkLion <prebid@pinklion.io> Co-authored-by: andreafassina <127768714+andreafassina@users.noreply.github.com> Co-authored-by: Andrea Fassina <fasenderos@gmail.com> Co-authored-by: nicoladellamorte <nicola.dellamorte@nativery.com> Co-authored-by: Agustin Insua <Nistenf@users.noreply.github.com> Co-authored-by: Quentin Gallard <QuentinGallard@users.noreply.github.com> Co-authored-by: QuentinGallard <quentin.gallard@digitalnolimit.com> Co-authored-by: pb360playvid <prebid@360playvid.com> Co-authored-by: mediayo <yehuda@mediayo.co> Co-authored-by: Carlos Felix <carloshto@gmail.com> Co-authored-by: rbstdev <devgit204@gmail.com> Co-authored-by: Wenyuan Zhang <wzhang910@cable.comcast.com> Co-authored-by: wenyuanzhang <wenyuanzhang@freewheel.tv> Co-authored-by: Brian O'Kelley <bokelley@scope3.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Gabriel Gravel <ggravel@scope3.com> Co-authored-by: Amit Biton <56631148+amitbiton01@users.noreply.github.com> Co-authored-by: Shane Lacey <shanel262@hotmail.com> Co-authored-by: Nick Llerandi <nick.llerandi@kargo.com> Co-authored-by: Sebastien Boisvert <sebastien.boisvert@contxtful.com> Co-authored-by: rufiange <sebastien@rufiange.com> Co-authored-by: richardngo-consumable <rngo@consumable.com> Co-authored-by: Karim Mourra <karim@jwplayer.com> Co-authored-by: Joshua Tomlinson <og.ginger@gmail.com> Co-authored-by: metax-kehan <115962296+metax-kehan@users.noreply.github.com> Co-authored-by: Denis Logachov <ckbo3hrk@gmail.com> Co-authored-by: Rafal Sieczka <rafal.sieczka@openx.com> Co-authored-by: Dmitry Ermakov <dimurer@yandex-team.ru> Co-authored-by: a.popov <60257866+allar15@users.noreply.github.com> Co-authored-by: Marc-Enzo Bonnafon <marcenzo.bonnafon@gmail.com> Co-authored-by: Olivier <osazos@adagio.io> Co-authored-by: Jozef Bartek <31618107+jbartek25@users.noreply.github.com> Co-authored-by: Appstock LTD <sdksupport@app-stock.com> Co-authored-by: Kanceliarenko <kostiantyn.k@teqblaze.com> Co-authored-by: teqblaze <162988436+teqblaze@users.noreply.github.com> Co-authored-by: Simon Aebli <simon.aebli@goldbach.com> Co-authored-by: Filip Stamenkovic <ficadub@gmail.com> Co-authored-by: quietPusher <129727954+quietPusher@users.noreply.github.com> Co-authored-by: Mikael Lundin <mikael-lundin@users.noreply.github.com> Co-authored-by: BoldwinDev <sdk@bold-win.com> Co-authored-by: Gabriel Gravel <gravelg@users.noreply.github.com> Co-authored-by: Pubmatic-Supriya-Patil <131644110+Pubmatic-Supriya-Patil@users.noreply.github.com> Co-authored-by: andreasgreen <andreas.green@gmail.com> Co-authored-by: Alessandro Rosetti <alessandro.rosetti@gmail.com> Co-authored-by: cto-clydo <cto@clydo.io> Co-authored-by: Eugene Dorfman <eugene.dorfman@gmail.com> Co-authored-by: postindustria-code <oss@postindustria.com>
* RTBHouse: PMP removal, publisherId parameter extraction (prebid#4564) * Adnuntius: Pass targeting to ad server (prebid#4545) * New Adapter: Boldwin Rapid (prebid#4478) * Scope3: Various fixes to RTD module (prebid#4544) * PubMatic: Pass alias bidder name through (prebid#4588) * BidTheatre: Update endpoint and add redirect user sync (prebid#4562) * CWire: Add userSync (prebid#4516) * New Adapter: Clydo (prebid#4535) * New Adapter: Microsoft (msft) (prebid#4592) * Syncer: Allow Duplicate Syncer Key For Identical Config (prebid#4622) * Adagio: Enable site capability (prebid#4590) * Adagio: Remove hardcoded seat name (prebid#4596) * Warn in auction response for bidders blocked by privacy settings (prebid#4537) Co-authored-by: oleksandr <oleksandr@assertive.ai> * Logging: Add interface with default glog implementation (prebid#4085) Co-authored-by: postindustria-code <oss@postindustria.com> * Post Upgrade Patch: Fix Test Cases * Add Builder for Microsoft Bidder --------- Co-authored-by: Piotr Jaworski <109736938+piotrj-rtbh@users.noreply.github.com> Co-authored-by: Mikael Lundin <mikael-lundin@users.noreply.github.com> Co-authored-by: BoldwinDev <sdk@bold-win.com> Co-authored-by: Gabriel Gravel <gravelg@users.noreply.github.com> Co-authored-by: Pubmatic-Supriya-Patil <131644110+Pubmatic-Supriya-Patil@users.noreply.github.com> Co-authored-by: andreasgreen <andreas.green@gmail.com> Co-authored-by: Alessandro Rosetti <alessandro.rosetti@gmail.com> Co-authored-by: cto-clydo <cto@clydo.io> Co-authored-by: Scott Kay <noreply@syntaxnode.com> Co-authored-by: Olivier <osazos@adagio.io> Co-authored-by: linux019 <anode.dev@gmail.com> Co-authored-by: oleksandr <oleksandr@assertive.ai> Co-authored-by: Eugene Dorfman <eugene.dorfman@gmail.com> Co-authored-by: postindustria-code <oss@postindustria.com>
Co-authored-by: postindustria-code <oss@postindustria.com>
This PR introduces a
loggershim and was meant to accompany a more global proposal: #4084. With this different loggers can be swapped in compile-time, however leaving the possibility to also swap a global instance in runtime (during tests f.e.)The global object is instantiated in logger/logger.go, the interface is defined in
logger/interface.go, while implementations of loggers are inlogger/default.goandlogger/alternative.go.