incompatible gzip request compression support between Prebid.js and prebid-server

[postindustria-code]

Background

Prebid.js prebidServerAdapter suggests to use endpointCompression param on the s2sConfig that enables the following behavior:

• Content-Type is text/plain, but body is gzip-compressed
• gzip=1 URL param is added
• Content-Encoding is not set to avoid pre-flight request by the browser

Prebid Server has this code:

prebid-server/endpoints/openrtb2/auction.go

Lines 418 to 430 in 20a81a9

	reqContentEncoding := httputil.ContentEncoding(httpRequest.Header.Get("Content-Encoding"))
	if reqContentEncoding != "" {
	if !deps.cfg.Compression.Request.IsSupported(reqContentEncoding) {
	errs = []error{fmt.Errorf("Content-Encoding of type %s is not supported", reqContentEncoding)}
	return
	} else {
	r, err = getCompressionEnabledReader(httpRequest.Body, reqContentEncoding)
	if err != nil {
	errs = []error{err}
	return
	}
	}
	}

that expects Content-Encoding: gzip header on the request. Thus Prebid.js and Prebid Server do not meet in their expectations.

Proposed fix

Change the Prebid Server check to rather watch for gzip=1 param on the request and if set then attempt to decompress the request body. Or to watch for either of Content-Encoding: gzip header (since it is already implemented and somebody may be using it??) or gzip=1 URL param - and then decompress.

Prebid Server Java

It looks like PBS-Java has the same compatibility issue as it is built on Spring Boot with Netty, which typically relies on the Content-Encoding header to detect and decompress gzipped request bodies.

Related

prebid/Prebid.js#12973 here gzip=1 was implemented
#2438 - apparently the Content-Encoding support appeared earlier in PBS than gzip=1 appeared in PB.js

[postindustria-code]

The current workaround is to add Content-Encoding: gzip request header on the gateway s.a. nginx if gzip=1 param was present:

map $arg_gzip $content_encoding {
	default	$http_content_encoding;
	"1"	"gzip";
}
proxy_set_header "Content-Encoding" $content_encoding;

[Net-burst]

Discussed in the Prebid Server committee. Let's look into whether server libraries Go and Java are using can do an automatic detection of the content-encoding. If yes, we should utilize that to make PBS resilient to the possibility of incoming requests being gzipped.

If not, look into how big of a lift it will be to automate the request decoding manually and how big of a performance hit it will be.

Leaving in community review status for now, but the issue is pretty legit so we should prioritize it.

[linux019]

The server shouldn't attempt to predict the content type. It applies the requested decoding algorithm, and ignores requests with unsupported encoding.