Cache does not decode compressed responses properly

[michael42]

Describe the bug

• Node.js version: v16.16.0
• OS & version: Linux 5.15

We noticed that got returned the wrong response for every second request, when making requests with enabled caching.

Actual behavior

Apparently, when making requests with the cache option, got does not decode the cached response properly (based on the Content-Encoding header of the cached response), but instead uses the Content-Encoding-header of the 304 Not Modified response.

Expected behavior

got should be able to cache and re-use cached response, even if the 304 response is sent with a different Content-Encoding header.

Code to reproduce

import http from 'node:http';
import zlib from 'node:zlib';
import got from 'got';

const gzipData = zlib.gzipSync('test');
const etag = '"asdf"';

const server = http
  .createServer((req, res) => {
    if (req.headers['if-none-match'] === etag) {
      res.writeHead(304); // not, no Content-Encoding header
      res.end();
      return;
    }

    res.writeHead(200, { 'content-encoding': 'gzip', etag });
    res.write(gzipData);
    res.end();
  })
  .listen(2345);

const map = new Map();
for (let i = 0; i < 4; i++) {
  const result = await got.get({
    url: 'http://localhost:2345',
    cache: map,
    responseType: 'text',
  });
  console.log("response", i, result.body.slice(0, 32));
}

server.close();

Checklist

• I have read the documentation.
• I have tried my code with the latest version of Node.js and Got.

[sindresorhus]

I believe the issue is in the cacheable-request dependency, not Got's code.

When a 304 Not Modified response has completely empty headers, cacheable-request fails to preserve the original Content-Encoding header from the cached 200 response. This causes Got to return raw gzipped data instead of decompressing it.

---

// @jaredwray

[jaredwray]

@sindresorhus - thanks so much and I am working on a fix for it. Should have something this week.

[jaredwray]

@sindresorhus, @michael42, and @szmarczak - this bug is actually in http-cache-semantics but we have built a work around to preserve the critical headers. jaredwray/cacheable#1409

This should be out in the next week or so.