Decompression Interceptor

[FelixVaughan]

This relates to...

Address proposal brought up here
and is related to #4316

Rationale

This PR implements a decompression interceptor for Undici, as discussed in the linked proposal and issue. The goal is to provide automatic response decompression for request() (matching the behavior of fetch()), reducing boilerplate and improving developer experience when working with compressed HTTP responses.

Changes

• Adds a new decompression interceptor for Undici clients
• Supports gzip, deflate, and brotli content-encoding
• Skips decompression for 4xx/5xx, 204, and 304 responses (per spec and browser behavior)
• Removes content-encoding and content-length headers when decompressing
• Streams decompressed data for memory efficiency
• Additional tests for supported encodings and edge cases

Features

• Automatic decompression for gzip, deflate, brotli, and zstd responses
• Transparent header cleanup
• Skips decompression for error and no-content responses
• Streaming implementation for large responses
• Case-insensitive encoding support
• Pass-through for unsupported encodings

Status

• I have read and agreed to the Developer's Certificate of Origin
• Tested
• Benchmarked (optional)
• Documented
• Review ready
• In review
• Merge ready

[metcoder95]

Nice job!

I'd just suggest to implement the decompressor in accordance to RFC-9110, in that way decompressor can work with different combination in according to standards.

I believe the only missing part might be the support for multiple encodings on a response

[FelixVaughan]

@metcoder95 Thanks! All the above have been implemented along with documentation. Let me know if anything else is required

[metcoder95]

Nice job! I left few commenets.

Do not forget about documentation and TS types

[metcoder95]

Suggested change

console.log('pause')

[metcoder95]

The whole is looking great, you aligned the the backpressure from each chunk processing of the handler when handling the decompressed response, as well as to align the decompressor stream state with the chunk processing one, tho there's a step still missed.

Within the handler there's a controller that also states the state of the chunk processing with the handler, if the controller get's called with pause, or resume the decompressing stream should react to that. See:

undici/lib/handler/unwrap-handler.js

Lines 20 to 29 in a36e299

	pause () {
	this.#paused = true
	}
	resume () {
	if (this.#paused) {
	this.#paused = false
	this[kResume]?.()
	}
	}

If we can add testing around it will be amazing.

[FelixVaughan]

Right on! What're your thoughts on something like this in the decompressor's onResponseStart?

const superPause = controller.pause.bind(controller)
const superResume = controller.resume.bind(controller)

controller.pause = () => {
  const result = superPause()
  if (this.#decompressors.length > 0) {
    this.#decompressors.forEach(d => {
      if (!d.readableEnded && !d.destroyed) {
        d.pause()
      }
    })
  }
  return result
}

controller.resume = () => {
  const result = superResume()
  if (this.#decompressors.length > 0) {
    this.#decompressors.forEach(d => {
      if (!d.readableEnded && !d.destroyed) {
        d.resume()
      }
    })
  }
  return result
}

Happy to add additional tests, however I'm unsure how to simulate pauses (and backpressure) in the handlers without exposing internals. Open to suggestions!

[metcoder95]

LGTM, although you can just pause the head of the compressors, and the rest will stop processing any more chunks as they will be waiting for the head to continue pass more chunks.

Additionally you can also pause the resulting stream of pipeline if available when the controller gets called (especially as the other streams will continue processing their buffered data).

For the tests, it should be enough by calling client.dispatch and set a custom request handler. As you receive the controller during onRequsestStart, you should have the chance to pause and resume as needed.

[metcoder95]

Pipeline returns a stream which you can subscribe for the chunks of data

[metcoder95]

LGTM, although you can just pause the head of the compressors, and the rest will stop processing any more chunks as they will be waiting for the head to continue pass more chunks.

Additionally you can also pause the resulting stream of pipeline if available when the controller gets called (especially as the other streams will continue processing their buffered data).

For the tests, it should be enough by calling client.dispatch and set a custom request handler. As you receive the controller during onRequsestStart, you should have the chance to pause and resume as needed.