How to Disable Transfer-Encoding: chunked in openapi-fetch Requests?

[narnau]

Hi,

I'm currently using openapi-fetch in a Next.js action (server-side) to handle my API requests. However, when I make POST or PUT requests, the requests are automatically being sent with Transfer-Encoding: chunked. This is causing issues with my backend (Django), as it doesn't handle chunked requests properly.

Here is an example of the raw request:

POST /api/profile/upsert HTTP/1.1
Host: my-api-host.com
User-Agent: undici
Transfer-Encoding: chunked
Accept: */*
Accept-Encoding: br, gzip, deflate
Content-Type: application/json

29f
{
  "id": 112,
  "first_name": "",
  "last_name": "",
  "display_name": "sample-display-name",
  "email": "[email protected]",
  "picture": "https://sample-url.com/picture.png",
  ...
}
0

In this case, the Transfer-Encoding: chunked is automatically applied, even though I'm passing the full payload at once. This behavior seems to be causing issues on the Django backend.

My Question: Is there a way to configure openapi-fetch so that it doesn't use Transfer-Encoding: chunked and sends the request with a Content-Length header instead?

Any guidance on how to resolve this would be greatly appreciated!

Additional Info:

Environment: Running this in a Next.js action (server-side)
Backend: Django, which is having issues with chunked requests
Thanks in advance!

[drwpow]

This is being added by Next.js’ internal serverside fetch API, which operates differently than the spec fetch API. MY guess is that it’s being added after openapi-fetch runs (openapi-fetch basically hands off the request to Next.js), but to confirm you could add openapi-fetch middleware that just logs the Request (middleware is the last thing to touch thr Request in this library).

Beyond that I don’t work with Next.js enough to know how to change this behavior

[narnau]

Hello @drwpow and thank you for your quick assistance!

That sounds strange cause meanwhile I don't resolve this problem with openapi-fetch, I'm using the regular-native fetch and the requests are normal.

This is the code I'm using:

const requestOptions: RequestInit = {
      method,
      headers: {
        'Content-Type': 'application/json',
      },
    };

if (body) {
   requestOptions.body = JSON.stringify(body);
}

try {
   const response = await fetch(url, requestOptions);
}

And the RAW request is not chunked:

POST /api/profile/upsert HTTP/1.1
Host: my-api-host.com
User-Agent: undici
Content-Length: 671
Accept: */*
Accept-Encoding: br, gzip, deflate
Accept-Language: *
Content-Type: application/json
Sec-Fetch-Mode: cors
X-Forwarded-For: ...
X-Forwarded-Host: ...
X-Forwarded-Proto: https

{"id":112,"first_name":"","last_name":"","display_name":"sample-display-name",...}

I added the middleware also to openapi-fetch, and this is what it carries:

CustomRequest [Request] {
  [Symbol(realm)]: {
    settingsObject: { baseUrl: undefined, origin: [Getter], policyContainer: [Object] }
  },
  [Symbol(state)]: {
    method: 'PATCH',
    localURLsOnly: false,
    unsafeRequest: false,
    body: {
      stream: undefined,
      source: '{"first_name":"...","last_name":"....","display_name":"narnau"}',
      length: 82
    },
    client: { baseUrl: undefined, origin: [Getter], policyContainer: [Object] },
    reservedClient: null,
    replacesClientId: '',
    window: 'client',
    keepalive: false,
    serviceWorkers: 'all',
    initiator: '',
    destination: '',
    priority: null,
    origin: 'client',
    policyContainer: 'client',
    referrer: 'client',
    referrerPolicy: '',
    mode: 'cors',
    useCORSPreflightFlag: false,
    credentials: 'same-origin',
    useCredentials: false,
    cache: 'default',
    redirect: 'follow',
    integrity: '',
    cryptoGraphicsNonceMetadata: '',
    parserMetadata: '',
    reloadNavigation: false,
    historyNavigation: false,
    userActivation: false,
    taintedOrigin: false,
    redirectCount: 0,
    responseTainting: 'basic',
    preventNoCacheCacheControlHeaderModification: false,
    done: false,
    timingAllowFailed: false,
    headersList: HeadersList {
      cookies: null,
      [Symbol(headers map)]: [Map],
      [Symbol(headers map sorted)]: null
    },
    urlList: [ [URL] ],
    url: URL {
      href: 'https://4866-37-14-181-10.ngrok-free.app/api/profile/?user_auth0=email%7C....',
      origin: 'https://4866-37-14-181-10.ngrok-free.app',
      protocol: 'https:',
      username: '',
      password: '',
      host: '4866-37-14-181-10.ngrok-free.app',
      hostname: '4866-37-14-181-10.ngrok-free.app',
      port: '',
      pathname: '/api/profile/',
      search: '?user_auth0=email%7C...',
      searchParams: URLSearchParams { 'user_auth0' => 'email|66....' },
      hash: ''
    }
  },
  [Symbol(signal)]: AbortSignal { aborted: false },
  [Symbol(headers)]: HeadersList {
    cookies: null,
    [Symbol(headers map)]: Map(1) { 'content-type' => [Object] },
    [Symbol(headers map sorted)]: null
  }
}
Headers: { 'content-type': 'application/json' }

If the problem is related with NextJS, I don't understand why it does only fail when I'm using the library :/
Do you have any clue?

Thank you

[drwpow-figma]

Complete shot in the dark, but the fetch spec has 2 versions:

fetch(resource)           // ex: fetch(new Request(…))
fetch(resource, options)  // ex: fetch("https://…", { … })

In practice, across browsers, and Node.js, and libraries that have internal fetchers (which I believe Next.js does), they either implement one, or both versions differently (even Chrome vs Firefox implement both versions differently).

openapi-fetch uses the 1st (single param) API, while many naïve implementations primarily use the 2nd (double param).

I don’t have any answers, but just for debugging purposes I’m curious what Next.js does when handed fetch(new Request(…)) directly (not within openapi-fetch).

[narnau]

Nice shot!

I've implemented it like this:

const response = await fetch(new Request(url, requestOptions));

And...you were right! The request is chunked again!

PATCH /api/profile/?user_auth0=email%7C665d6f1f4d6bdfd03446c0b5 HTTP/1.1
Host: 4866-37-14-181-10.ngrok-free.app
User-Agent: undici
Transfer-Encoding: chunked
Accept: */*
Accept-Encoding: br, gzip, deflate
Accept-Language: *
Content-Type: application/json
Sec-Fetch-Mode: cors
X-Forwarded-For: 37.14.181.10
X-Forwarded-Host: 4866-37-14-181-10.ngrok-free.app
X-Forwarded-Proto: https

52
{"first_name":"...","last_name":"...","display_name":"..."}
0

Now that we know what is happening, can you suggest me any solution to use openapi-fetch without chunked requests? :)

Thank you @drwpow @drwpow-figma

[narnau]

Hi again @drwpow @drwpow-figma !

Now that we know what is happening, can you suggest me any solution to use openapi-fetch without chunked requests? :)
I saw that there's an option in the createClient

fetch | fetch | Fetch instance used for requests (default: globalThis.fetch)
But I'm not sure what can I do to force using the "right one" (ex: fetch("https://…").

Can someone help me?

Thank you,

[pannkotsky]

@narnau I have the same stack as you (Nextjs, Django/DRF API, openapi-fetch), and the following configurations seems to do the trick for me:

export const client = createClient<paths>({
  baseUrl: process.env.NEXT_PUBLIC_API_ROOT,
  headers: {
    Accept: 'application/json',
    'Content-Type': 'application/json',
  },
  // Override the default Request constructor usage
  Request: undefined,
  // Override the default fetch usage. Using signature `fetch(new Request(...))`
  // causes `Transfer-Encoding: chunked` and no `Content-Length` in request headers
  // which doesn't work with Django/DRF API
  fetch: async (input: Request) => {
    const url = input.url;
    const body = input.body ? await input.text() : undefined;
    const init = {
      method: input.method,
      headers: Object.fromEntries(input.headers.entries()),
      body,
    };
    return fetch(url, init);
  },
});

[pannkotsky]

one note: this only happens to me in production setup for some reason (on real remote servers). locally, even if I use production build of Next.js app and gunicorn for serving Django, the Transfer-Encoding: chunked does not appear in the headers, and Content-Length is properly set