Retries with got

[Igor-Y75]

New got user here.
This is a simple script to test the retry functionality of the got library.
The code works and it retries if you add an invalid url (I'm using an invalid url for test), but it won't stop trying after the set limit (5 attemps).
I followed the documentation on GitHub; is the problem related on the status code returned by the server (error message: Bad response: 502)? If that were the case, if you try with another valid url (I don't have anyone) or simple disconnect your pc from the network (it throws the error connect ENETUNREACH) then you get an error that should be valid for the limit purpose (as indicated in the documentation). Where is the error?
Thanks.

import { HttpsProxyAgent, HttpsProxyAgentOptions } from "hpagent";

async function gotDownloader() {
  let proxyParametersSetup: HttpsProxyAgentOptions = {
    keepAlive: true,
    keepAliveMsecs: 1000,
    maxSockets: 256,
    maxFreeSockets: 256,
    scheduling: "lifo",
    proxy: "http://_put_your_proxy_url_here:port",
  };

  const proxyParameters = {
    agent: {
      https: new HttpsProxyAgent({ ...proxyParametersSetup }),
    },
  };

  type RetryCounter = {
    [key: string]: number;
  };
  const retryCounter: RetryCounter = {};

  const url = "https://postman-echo.com/get?foo1=bar1&foo2=bar2";
  // const url = "https://www.www.www"; // wrong url to test
  try {
    let gotIstance = got.extend({
      retry: {
        limit: 5,
        calculateDelay({ error }) {
          let valueToReturn = 0;
          let counter;
          if (!retryCounter[url]) {
            counter = retryCounter[url] = 1;
          } else {
            counter = retryCounter[url]++;
          }
          if (error.code !== "200") {
            console.error(
              `ERROR with code ${error.code} - error message: ${error.message} - parametersForDataDownload: ${url} - counter: ${counter}`
            );
            valueToReturn = 1000;
          } else {
            console.error(
              `NOT 200 ERROR - code ${error.code} - error message: ${error.message} - parametersForDataDownload: ${url} - counter: ${counter}`
            );
            valueToReturn = 0;
          }
          return valueToReturn;
        },
        methods: ["GET"],
      },
    });
    let jsonDataFromUrl = await gotIstance(encodeURI(url), {
      ...proxyParameters,
    });
    return true;
  } catch (error) {
    console.error(`GOT DOWNLOADER ERROR with url: ${url} - Error: ${error}`);
    return false;
  }
}

gotDownloader().then((returnedValue) => {
  if (returnedValue) {
    console.log(`OK-->resolve: ${returnedValue}`);
  } else {
    console.log(`KO-->reject: ${returnedValue}`);
  }
});

[Rychu-Pawel]

I'm not sure but from the documentation I understand that when providing your own calculateDelay implementation it will be retrying as long as it returns number > 0.

[sindresorhus]

When you provide a custom calculateDelay function, you take full control of retry logic. The limit option is only automatically enforced if you check computedValue in your function.

The issue is that if your calculateDelay always returns a positive number, retries will continue indefinitely regardless of the limit. You must check computedValue to respect the default retry rules:

const gotInstance = got.extend({
	retry: {
		limit: 5,
		calculateDelay({ computedValue, error }) {
			// IMPORTANT: Return 0 when default logic says don't retry
			if (computedValue === 0) {
				return 0;
			}
			
			// Your custom delay logic
			return computedValue * 2; // Example: double the delay
		},
		methods: ['GET']
	}
});

Alternatively, use enforceRetryRules: true:

const gotInstance = got.extend({
	retry: {
		limit: 5,
		enforceRetryRules: true, // Enforce rules BEFORE calling calculateDelay
		calculateDelay({ computedValue }) {
			// With enforceRetryRules: true, this is only called when retry is allowed
			return computedValue * 2;
		},
		methods: ['GET']
	}
});

When enforceRetryRules: true, Got automatically enforces limit, methods, statusCodes, and errorCodes before calling your calculateDelay function, so you don't need to manually check computedValue.