Proposal: Retries

[biochimia]

This is a proposal to introduce retry functionality in Cats Effect. The intent here is to focus the discussion on the interface, before moving on to implementation in a dedicated pull request. The ideas in this proposal have been experimented with in a free-standing implementation available at https://github.com/biochimia/scala-retry.

Proposal

The Error Type

type E = Throwable

Retries are offered for monads with Throwable errors.

While it would be possible to abstract the proposal to work with a generic MonadError[A, E], some parts of the proposal prescribe the error type:

• the use of Resource to manage the lifecycle of a retryable operation;
• the ability to communicate an out-of-retries condition, while still wrapping the observed error, which is done via OutOfRetriesException.

Retry Policy

trait RetryPolicy[F[_]] {
  def shouldRetry: Boolean
  def redeem(e: Throwable): F[Unit]
}

The retry policy bears some relation to a plain Iterator[F[Unit]], but is given a dedicated interface for expressivity and flexibility. Many interesting retry policies can be expressed that ignore the redeemed error.

This retry policy is not primarily responsible for determining which errors to retry on, we’ll get back to that when we look at the retry loop. The policy does decide whether to retry an error that was separately deemed to be retryable.

Retryable errors are redeemed effectfully. This allows retry policies to implement various behaviors such as logging of errors, and back off strategies.

Lifecycle of a Retry Policy

import cats.effect.kernel.MonadCancelThrow
import cats.effect.kernel.Resource

type F
implicit val F: MonadCancelThrow[F]
implicit val retryPolicy: Resource[F, RetryPolicy[F]]

The retry policy is meant to be instantiated on each use, so that it can keep track of failed attempts for each use. To this end, policies are made available wrapped in a Resource that manages their lifetime.

The use of Resource gives retry policy implementations additional hooks to customize behavior, and allows policies to maintain state–if desired.

The OutOfRetriesException

final case class OutOfRetriesException(cause: Throwable) extends Exception

OutOfRetriesException is used to wrap retryable errors when the retry policy precludes further attempts.

With this, a retryable operation can have one of the following outcomes:

• it may succeed;
• it may fail with a non-retryable error, which gets propagated as is;
• it may fail with a retryable error, wrapped by OutOfRetriesException;
• it may be canceled.

Determining Retryable Errors

PartialFunction[Throwable, ?]

The interface prescribes the use of partial functions to determine whether an error is retryable. This is aligned with the use of PartialFunction to deal with errors throughout the interface of MonadError: adaptError, onError, recover, and recoverWith.

Proposed Interface

import scala.reflect.{ClassTag, classTag}

trait MonadCancelThrow[F] {

  def retry[A](fa: F[A])(pf: PartialFunction[Throwable, Unit])(implicit
      P: Resource[F, RetryPolicy[F]]
  ): F[A] =
    retryWith(fa)(pf.andThen(_ => F.unit))

  def retryNarrow[A, EE <: Throwable: ClassTag](fa: F[A])(implicit
      P: Resource[F, RetryPolicy[F]]
  ): F[A] =
    retryWith(fa) { case e if classTag[EE].runtimeClass.isInstance(e) => () }

  def retryWith[A](fa: F[A])(pf: PartialFunction[Throwable, F[Unit]])(implicit
      P: Resource[F, RetryPolicy[F]]
  ): F[A]

}

The methods retry and retryWith closely align with the existing recover and recoverWith methods. They allow for the identification of retryable errors, and also allow authors to introduce custom error handling, outside the retry policy.

retryNarrow provides a shorthand notation to retry errors of a single exception class. This aligns with attemptNarrow.

Syntax

fa.retry {
  case e: SomeRetryableError if someCondition(e) =>
    // …
    ()
}

fa.retryNarrow[SomeRetryableError]

fa.retryWith {
  case e: SomeRetryableError if someCondition(e) =>
    // …
    F.unit
}

The Retry Loop

type F[A]
implicit val F: MonadCancelThrow[F]
implicit val P: Resource[F, RetryPolicy[F]]

def retryWith[A](fa: F[A])(
    pf: PartialFunction [Throwable, Unit]
)(implicit P: Resource[F, RetryPolicy[F]]): F[A] =
  P.use { policy =>
    def attempt: F[A] =
      fa.recoverWith {
        case error if pf.isDefinedAt(error) =>
          if (policy.shouldRetry)
            F.flatMap(pf(error)) { _ =>
               F.flatMap(policy.redeem(error)) { _ =>
                 attempt
               }
            }
          else
            F.raiseError(OutOfRetriesException(error))
      }

    attempt
  }

Prior Art

(This section does not intend to be exhaustive, but still reference previous attempts to introduce retry functionality in Cats Effect.)

Retry functionality is offered in cats-retry, the inclusion of this functionality in Cats Effect has been proposed before in a couple of issues and in concrete pull requests:

• Add IO#retry and its variations #212
• Fold cats-retry into CE 3? #1459
• Retry functionality #3135
• Implement Retry functionality #4109
• https://github.com/biochimia/scala-retry

[iRevive]

Hey, thanks for putting everything together.

One question has bothered me for a while: should retries be supported for a successful outcome?

Let's consider the following example:

val client: Client[F] = ???
val response: F[Response[F]] = client.get("https://example.com")

Here, I would like to have an option to retry in the following cases:

• an unhandled exception (e.g. IOException)
• a response code isn't 200

It will notably affect the API design if we want to support this.

cc @djspiewak @armanbilge

[iRevive]

We should also keep Handle from the cats-mtl in mind.
I would prefer the retry algebra to work out of the box with MTL.

[iRevive]

A bit of self-reflection about my PR #4109.

What I don't like:

• No way to retry on the successful outcome
• No clear separation between retry strategy and error matching, everything is bundled into Retry
• Very confusing limits: time cap, max attempts, etc

[biochimia]

Retrying Successful Outcomes

I'm tempted to look at this issue as stemming from an incomplete mapping of values to errors. This can addressed with a flatMap operation, outside of the retry:

case class Retryable[A](a: A) extends Throwable

val response: F[Response[F]] =
  client
    .get("https://example.com")
    .flatMap { response =>
      if (response.statusCode == 400)
        F.raiseError(Retryable(Right(response)))
      else
        F.pure(response)
    }
    .retry {
      case _: IOException =>
      case Retryable(_)   =>
    }

Other alternatives include the use of attempt.map.rethrow or redeemWith, paired with retryNarrow, to keep mapping of success and error states close together, and to avoid repetition.

case class Retryable[A](a: A) extends Throwable

val response: F[Response[F]] =
  client
    .get("https://example.com")
    .attempt
    .map { response =>
      case Right(r) if r.statusCode == 400 => Left(Retryable(r))
      case Left(e: IOException)            => Left(Retryable(e))
      case other                           => other
    }
    .rethrow
    .retryNarrow[Retryable[?]]

I see retry functionality aligning closer with error handling methods, such as adaptError, handleError, recover, which specifically target error states. Mapping of specific successful states to errors seems to be well supported with existing functionality that I'm not sure it makes sense to add specific syntax for it in retry methods.

That said, this is one opinion 🙂.

[biochimia]

Integration with cats-mtl

I'll start by saying that I don't have specific experience with cats-mtl.

Based on a quick look of the library, I think it would be possible to translate the proposed functionality to work with Handle[F[_], Throwable] (which includes Raise[F[_], Throwable]). This would be used to handle errors in the retry loop, and raise OutOfRetriesException. In the proposal, the retry loop is written in terms of recoverWith, which is not provided by Handle, but it could be rewritten in terms of Handle.handleWith, by re-raising unhandled errors.

I'm not entirely sure what the requirement would be to instantiate the RetryPolicy in cats-mtl. The proposal uses Resource which manages instantiation and finalizers, but I don't find a match for it. I suppose simple policies could be supported via Ask[F[_], RetryPolicy[F]]: the bare minimum would be that the policy is instantiated on each ask, so that it can reset attempt counts.

[biochimia]

Any other thoughts on this topic?