`req_throttle()` realms lost/uninitialized using `library()` but works with `load_all()`

[smartiing]

Hi and thank you for this amazing package.

I'm running into an issue with req_throttle() when used in a function factory inside an unpublished package. I suspect this may be due to how the throttling state is stored and initialized, possibly combined with my use of function factory/closures.

Here's a simplified version of my code without documentation:

# --- I define a function factory
#' @export
req_function_factory = function(endpoint, body, realm = c("email", "sms", "others")) {
  realms = list(
    email  = list(capacity = 6000L, fill_time_s = 1L),
    sms    = list(capacity = 9L, fill_time_s = 1L),
    others = list(capacity = 60L, fill_time_s = 1L)
  )
  
  realm = match.arg(realm)
  
  req = 
    httr2::request("https://api.example.com") |>
    httr2::req_url_path_append(endpoint) |>
    httr2::req_method("GET") |>
    httr2::req_headers("api-key" = "secret") |>
    httr2::req_throttle(
      capacity    = realms[[realm]]$capacity,
      fill_time_s = realms[[realm]]$fill_time_s,
      realm       = realm
    )
  
  function(body = NULL) {
    req |>
      httr2::req_body_json(body, auto_unbox = FALSE, null = "list")
  }
}

# --- Then I create the functions for the endpoints that I need to use
#' @export
req_get_list = req_function_factory("/list", NULL, "email")

#' @export
req_get_sender = req_function_factory("/sender", NULL, "sms")

#...

I have been using this code for months and it works fine. I recently however added the call to httr2::req_throttle() and it didn't work as expected. When I install and load the package normally with library(mypackage), creating a request with one of the function works but performing the request doesn't.

Error in the$throttle[[req$policies$throttle_realm]]$take_token() : 
  attempt to apply non-function

But when I use devtools::load_all(), the same call works without issue.

I stepped into httr2:::throttle_delay() and noticed that the$throttle is empty after using library(). When inspecting the content of httr2:::the$throttle, it is indeed different:

> rlang::env_names(httr2:::the)
# [1] "pool_pollers"   "token_cache"    "last_response"  "breaker"        "last_request"   "throttle"       "cache_throttle" "last_interrupt"
> rlang::env_names(httr2:::the$throttle)
# character(0)
> library(mypackage)
> rlang::env_names(httr2:::the$throttle)
# character(0)
> devtools::load_all()
# ℹ Loading mypackage
> rlang::env_names(httr2:::the$throttle)
# [1] "all_contacts" "all_others"   "rest_smtp"   
> httr2:::the$throttle$all_contacts
# <TokenBucket>
#   Public:
#     capacity: 60
#     clone: function (deep = FALSE) 
#     fill_rate: 60
#     initialize: function (capacity, fill_time_s) 
#     last_fill: 1752663218
#     refill: function () 
#     return_token: function () 
#     take_token: function () 
#     token_wait_time: function () 
#     tokens: 60

I probably simply don't understand why there is a difference in the way httr2 is setting values in httr2:::the$throttle when using library() vs load_all(). I'm not sure whether this is expected behavior or if there's a better pattern I should be following that I don't know.

Thank you in advance for your help.

[hadley]

Hmmm, I'm not sure what's going on here, but I think the first thing you should do is to eliminate the function factory. It's unlikely to be involved, but I does make everything a little bit more complicated, and I don't think it's necessary. I would write it more like this:

req_base = function(endpoint, realm = c("email", "sms", "others")) {
  realms = list(
    email = list(capacity = 6000L, fill_time_s = 1L),
    sms = list(capacity = 9L, fill_time_s = 1L),
    others = list(capacity = 60L, fill_time_s = 1L)
  )

  realm = match.arg(realm)

  httr2::request("https://api.example.com") |>
    httr2::req_url_path_append(endpoint) |>
    httr2::req_method("GET") |>
    httr2::req_headers("api-key" = "secret") |>
    httr2::req_throttle(
      capacity = realms[[realm]]$capacity,
      fill_time_s = realms[[realm]]$fill_time_s,
      realm = realm
    )
}

req_base(endpoint) |> 
  httr2::req_body_json(body, auto_unbox = FALSE, null = "list")

[smartiing]

Thanks for the suggestion. I agree that using a factory makes things a bit more complex to debug, but for my use case, it's really convenient. I have many endpoints to wrap and the factory lets me define them all in a consistent and compact way without having to remember urls or making changes in multiple places in case of API changes. That's why I’d prefer to stick with that pattern. It allows me to easily create a function that will create a request for a specific endpoint with default values such as this :

sb_req_get_attributes = sb_req_function_factory(
  glue_endpoint = "contacts/attributes",
  method = "GET",
  realm = "all_contacts"
)

sb_req_create_attribute = sb_req_function_factory(
  glue_endpoint = "contacts/attributes/{category}/{name}",
  method = "POST",
  default_body = list(type = jsonlite::unbox("text")),
  realm = "all_contacts"
)

sb_req_get_lists_in_folder = sb_req_function_factory(
  glue_endpoint = "contacts/folders/{id}/lists",
  method = "GET",
  default_query = list(
    limit = 50L,
    offset = 0L,
    sort = "desc"
  ),
  realm = "all_contacts"
)

# Example :
sb_req_get_lists_in_folder(id = 41224, query = list(limit = 3)) |> httr2::req_perform()
# <httr2_response>
# GET https://api.brevo.com/v3/contacts/folders/41224/lists?limit=3&offset=0&sort=desc
# Status: 200 OK
# Content-Type: application/json
# Body: In memory (362 bytes)

The issue isn’t the factory per se, but when and how the function factory is called because this is then the only time I call req_throttle() and create the realms. Specifically, because I call req_throttle() inside the factory, and the factory is evaluated at package load time, it apparently tries to mutate httr2:::the$throttle. That works fine when using load_all() but fails silently when using library() on the installed package. I can imagine 2 reasons but this is a bit above my expertise:

1. R prevents modifying another package’s internal environment during that stage and I didn't know it.
2. It is modified but it is set back to nothing because the code from R/httr2-package.R is ran at some point afterwards when using library() but not when using load_all():

the <- new_environment()
the$throttle <- new_environment()

Anyway, now that I understand the problem, I tested and the 3 solutions below work:

1. I re-recreate the realms by creating dummy request in .onLoad:

.onLoad = function(libname, pkgname) {
  httr2::request("url") |> httr2::req_throttle(capacity = 6000L, fill_time_s =  1L, realm = "smtp_email_or_blockedcontacts")
  httr2::request("url") |> httr2::req_throttle(capacity = 9L, fill_time_s =  1L, realm = "get_smtp_emails")
  httr2::request("url") |> httr2::req_throttle(capacity = 60L, fill_time_s =  1L, realm = "post_events")
  httr2::request("url") |> httr2::req_throttle(capacity = 1800L, fill_time_s =  60L*60L, realm = "rest_smtp")
  httr2::request("url") |> httr2::req_throttle(capacity = 60L, fill_time_s =  1L, realm = "all_contacts")
  httr2::request("url") |> httr2::req_throttle(capacity = 600L, fill_time_s =  60L*60L, realm = "all_others")
}

2. Or by changing the environment manually (not a good solution but it works):

.onLoad = function(libname, pkgname) {
  rlang::env_bind(
    httr2:::the$throttle,
    smtp_email_or_blockedcontacts = httr2::TokenBucket$new(6000L, 1L),
    get_smtp_emails = httr2::TokenBucket$new(9L, 1L),
    post_events = httr2::TokenBucket$new(60L, 1L),
    rest_smtp = httr2::TokenBucket$new(1800L, 60L*60L),
    all_contacts = httr2::TokenBucket$new(60L, 1L),
    all_others = httr2::TokenBucket$new(600L, 60L*60L)
  )
}

3. Or by simply putting the call to req_throttle inside the body of the function returned by the function factory rather than outside of it so req_throttle is not executed during library().

All in all, this seems like a subtle aspect of how environments and namespace loading interact in R. I didn’t expect that the timing of calling req_throttle() could make such a difference, but I get it now.

If you know the reason why httr2::the$throttle is empty after library() but not after load_all() and don't mind explaining or can redirect me to some doc, it would be greatly appreciated. In any case, the problem is solved and I don't think this would be an issue except if someone else is trying to define realms for requests at package loading. I could imagine someone creating some sort of default request in the namespace of their package such as .req_base (where they would define proxy, headers, retry behaviour, throttle behaviour, etc), they would thus have the same problem because the requests would have a throttle_realm that isn't defined in httr2:::the$throttle.

Thanks again for your help and your time.