Connection dies after a few requests

[jeroen]

If you run this a couple of times at some point we get Connection died, retrying a fresh connect (retry count: 1)

test <- httr2::example_url()
req <- curl::curl_fetch_memory(test, handle = curl::new_handle(verbose=T))

It does not error because libcurl will just try again. At this point the request has to be sent twice:

> req <- curl::curl_fetch_memory(httr2::example_url(), handle = curl::new_handle(verbose=T))
* Found bundle for host: 0x14d117c30 [serially]
* Can not multiplex, even if we wanted to
* Re-using existing connection with host 127.0.0.1
> GET / HTTP/1.1
Host: 127.0.0.1:50789
User-Agent: RStudio Desktop (2024.12.0.467); R (4.5.0 aarch64-apple-darwin20 aarch64 darwin20)
Accept: */*
Accept-Encoding: gzip

* Request completely sent off
* Connection died, retrying a fresh connect (retry count: 1)
* Closing connection
* Issue another request to this URL: 'http://127.0.0.1:50789/'
* NTLM-proxy picked AND auth done set, clear picked
* Hostname 127.0.0.1 was found in DNS cache
*   Trying 127.0.0.1:50789...
* Connected to 127.0.0.1 (127.0.0.1) port 50789
> GET / HTTP/1.1
Host: 127.0.0.1:50789
User-Agent: RStudio Desktop (2024.12.0.467); R (4.5.0 aarch64-apple-darwin20 aarch64 darwin20)
Accept: */*
Accept-Encoding: gzip

* Request completely sent off
< HTTP/1.1 200 OK
< Date: Mon, 31 Mar 2025 15:38:18 GMT
< Content-Type: text/html
< Content-Length: 357062
< ETag: "c165e293"
< 

[gaborcsardi]

I can't repro this:

❯ for (i in 1:100) { cat(".") ; req <- curl::curl_fetch_memory(test, handle = curl::new_handle()) }
....................................................................................................

[jeroen]

It does not error, you can only see it from the verbose output. It's not a biggie, as curl will deal with it. But it does affect httr2 snapshot tests.

[gaborcsardi]

What I am supposed to look at from those messages? The server will not keep the connection alive by default unless you start it with the keepalive option.

[jeroen]

The httr2::example_url() starts a keepalive server. After a few attempts we get Connection died in the verbose output, and we see the same GET getting resent by curl.

Here is a full reprex that errors

url <- httr2::example_url()


test <- function(){
  con <- rawConnection(raw(), "wb")
  on.exit(close(con))
  h <- curl::new_handle(verbose=TRUE, debugfunction = function(type, msg){
    if(type < 3) writeBin(msg, con)
  })
  req <- curl_fetch_memory(httr2::example_url(), handle = h)
  logs <- rawToChar(rawConnectionValue(con))
  if(grepl("Connection died", logs, fixed = TRUE)){
    cat(logs, file = stderr())
    stop("Connection died")
  }
}


for(i in 1:100){
  test()
  Sys.sleep(i * 0.1)
  message("OK", i)
}