wasapi-downloader complains about invalid cookie header

[lwrubel]

However, the cookie-header expires value looks valid at first glance: Thu, 28 May 2026 19:08:46 GMT

Backtrace

line 33 of [PROJECT_ROOT]/app/jobs/fetch_job.rb: FetchJob#fetch_warcs
line 21 of [PROJECT_ROOT]/app/jobs/fetch_job.rb: FetchJob#perform

View full backtrace and more info at honeybadger.io

[was-registrar-app/qa] RuntimeError: Fetching WARCs failed: May 29, 2025 12:08:46 PM org.apache.http.client.protocol.ResponseProcessCookies processCookiesWARNING: Invalid cookie header: "set-cookie: csrftoken=v4TLeIV3mISV5iEBgSTYNQFbCGPYRXcO; expires=Thu, 28 May 2026 19:08:46 GMT; Max-Age=31449600; Path=/; SameSite=Lax". Invalid 'expires' attribute: Thu, 28 May 2026 19:08:46 GMTMay 29, 2025 12:08:46 PM org.apache.http.client.protocol.ResponseProcessCookies processCookiesWARNING: Invalid cookie header:...```

[lwrubel]

Wondering if this is related to the recent Ubuntu upgrade to 22 on the WRA VMs. We still have the same version of Java: 17, so there was not any change with that.

[lwrubel]

I can replicate this by running wasapi-downloader on the command line on QA:

was@was-registrar-app-qa:~/was_registrar_app/current$ /opt/wasapi-downloader/wasapi-downloader/bin/wasapi-downloader --baseurl 'https://partner.archive-it.org/wasapi/v1/' --authurl 'https://partner.archive-it.org/login' --username <redacted> --password <redacted> --outputBaseDir 'tmp/warcs' --collectionId 1064 --crawlStartAfter 2009-08-01 --crawlStartBefore 2009-09-01
May 29, 2025 12:35:54 PM org.apache.http.client.protocol.ResponseProcessCookies processCookies
WARNING: Invalid cookie header: "set-cookie: csrftoken=cmMEpN8l04fNWi1a3nxKVaWHIC13DYWh; expires=Thu, 28 May 2026 19:35:54 GMT; Max-Age=31449600; Path=/; SameSite=Lax". Invalid 'expires' attribute: Thu, 28 May 2026 19:35:54 GMT
May 29, 2025 12:35:54 PM org.apache.http.client.protocol.ResponseProcessCookies processCookies
WARNING: Invalid cookie header: "set-cookie: sessionid=72hffk8v8b7z85cv8ri3fgdvw0j6e5wh; Domain=.archive-it.org; expires=Thu, 12 Jun 2025 19:35:54 GMT; HttpOnly; Max-Age=1209600; Path=/; SameSite=Lax". Invalid 'expires' attribute: Thu, 12 Jun 2025 19:35:54 GMT
Exception in thread "main" org.apache.http.client.HttpResponseException: Forbidden
	at edu.stanford.dlss.was.WasapiValidator.validateResponse(WasapiValidator.java:24)
	at edu.stanford.dlss.was.JsonResponseHandler.handleResponse(JsonResponseHandler.java:17)
	at edu.stanford.dlss.was.JsonResponseHandler.handleResponse(JsonResponseHandler.java:11)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:223)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:165)
	at edu.stanford.dlss.was.WasapiClient.execute(WasapiClient.java:48)
	at edu.stanford.dlss.was.WasapiConnection.jsonQuery(WasapiConnection.java:28)
	at edu.stanford.dlss.was.WasapiConnection.pagedJsonQuery(WasapiConnection.java:34)
	at edu.stanford.dlss.was.WasapiDownloader.downloadSelectedWarcs(WasapiDownloader.java:39)
	at edu.stanford.dlss.was.WasapiDownloader.executeFromCmdLine(WasapiDownloader.java:33)
	at edu.stanford.dlss.was.WasapiDownloader.main(WasapiDownloader.java:194)
```

[lwrubel]

Perhaps gradle 5.0 does not work with Java 17? We have openjdk 17 installed on the WRA VMs. https://docs.gradle.org/current/userguide/compatibility.html