Bug in `Submit Approved URLs` task

[maxachis]

Bug occurred today where Submit Approved URLs task failed with error.

On inspecting the runtime logs, I saw 401 authentication error. It indicates that it receives a 401 and then tries to refresh and get a new access token.

The associated error as reported in the task_errors table is as follows:

401, message='Error making RequestType.POST request to https://data-sources.pdap.io/api/auth/refresh-session: Unauthorized', url='https://data-sources.pdap.io/api/auth/refresh-session'
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/pdap_access_manager/access_manager.py", line 224, in make_request
    response.raise_for_status()
  File "/usr/local/lib/python3.11/site-packages/aiohttp/client_reqrep.py", line 1161, in raise_for_status
    raise ClientResponseError(
aiohttp.client_exceptions.ClientResponseError: 401, message='Unauthorized', url='https://data-sources.pdap.io/api/source-collector/data-sources'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/app/src/core/tasks/operators/base.py", line 51, in run_task
    await self.inner_task_logic()
  File "/app/src/core/tasks/operators/submit_approved_url/core.py", line 34, in inner_task_logic
    submitted_url_infos = await self.pdap_client.submit_urls(tdos)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/src/pdap_api/client.py", line 132, in submit_urls
    response_info = await self.access_manager.make_request(request_info)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pdap_access_manager/access_manager.py", line 233, in make_request
    await self.refresh_access_token()
  File "/usr/local/lib/python3.11/site-packages/pdap_access_manager/access_manager.py", line 210, in refresh_access_token
    rsi = await self.make_request(rqi, allow_retry=False)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pdap_access_manager/access_manager.py", line 238, in make_request
    raise e
  File "/usr/local/lib/python3.11/site-packages/pdap_access_manager/access_manager.py", line 224, in make_request
    response.raise_for_status()
  File "/usr/local/lib/python3.11/site-packages/aiohttp/client_reqrep.py", line 1161, in raise_for_status
    raise ClientResponseError(
aiohttp.client_exceptions.ClientResponseError: 401, message='Error making RequestType.POST request to https://data-sources.pdap.io/api/auth/refresh-session: Unauthorized', url='https://data-sources.pdap.io/api/auth/refresh-session'

I restarted the application, and the task ran without error. Hooray!

All that suggests one of the following:

1. My first instinct was that perhaps the refresh token expired, but the source collector was last restarted on June 5th, which does not seem to be enough time for the refresh token to have expired. Still, it is possible.
2. It's possible there is a bug in the refresh process on the data-sources side,
3. Or in how pdap_access_manager handles the refresh process

At the moment, we seem to have a workaround in just restarting the process, but that's not a great long-term solution.

[maxachis]

Solution implemented by making changes to pdap_access_manager -- the problem was that, if it received an unauthorized response from refresh access token (which in this case meant the token expired), it didn't try to re-login. So I've modified it so that it does that.

Another thing we could do is just set up a small scheduled task to refresh the access token regularly, but for now I went with this, and we'll see if it recurs.

[maxachis]

It's been three weeks with no reoccurrence. For now, we'll consider it closed and reopen if need be.