[BUG] OpenSearch Dashboards: Parallel refresh token requests cause premature logout with Keycloak

[gigallucci]

Description

When using OpenSearch Dashboards with Keycloak as an IdP (OIDC), users are redirected to the login page prematurely, even though the Keycloak session is still valid.

This happens because Dev Tools in OpenSearch Dashboards sends multiple parallel background requests (autocomplete refresh) that all attempt to use the same refresh token.
If Keycloak is configured with Refresh Token Max Reuse = 1, these parallel requests trigger Keycloak's automatic reuse detection. The refresh token is marked as invalid → Dashboards redirect to the login page.

The underlying Keycloak session is still active, so clicking "Login" continues the session, but this behavior is confusing for users.

---

Steps to Reproduce

1. Configure OpenSearch Dashboards with Keycloak OIDC authentication.
2. In Keycloak set Refresh Token Max Reuse = 1.
3. Log into OpenSearch Dashboards.
4. Open Dev Tools.
   • Dev Tools automatically refreshes autocomplete entries every minute.
   • Three parallel REFRESH_TOKEN requests are triggered almost simultaneously.
5. Observe that one of these requests fails with "token expired" → Dashboards redirect to the login page.
6. Clicking "Login" works without entering credentials, because the Keycloak session is still valid.

---

Expected Behavior

• OpenSearch Dashboards should handle refresh tokens in a way that avoids parallel reuse.
• Ideally, only one refresh request should be sent at a time, or refresh tokens should be cached until new ones are available.
• Users should not be redirected to the login page while their Keycloak session is still valid.

---

Actual Behavior

• Multiple parallel refresh requests are issued.
• With Max Reuse = 1, Keycloak invalidates the refresh token.
• Dashboards interpret this as session expiration and redirect to the login page.
• Users can resume by clicking "Login", but the UX is broken.

---

Impact

• Users perceive random logouts when working in Dashboards.
• Strongly affects environments with strict Keycloak settings (high security).
• Leads to frustration and confusion.

---

Logs / Evidence

 kubectl logs -n infra -f infra-keycloak-0 -c keycloak \
| jq -r '
  def pad($s; $n):
    ($s // "") as $x
    | ($n - ($x|length)) as $d
    | $x + (if $d>0 then (reduce range(0;$d) as $i (""; .+" ")) else "" end);

  select(type=="object" and .loggerName=="org.keycloak.events" and (.message|test("clientId=\"opensearch\"")))
  | (.message
     | capture("type=\"(?<type>[^\"]+)\".*realmName=\"(?<realm>[^\"]+)\".*clientId=\"(?<client>[^\"]+)\".*userId=\"(?<user>[^\"]+)\".*sessionId=\"(?<session>[^\"]+)\".*ipAddress=\"(?<ip>[^\"]+)\"(.*access_token_expiration_time=\"(?<access_token_expiration_time>[^\"]+)\".*updated_refresh_token_id=\"(?<updated_refresh_token_id>[^\"]+)\".*age_of_refresh_token=\"(?<age_of_refresh_token>[^\"]+)\".*refresh_token_id=\"(?<refresh_token_id>[^\"]+)\")?")
    ) as $m
  | pad(.timestamp; 30)    + " "
  + pad($m.type; 19)       + " "
  + pad($m.realm; 16)      + " "
  + pad($m.client; 10)     + " "
  + pad($m.user // "not set"; 36 ) + " "
  + pad(($m.session // ""); 0) + " "
  + pad($m.ip; 13)         + " "
  + pad(($m.access_token_expiration_time // ""); 3) + " "
  + pad(($m.updated_refresh_token_id // ""); 3) + " "
  + pad(($m.age_of_refresh_token // ""); 3) + " "
  + pad(($m.refresh_token_id // ""); 0)
'

Time	Type	Realm	Client	User	Session	IP	AT-Exp-Time	Updated-RT-ID	Age-of-RT	RT-ID
2025-09-01T09:52:14.767861836Z	LOGIN	SIEM-Application	opensearch	f361b1d7-8b49-46a9-b753-462a50b2d371	74c5ccdd-dc71-4609-8012-3aae7e5f4cbe	10.244.0.1
2025-09-01T09:52:14.867037387Z	CODE_TO_TOKEN	SIEM-Application	opensearch	f361b1d7-8b49-46a9-b753-462a50b2d371	74c5ccdd-dc71-4609-8012-3aae7e5f4cbe	10.244.0.1
2025-09-01T09:54:16.866995813Z	REFRESH_TOKEN	SIEM-Application	opensearch	f361b1d7-8b49-46a9-b753-462a50b2d371	74c5ccdd-dc71-4609-8012-3aae7e5f4cbe	10.244.0.1	30	757c5f23-b14e-4944-9520-e15464f1694e	122	e8d15b15-69cd-495e-b836-97aa2a2116ed
2025-09-01T09:54:16.868422346Z	REFRESH_TOKEN	SIEM-Application	opensearch	f361b1d7-8b49-46a9-b753-462a50b2d371	74c5ccdd-dc71-4609-8012-3aae7e5f4cbe	10.244.0.1	30	b03a830b-fae5-469f-a68a-ecd3642f364f	122	e8d15b15-69cd-495e-b836-97aa2a2116ed
2025-09-01T09:54:16.923588635Z	REFRESH_TOKEN	SIEM-Application	opensearch	f361b1d7-8b49-46a9-b753-462a50b2d371	74c5ccdd-dc71-4609-8012-3aae7e5f4cbe	10.244.0.1	30	911088a0-0ad0-4f95-b435-d6c5c1b5d305	122	e8d15b15-69cd-495e-b836-97aa2a2116ed
2025-09-01T09:54:16.925094337Z	REFRESH_TOKEN	SIEM-Application	opensearch	f361b1d7-8b49-46a9-b753-462a50b2d371	74c5ccdd-dc71-4609-8012-3aae7e5f4cbe	10.244.0.1	30	f1931afe-fc68-489d-a8c6-98940c3e6a38	122	e8d15b15-69cd-495e-b836-97aa2a2116ed
2025-09-01T09:54:16.928059203Z	REFRESH_TOKEN	SIEM-Application	opensearch	f361b1d7-8b49-46a9-b753-462a50b2d371	74c5ccdd-dc71-4609-8012-3aae7e5f4cbe	10.244.0.1	30	c5917072-3d20-4037-9694-d1a591cb6337	122	e8d15b15-69cd-495e-b836-97aa2a2116ed
2025-09-01T09:55:17.435780161Z	REFRESH_TOKEN	SIEM-Application	opensearch	f361b1d7-8b49-46a9-b753-462a50b2d371	74c5ccdd-dc71-4609-8012-3aae7e5f4cbe	10.244.0.1	30	1e412b3e-7cbe-4045-89c6-973cea2d59bb	61	911088a0-0ad0-4f95-b435-d6c5c1b5d305
2025-09-01T09:55:17.436153011Z	REFRESH_TOKEN	SIEM-Application	opensearch	f361b1d7-8b49-46a9-b753-462a50b2d371	74c5ccdd-dc71-4609-8012-3aae7e5f4cbe	10.244.0.1	30	0310bba4-65e2-4333-8000-738f587868bf	61	911088a0-0ad0-4f95-b435-d6c5c1b5d305
2025-09-01T09:55:17.437456054Z	REFRESH_TOKEN	SIEM-Application	opensearch	f361b1d7-8b49-46a9-b753-462a50b2d371	74c5ccdd-dc71-4609-8012-3aae7e5f4cbe	10.244.0.1	30	619f3da1-d05f-4447-b533-6e73d8e71146	61	911088a0-0ad0-4f95-b435-d6c5c1b5d305
2025-09-01T09:56:17.868087379Z	REFRESH_TOKEN	SIEM-Application	opensearch	f361b1d7-8b49-46a9-b753-462a50b2d371	74c5ccdd-dc71-4609-8012-3aae7e5f4cbe	10.244.0.1	30	733b05fd-550b-483b-b9bc-450f18894b1e	60	0310bba4-65e2-4333-8000-738f587868bf
2025-09-01T09:56:17.869802632Z	REFRESH_TOKEN	SIEM-Application	opensearch	f361b1d7-8b49-46a9-b753-462a50b2d371	74c5ccdd-dc71-4609-8012-3aae7e5f4cbe	10.244.0.1	30	11e6e51c-2503-4d3a-84b7-a23ce46c3708	60	0310bba4-65e2-4333-8000-738f587868bf
2025-09-01T09:56:17.870989704Z	REFRESH_TOKEN	SIEM-Application	opensearch	f361b1d7-8b49-46a9-b753-462a50b2d371	74c5ccdd-dc71-4609-8012-3aae7e5f4cbe	10.244.0.1	30	e4499f39-76e1-45ef-a2d7-1777069711bf	60	0310bba4-65e2-4333-8000-738f587868bf
2025-09-01T09:57:18.670798739Z	REFRESH_TOKEN	SIEM-Application	opensearch	f361b1d7-8b49-46a9-b753-462a50b2d371	74c5ccdd-dc71-4609-8012-3aae7e5f4cbe	10.244.0.1	30	3d24f0bc-d1b1-4dd0-b74a-d57f4af4bfca	61	11e6e51c-2503-4d3a-84b7-a23ce46c3708
2025-09-01T09:57:18.672677402Z	REFRESH_TOKEN	SIEM-Application	opensearch	f361b1d7-8b49-46a9-b753-462a50b2d371	74c5ccdd-dc71-4609-8012-3aae7e5f4cbe	10.244.0.1	30	414de8b5-8cb5-40d0-950a-5a43b2b9691d	61	11e6e51c-2503-4d3a-84b7-a23ce46c3708
2025-09-01T09:57:18.674653445Z	REFRESH_TOKEN	SIEM-Application	opensearch	f361b1d7-8b49-46a9-b753-462a50b2d371	74c5ccdd-dc71-4609-8012-3aae7e5f4cbe	10.244.0.1	30	cd84a7b7-ebd9-4cc5-bfdf-cf0f14212575	61	11e6e51c-2503-4d3a-84b7-a23ce46c3708
2025-09-01T09:57:59.010591324Z	REFRESH_TOKEN	SIEM-Application	opensearch	f361b1d7-8b49-46a9-b753-462a50b2d371	74c5ccdd-dc71-4609-8012-3aae7e5f4cbe	10.244.0.1	30	1cf08ebf-362a-4694-b308-bdacb96aad80	41	3d24f0bc-d1b1-4dd0-b74a-d57f4af4bfca
2025-09-01T09:59:20.571454607Z	REFRESH_TOKEN	SIEM-Application	opensearch	f361b1d7-8b49-46a9-b753-462a50b2d371	74c5ccdd-dc71-4609-8012-3aae7e5f4cbe	10.244.0.1	30	7cea83db-434d-4a4b-9ede-23530d439935	81	1cf08ebf-362a-4694-b308-bdacb96aad80
2025-09-01T09:59:20.571983868Z	REFRESH_TOKEN	SIEM-Application	opensearch	f361b1d7-8b49-46a9-b753-462a50b2d371	74c5ccdd-dc71-4609-8012-3aae7e5f4cbe	10.244.0.1	30	28640dd6-7df9-40a9-b25d-548318252537	81	1cf08ebf-362a-4694-b308-bdacb96aad80
2025-09-01T09:59:20.573469027Z	REFRESH_TOKEN	SIEM-Application	opensearch	f361b1d7-8b49-46a9-b753-462a50b2d371	74c5ccdd-dc71-4609-8012-3aae7e5f4cbe	10.244.0.1	30	5b3f72db-f558-4210-a934-fdf66dfc48b9	81	1cf08ebf-362a-4694-b308-bdacb96aad80
2025-09-01T10:00:21.571250535Z	REFRESH_TOKEN	SIEM-Application	opensearch	f361b1d7-8b49-46a9-b753-462a50b2d371	74c5ccdd-dc71-4609-8012-3aae7e5f4cbe	10.244.0.1	30	7efef8aa-096b-4367-8e8d-8e7a96e2c62c	61	28640dd6-7df9-40a9-b25d-548318252537
2025-09-01T10:00:21.574557043Z	REFRESH_TOKEN	SIEM-Application	opensearch	f361b1d7-8b49-46a9-b753-462a50b2d371	74c5ccdd-dc71-4609-8012-3aae7e5f4cbe	10.244.0.1	30	7551ac2e-91a0-4822-991c-22d60f37b230	61	28640dd6-7df9-40a9-b25d-548318252537
2025-09-01T10:00:21.576731705Z	REFRESH_TOKEN	SIEM-Application	opensearch	f361b1d7-8b49-46a9-b753-462a50b2d371	74c5ccdd-dc71-4609-8012-3aae7e5f4cbe	10.244.0.1	30	99f6fd5f-84a6-4642-b1a8-7a0276a331e8	61	28640dd6-7df9-40a9-b25d-548318252537
2025-09-01T10:01:14.417333155Z	REFRESH_TOKEN	SIEM-Application	opensearch	f361b1d7-8b49-46a9-b753-462a50b2d371	74c5ccdd-dc71-4609-8012-3aae7e5f4cbe	10.244.0.1	30	b60d3128-8943-4847-b74d-a06ed96f25d3	53	7efef8aa-096b-4367-8e8d-8e7a96e2c62c
2025-09-01T10:06:23.733101077Z	REFRESH_TOKEN	SIEM-Application	opensearch	f361b1d7-8b49-46a9-b753-462a50b2d371	74c5ccdd-dc71-4609-8012-3aae7e5f4cbe	10.244.0.1	30	0587d2f1-cb49-484f-a100-2258f84022a3	309	b60d3128-8943-4847-b74d-a06ed96f25d3
2025-09-01T10:13:11.644057621Z	REFRESH_TOKEN	SIEM-Application	opensearch	f361b1d7-8b49-46a9-b753-462a50b2d371	74c5ccdd-dc71-4609-8012-3aae7e5f4cbe	10.244.0.1	30	94d72cf5-51d8-43b2-822b-b2604e8c7a14	408	0587d2f1-cb49-484f-a100-2258f84022a3

Here is a description of each column in the table:

• Time: The precise timestamp when the event occurred, including date and milliseconds. This helps track the sequence and timing of authentication and token operations.

• Type: The kind of event being logged, such as LOGIN (user authentication), CODE_TO_TOKEN (exchange of authorization code for tokens), or REFRESH_TOKEN (refreshing an expired access token).

• Realm: The Keycloak realm in which the event took place. Realms are isolated authentication domains within Keycloak.

• Client: The application or service requesting authentication, in this case "opensearch".

• User: The unique identifier (UUID) of the user involved in the event. This allows tracking activity per user.

• Session: The unique session ID assigned by Keycloak for the user's login session. It is used to correlate events within the same session.

• IP: The IP address from which the request originated. Useful for auditing and detecting suspicious activity.

• AT-Exp-Time: The expiration time (in seconds) of the access token, if available. Indicates how long the access token remains valid.

• Updated-RT-ID: The identifier of the newly issued refresh token, if the refresh operation succeeded. This helps trace the lineage of refresh tokens.

• Age-of-RT: The age (in seconds) of the refresh token at the time of the event. Useful for understanding token lifecycle and reuse.

• RT-ID: The identifier of the refresh token used in the event. This allows tracking which refresh token was involved and whether it was reused or replaced.

---

Workarounds

• Increase Refresh Token Max Reuse in Keycloak (>1).
  • This avoids the issue but weakens security, since tokens can be reused multiple times.
• Disable Autocomplete in Dev Tools (stops background refresh requests).
• Avoid using multiple parallel Dashboard tabs with auto-refresh enabled.

---

Environment

• OpenSearch Dashboards: 2.19.2
• OpenSearch: 2.19.2
• Keycloak: 26.2.5

---

Suggested Fix

• Serialize or de-duplicate refresh requests in Dashboards, so that only one request at a time uses the refresh token.
• Cache the newly issued refresh token across parallel requests.
• Ensure that Dev Tools autocomplete does not trigger parallel refreshes.

[cwperks]

[Triage] @gigallucci Thank you for filing this issue. I will leave it in the security-dashboards-plugin repo, but it may make sense for the solution to this to be in the OSD Core repository by debouncing these requests. Is it sending a new request per keystroke?

[gigallucci]

@cwperks : Thanks for looking into this!
The requests are not triggered per keystroke. DEV-Tools sends three requests in parallel as part of the AutoSuggestion feature every minute.
https://github.com/opensearch-project/OpenSearch-Dashboards/blob/58f770e095e859007d66543936df6ce90799df55/src/plugins/console/public/lib/mappings/mappings.ts#L409

[gigallucci]

Hi @cwperks, let me know if you need any additional information. Or do you think this bug is more likely to be found in the opensearch core project and we should move it?

[cwperks]

@gigallucci What are the requests that are occurring in parallel? I think the solution is either to have these run in serial order, or to add retry logic to the requests to retry on 401 assuming one of them will "win" the new access token and then on retry they can use the refreshed access token.

[gigallucci]

@cwperks :

For better clarity, here is a small test:
For the test, I set the access token lifespan to 3 minutes and max reuse to 0.

Then log in to Opensearch Dashboards via OIDC and switch to DEV Tools. Wait there for 3 minutes (go get a coffee or something :) ).

Three GET requests are called up in the background every minute as soon as Dev Tools is opened. No further interaction is necessary.


• https://dashboards.local/api/console/proxy?path=_mapping&method=GET&dataSourceId=
• https://dashboards.local/api/console/proxy?path=_template&method=GET&dataSourceId=
• https://dashboards.local/api/console/proxy?path=_aliases&method=GET&dataSourceId=

After three minutes (refresh token expires), you will see something like the following image:

If I understand the source code correctly, the requests are executed asynchronously. 

See https://github.com/opensearch-project/OpenSearch-Dashboards/blob/58f770e095e859007d66543936df6ce90799df55/src/plugins/console/public/lib/mappings/mappings.ts#L409


So executing the requests sequentially would basically work. However, it cannot be ruled out that this situation could also occur in other apps in dashboards. It is also conceivable that several tabs open in the browser with Opensearch dashboards and autorefresh set in the timepicker could be lead to this situation.
Basically, you would have to prevent multiple refresh token requests from being called at the same time.

[cwperks]

@gigallucci I think there's a larger architectural issue around client-side sessions for dashboards. I think this issue would be completely mitigated by moving session management server-side.

[cwperks]

There is some discussion on this here: #1711

Actually, I can't find a good issue that describes it.