[CI] MlWithSecurityIT test {yaml=ml/inference_crud/Test force delete given model with alias referenced by pipeline} failing

[elasticsearchmachine]

Build Scans:

• elasticsearch-periodic #4769 / openjdk23_checkpart4_java-matrix
• elasticsearch-pull-request #40154 / part-4
• elasticsearch-pull-request #40130 / part-4

Reproduction Line:

./gradlew ":x-pack:plugin:ml:qa:ml-with-security:yamlRestTest" --tests "org.elasticsearch.smoketest.MlWithSecurityIT.test {yaml=ml/inference_crud/Test force delete given model with alias referenced by pipeline}" -Dtests.seed=3DF1D608D891EC -Dtests.locale=be-TARASK -Dtests.timezone=BST -Druntime.java=23

Applicable branches:
main

Reproduces locally?:
N/A

Failure History:
See dashboard

Failure Message:

org.elasticsearch.client.ResponseException: method [GET], host [http://127.0.0.1:42397], URI [/_ml/trained_models/_stats?size=10000], status line [HTTP/1.1 500 Internal Server Error]
{"error":{"root_cause":[],"type":"exception","reason":"Searching for stats for models [h-classification-model,d-classification-model,c-classification-model,g-classification-model,i-classification-model,b-classification-model,lang_ident_model_1,y-classification-model,f-classification-model,k-classification-model,j-classification-model,z-classification-model,a-regression-model-1,a-regression-model-0,e-classification-model] failed","caused_by":{"type":"search_phase_execution_exception","reason":"","phase":"query","grouped":true,"failed_shards":[],"caused_by":{"type":"search_phase_execution_exception","reason":"Search rejected due to missing shards [[.ml-stats-000001][0]]. Consider using `allow_partial_search_results` setting to bypass this error.","phase":"query","grouped":true,"failed_shards":[]}}},"status":500}

Issue Reasons:

• [main] 3 failures in test test {yaml=ml/inference_crud/Test force delete given model with alias referenced by pipeline} (1.1% fail rate in 275 executions)
• [main] 2 failures in step part-4 (1.5% fail rate in 136 executions)
• [main] 2 failures in pipeline elasticsearch-pull-request (1.5% fail rate in 137 executions)

Note:
This issue was created using new test triage automation. Please report issues or feedback to es-delivery.

[elasticsearchmachine]

This has been muted on branch main

Mute Reasons:

• [main] 3 failures in test test {yaml=ml/inference_crud/Test force delete given model with alias referenced by pipeline} (1.1% fail rate in 275 executions)
• [main] 2 failures in step part-4 (1.5% fail rate in 136 executions)
• [main] 2 failures in pipeline elasticsearch-pull-request (1.5% fail rate in 137 executions)

Build Scans:

• elasticsearch-periodic #4769 / openjdk23_checkpart4_java-matrix
• elasticsearch-pull-request #40154 / part-4
• elasticsearch-pull-request #40130 / part-4

[elasticsearchmachine]

Pinging @elastic/ml-core (Team:ML)