Retry settings not respected when guardrail fails

[AgileEduLabs]

What Happened?

If the guardrail checks as defined in the config fails (using output_guardrails, with deny set to true), then the portkey.chat.completions.create request immediately errors with a status of 446. This happens even if the retry configuration object is set.

This behaviour is the same whether I use the hosted Developer plan or the open source gateway.

What Should Have Happened?

The expectation is that if the guardrail fails, the completion request will try again until the guardrail passes or the number of maximum attempts have been reached.

Relevant Code Snippet

No response

Your Twitter/LinkedIn

No response

[AgileEduLabs]

@narengogi This is a core issue severely affecting the functionality of the gateway. Hope you can help.

[narengogi]

@VisargD to comment on if this is expected behaviour or needs changing

[roh26it]

@AgileEduLabs - are you setting on_status_codes = [446] in the retry object?

https://portkey.ai/docs/product/ai-gateway/automatic-retries#retry-only-on-specific-error-codes

[AgileEduLabs]

@AgileEduLabs - are you setting on_status_codes = [446] in the retry object?

https://portkey.ai/docs/product/ai-gateway/automatic-retries#retry-only-on-specific-error-codes

Unfortunately, it doesn't matter what you set for the on_status_codes. Even with = [446], it immediately fails

[roh26it]

@AgileEduLabs Can you share the config you're using?