ApplyGuardrail doesnt apply ContextualGrouding on boto3 API

[dannielvieira-hotmart]

Describe the bug

Setting up a Guardrail with only ContextualGrounding Policy (threshold of 0,8 on both scores) and then running ApplyGuardrail passing each qualifier type doesnt work, but it does on Console. Images for reference:

• Boto3

  • Creating Guardrail
  
  • Testing Guardrail
  

• Console
  

Regression Issue

• Select this option if this issue appears to be a regression.

Expected Behavior

Guardrail should work the same on boto3 and console.

Current Behavior

ContextualGrouding doesn't work the on boto3

Reproduction Steps

1. Create a Guardrail with this setup

response = bedrock.create_guardrail(
name='testing_guardrail_10',
blockedInputMessaging='Input blocked!',
blockedOutputsMessaging='Output blocked!',
contextualGroundingPolicyConfig=
{
'filtersConfig':
[
{
'type': 'GROUNDING',
'threshold': 0.8
},
{
'type': 'RELEVANCE',
'threshold': 0.8
}
]
}
)

2. Run ApplyGuardrail:

source = 'London is the capital of UK. Tokyo is the capital of Japan'
input = 'What is 2+2?'
output = 'Tokyo is the capital city of Japan, while some sources list Nagoya or Osaka as the largest city by population. London is the capital city of the United Kingdom, but some sources list Birmingham as the largest city by population.'
bedrock_runtime.apply_guardrail(
guardrailIdentifier='tvc6ts6fnjrh',
guardrailVersion='DRAFT',
source='INPUT',
content=[
{
'text': {
'text': source,
'qualifiers': ['grounding_source']
}
},
{
'text': {
'text': input,
'qualifiers': ['query']
}
},
{
'text': {
'text': output,
'qualifiers': ['guard_content']
}
}
]
)

3. Open same Guardrail on Console, and run with same prompt and source.

### Possible Solution

_No response_

### Additional Information/Context

_No response_

### SDK version used

1.35.31

### Environment details (OS name and version, etc.)

Sagemaker Distribution 1.4

[tim-finnigan]

Thanks for reaching out. The create_guardrail and apply_guardrail commands make requests to the CreateGuardrail and ApplyGuardrail APIs, so if the behavior is not working as expected then that could be an issue with the APIs rather than the SDK directly.

Based on these docs, I would expect to see 'action': 'GUARDRAIL_INTERVENED', in the response: https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails-use-independent-api.html. There's another section in the Bedrock developer guide here on testing guardrails using InvokeModel: https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails-test.html. Have you also tried testing with invoke_model?

Can also you confirm that you're referencing the correct guardrail in your apply_guardrail command? If you could share your debug logs (with sensitive info redacted) by adding boto3.set_stream_logger('') to your script, then we can see the full API response being returned.

[github-actions]

Greetings! It looks like this issue hasn’t been active in longer than five days. We encourage you to check if this is still an issue in the latest release. In the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please feel free to provide a comment or upvote with a reaction on the initial post to prevent automatic closure. If the issue is already closed, please feel free to open a new one.

[dannielvieira-hotmart]

Hi @tim-finnigan, sorry for the wait.

I haven't been able to use Guardrails using invoke_model, it seems to have something wrong with the body formatting and fields. Currently, i'm trying this:

model_id = "ai21.j2-mid-v1"

# Define the prompt for the model.
prompt = """
<amazon-bedrock-guardrails-groundingSource_xyz>London is the capital of UK. Tokyo is the capital of Japan. </amazon-bedrock-guardrails-groundingSource_xyz>

<amazon-bedrock-guardrails-query_xyz>What is the capital of Japan?</amazon-bedrock-guardrails-query_xyz>
"""

# Format the request payload using the model's native structure.
native_request = {
    "prompt": prompt,
    "maxTokens": 512,
    "temperature": 0.5,
}

# Convert the native request to JSON.
request = json.dumps(native_request)

response = bedrock_runtime.invoke_model(modelId=model_id, body=request, guardrailIdentifier='tvc6ts6fnjrh')

And is giving me this:

ValidationException: An error occurred (ValidationException) when calling the InvokeModel operation: Guardrail was enabled but input is in incorrect format.

Based on this and this.

And here is the applyGuadrail debug log, triple checking on the correct Guadrail ID.

2024-10-09 18:11:42,713 botocore.hooks [DEBUG] Event before-parameter-build.bedrock-runtime.ApplyGuardrail: calling handler <function generate_idempotent_uuid at 0xANONYMIZED>
2024-10-09 18:11:42,714 botocore.regions [DEBUG] Calling endpoint provider with parameters: {'Region': 'us-east-1', 'UseDualStack': False, 'UseFIPS': False}
2024-10-09 18:11:42,716 botocore.regions [DEBUG] Endpoint provider result: https://bedrock-runtime.us-east-1.amazonaws.com
2024-10-09 18:11:42,718 botocore.hooks [DEBUG] Event before-call.bedrock-runtime.ApplyGuardrail: calling handler <function add_recursion_detection_header at 0xANONYMIZED>
2024-10-09 18:11:42,719 botocore.hooks [DEBUG] Event before-call.bedrock-runtime.ApplyGuardrail: calling handler <function inject_api_version_header_if_needed at 0xANONYMIZED>
2024-10-09 18:11:42,720 botocore.endpoint [DEBUG] Making request for OperationModel(name=ApplyGuardrail) with params: {'url_path': '/guardrail/ANONYMIZED/version/DRAFT/apply', 'query_string': {}, 'method': 'POST', 'headers': {'Content-Type': 'application/json', 'User-Agent': 'Boto3/1.35.31 md/Botocore#1.35.31 ua/2.0 os/linux#5.10.225-213.878.amzn2.x86_64 md/arch#x86_64 lang/python#3.10.14 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.35.31'}, 'body': b'{"source": "INPUT", "content": [{"text": {"text": "London is the capital of UK. Tokyo is the capital of Japan", "qualifiers": ["grounding_source"]}}, {"text": {"text": "What is 2+2?", "qualifiers": ["query"]}}, {"text": {"text": "Tokyo is the capital city of Japan, while some sources list Nagoya or Osaka as the largest city by population. London is the capital city of the United Kingdom, but some sources list Birmingham as the largest city by population.", "qualifiers": ["guard_content"]}}]}', 'url': 'https://bedrock-runtime.us-east-1.amazonaws.com/guardrail/ANONYMIZED/version/DRAFT/apply', 'context': {'client_region': 'us-east-1', 'client_config': <botocore.config.Config object at 0xANONYMIZED>, 'has_streaming_input': False, 'auth_type': None, 'unsigned_payload': None}}
2024-10-09 18:11:42,721 botocore.hooks [DEBUG] Event request-created.bedrock-runtime.ApplyGuardrail: calling handler <bound method RequestSigner.handler of <botocore.signers.RequestSigner object at 0xANONYMIZED>>
2024-10-09 18:11:42,722 botocore.hooks [DEBUG] Event choose-signer.bedrock-runtime.ApplyGuardrail: calling handler <function set_operation_specific_signer at 0xANONYMIZED>
2024-10-09 18:11:42,723 botocore.auth [DEBUG] Calculating signature using v4 auth.
2024-10-09 18:11:42,724 botocore.auth [DEBUG] CanonicalRequest:
POST
/guardrail/ANONYMIZED/version/DRAFT/apply

content-type:application/json
host:bedrock-runtime.us-east-1.amazonaws.com
x-amz-date:20241009T181142Z
x-amz-security-token:ANONYMIZED

content-type;host;x-amz-date;x-amz-security-token
ANONYMIZED
2024-10-09 18:11:42,724 botocore.auth [DEBUG] StringToSign:
AWS4-HMAC-SHA256
20241009T181142Z
20241009/us-east-1/bedrock/aws4_request
ANONYMIZED
2024-10-09 18:11:42,725 botocore.auth [DEBUG] Signature:
ANONYMIZED
2024-10-09 18:11:42,726 botocore.hooks [DEBUG] Event request-created.bedrock-runtime.ApplyGuardrail: calling handler <function add_retry_headers at 0xANONYMIZED>
2024-10-09 18:11:42,727 botocore.endpoint [DEBUG] Sending http request: <AWSPreparedRequest stream_output=False, method=POST, url=https://bedrock-runtime.us-east-1.amazonaws.com/guardrail/ANONYMIZED/version/DRAFT/apply, headers={'Content-Type': b'application/json', 'User-Agent': b'Boto3/1.35.31 md/Botocore#1.35.31 ua/2.0 os/linux#5.10.225-213.878.amzn2.x86_64 md/arch#x86_64 lang/python#3.10.14 md/pyimpl#CPython cfg/retry-mode#legacy Botocore/1.35.31', 'X-Amz-Date': b'20241009T181142Z', 'X-Amz-Security-Token': b'ANONYMIZED', 'Authorization': b'AWS4-HMAC-SHA256 Credential=ANONYMIZED/20241009/us-east-1/bedrock/aws4_request, SignedHeaders=content-type;host;x-amz-date;x-amz-security-token, Signature=ANONYMIZED', 'amz-sdk-invocation-id': b'ANONYMIZED', 'amz-sdk-request': b'attempt=1', 'Content-Length': '496'}>
2024-10-09 18:11:42,728 botocore.httpsession [DEBUG] Certificate path: /opt/conda/lib/python3.10/site-packages/certifi/cacert.pem
2024-10-09 18:11:42,729 urllib3.connectionpool [DEBUG] Starting new HTTPS connection (1): bedrock-runtime.us-east-1.amazonaws.com:443
2024-10-09 18:11:43,018 urllib3.connectionpool [DEBUG] https://bedrock-runtime.us-east-1.amazonaws.com:443 "POST /guardrail/ANONYMIZED/version/DRAFT/apply HTTP/1.1" 200 313
2024-10-09 18:11:43,020 botocore.parsers [DEBUG] Response headers: {'Date': 'Wed, 09 Oct 2024 18:11:43 GMT', 'Content-Type': 'application/json', 'Content-Length': '313', 'Connection': 'keep-alive', 'x-amzn-RequestId': 'ANONYMIZED'}
2024-10-09 18:11:43,023 botocore.parsers [DEBUG] Response body:
b'{"action":"NONE","assessments":[{}],"guardrailCoverage":{"textCharacters":{"guarded":228,"total":298}},"output":[],"outputs":[],"usage":{"contentPolicyUnits":0,"contextualGroundingPolicyUnits":0,"sensitiveInformationPolicyFreeUnits":0,"sensitiveInformationPolicyUnits":0,"topicPolicyUnits":0,"wordPolicyUnits":0}}'
2024-10-09 18:11:43,026 botocore.hooks [DEBUG] Event needs-retry.bedrock-runtime.ApplyGuardrail: calling handler <botocore.retryhandler.RetryHandler object at 0xANONYMIZED>
2024-10-09 18:11:43,027 botocore.retryhandler [DEBUG] No retry needed.
{'ResponseMetadata': {'RequestId': 'ANONYMIZED',
  'HTTPStatusCode': 200,
  'HTTPHeaders': {'date': 'Wed, 09 Oct 2024 18:11:43 GMT',
   'content-type': 'application/json',
   'content-length': '313',
   'connection': 'keep-alive',
   'x-amzn-requestid': 'ANONYMIZED'},
  'RetryAttempts': 0},
 'usage': {'topicPolicyUnits': 0,
  'contentPolicyUnits': 0,
  'wordPolicyUnits': 0,
  'sensitiveInformationPolicyUnits': 0,
  'sensitiveInformationPolicyFreeUnits': 0,
  'contextualGroundingPolicyUnits': 0},
 'action': 'NONE',
 'outputs': [],
 'assessments': [{}]}

[tim-finnigan]

Thanks for following up. Is your guardrail also in the us-east-1 region? I found a comment (aws/aws-sdk-js-v3#6482 (comment)) from someone using the JS SDK saying that using the wrong region was causing the same error.

Are you able to invoke the model successfully without specifying a guardrail? Also I didn't see the ValidationException in your error logs — can you share logs where that error is present?

[dannielvieira-hotmart]

Sorry again for the wait.

It appears that both the model and the guardrail are in us-east-1. I'm invoking both using this clients:

import boto3

session = boto3.Session(aws_access_key_id='KEY,
    aws_secret_access_key='KEY',
    aws_session_token='TOKEN'
)

bedrock = session.client('bedrock','us-east-1')
bedrock_runtime = session.client('bedrock-runtime','us-east-1')

Yes, when invoking the model without guardrails it work fine, but when calling with the guardrail it gives the ValidationException.
Sorry, didnt paste the entire error:

---------------------------------------------------------------------------
ValidationException                       Traceback (most recent call last)
Cell In[18], line 1
----> 1 response = bedrock_runtime.invoke_model(
      2     body=body,
      3     modelId='ai21.j2-mid-v1',
      4     guardrailIdentifier='5cyzlluxgzfp')

File /opt/conda/lib/python3.10/site-packages/botocore/client.py:569, in ClientCreator._create_api_method.<locals>._api_call(self, *args, **kwargs)
    565     raise TypeError(
    566         f"{py_operation_name}() only accepts keyword arguments."
    567     )
    568 # The "self" in this scope is referring to the BaseClient.
--> 569 return self._make_api_call(operation_name, kwargs)

File /opt/conda/lib/python3.10/site-packages/botocore/client.py:1023, in BaseClient._make_api_call(self, operation_name, api_params)
   1019     error_code = error_info.get("QueryErrorCode") or error_info.get(
   1020         "Code"
   1021     )
   1022     error_class = self.exceptions.from_code(error_code)
-> 1023     raise error_class(parsed_response, operation_name)
   1024 else:
   1025     return parsed_response

ValidationException: An error occurred (ValidationException) when calling the InvokeModel operation: Guardrail was enabled but input is in incorrect format.

[tim-finnigan]

Thanks for following up — can you share full debug logs (with any sensitive info redacted) when that error occurs? If we can see that API response then we can share that with the Bedrock team for further review.

[alps000alps]

I also got the same error message. The below snippet from documentation clearly states the possibility of error and I used guardrailVersion variable along with the invoke_model api

       response = bedrock.invoke_model(
        body=json.dumps(body),
        modelId="anthropic.claude-3-sonnet-20240229-v1:0",
        contentType="application/json",
        accept="application/json",
        guardrailIdentifier = guardrail_id,
        guardrailVersion=guardrail_version
    )

Using guardrailVersion resolved the error.

Below is a snippet from the boto3 docs