Update bedrock_guardrails.py

[kothamah]

prioritized over the configuration parameters.

Title

litellm bedrock guardrails block content feature : Litellm need to block the PII content before sending to LLM.

This change is corressponds to the fix required for implementing guardrails and not share PII data with models.

EXT_ Re_ EXT_ Re_ EXT_ Re_ EXT_ Next Release.msg

[vercel]

@kothamah is attempting to deploy a commit to the CLERKIEAI Team on Vercel.

A member of the Team first needs to authorize it.

[CLAassistant]

All committers have signed the CLA.

[krrishdholakia]

please add testing for this @kothamah

[kothamah]

@krrishdholakia Thanks Krirsh for reviewing this. I have added test cases for the bedrock guardrail changes.

[kothamah]

@krrishdholakia @TeddyAmkie @ishaan-jaff This change is required for avoiding PII data to be sent to LLM. I was not be able to execute the updated code due to restrictions in our environment, but certainly can test if this change can be added to any LiteLLM version.

This is the output of the bedrock guardrails.

[92m17:59:03 - LiteLLM Proxy:DEBUG[0m: bedrock_guardrails.py:408 - Bedrock AI response : {'action': 'GUARDRAIL_INTERVENED', 'actionReason': 'Guardrail blocked.', 'assessments': [{'contentPolicy': {'filters': [{'action': 'BLOCKED', 'confidence': 'HIGH', 'detected': True, 'filterStrength': 'LOW', 'type': 'PROMPT_ATTACK'}]}, 'invocationMetrics': {'guardrailCoverage': {'textCharacters': {'guarded': 4430, 'total': 4430}}, 'guardrailProcessingLatency': 686, 'usage': {'automatedReasoningPolicies': 0, 'automatedReasoningPolicyUnits': 0, 'contentPolicyImageUnits': 0, 'contentPolicyUnits': 5, 'contextualGroundingPolicyUnits': 0, 'sensitiveInformationPolicyFreeUnits': 0, 'sensitiveInformationPolicyUnits': 5, 'topicPolicyUnits': 0, 'wordPolicyUnits': 5}}, 'sensitiveInformationPolicy': {'piiEntities': [{'action': 'NONE', 'detected': True, 'match': '[REDACTED]', 'type': 'NAME'}, {'action': 'NONE', 'detected': True, 'match': '[REDACTED]', 'type': 'PHONE'}, {'action': 'NONE', 'detected': True, 'match': '[REDACTED]', 'type': 'EMAIL'}]}}], 'blockedResponse': 'Sorry, your question in its current format is unable to be answered.', 'guardrailCoverage': {'textCharacters': {'guarded': 4430, 'total': 4430}}, 'output': [{'text': 'Sorry, your question in its current format is unable to be answered.'}], 'outputs': [{'text': 'Sorry, your question in its current format is unable to be answered.'}], 'usage': {'automatedReasoningPolicies': 0, 'automatedReasoningPolicyUnits': 0, 'contentPolicyImageUnits': 0, 'contentPolicyUnits': 5, 'contextualGroundingPolicyUnits': 0, 'sensitiveInformationPolicyFreeUnits': 0, 'sensitiveInformationPolicyUnits': 5, 'topicPolicyUnits': 0, 'wordPolicyUnits': 5}}

I have attached the screenshots from the existing code execution and updated the function to prioritize the blocking before reaching LLM.

[kothamah]

This is the configuration used.

guardrails:

• guardrail_name: "AMP_BEDROCK_GUARDRAILS"
  litellm_params:
  guardrail: bedrock # supported values: "aporia", "bedrock", "lakera"

  mode: ["pre_call", "post_call"]

  mode : "pre_call"
  guardrailIdentifier: os.environ/GUARDRAIL_ID # your guardrail ID on bedrock
  guardrailVersion: "DRAFT" # your guardrail version on bedrock
  aws_region_name: "us-east-1" # region guardrail is defined
  mask_request_content: true # Enable Masking in User Requests
  mask_response_content: true # Enable masking in model responses
  disable_exception_on_block: false # Prevents exceptions when content is blocked
  default_on: false

[krrishdholakia]

@kothamah sounds good, i just need you to add a test for your changes inside test_litellm/ so we don't have any future regressions.

[kothamah]

@krrishdholakia I have added test cases for the bedrock guardrail changes, so please check and let me know for any questions.