Guide to: Configuring and Enabling a Model

• Bedrock Model Access
• Lambda Function Set-up
• API Gateway Configuration

Bedrock Model Access

• • Make sure you have access to the bedrock model by checking this url
  • You may need to adjust the region, depending on where you want the model to be hosted

Lambda Function Set-up

1. • Once your access to Amazon Bedrock is established you will need to create a Lambda Function to route requests to the bedrock api using boto3
   • Add Lambda Layer - boto3 does not have bedrock-runtime in Lambda by default, so you will need to add a lambda layer that has an updated version of boto3
   • Set your lambda function to use a python runtime, and upload the layer
   • Copy the following code to your Lambda Function (Note the below example is specific to Anthropic Claude 3; refer to the relevant AWS documentation for each individual model's inference paremeters and make changes to the code accordingly) 

import json 
import boto3 
import os 
# Initialize the Bedrock client 
bedrock = boto3.client(service_name="bedrock-runtime", region_name="us-east-1") 
def lambda_handler(event, context):
    print(event) 
    api_key = os.environ['API_KEY'] 
    auth = event.get('headers').get('authorization') 
    if auth != f'Bearer {api_key}': 
        return 
            { 'statusCode': 401, 
            'body': "Invalid API key." 
        } 
    # Extract parameters from the event 
    input_payload = json.loads(event.get('body')) 
    modelId = input_payload['model'] 
    temperature = float(input_payload.get('temperature', 0.0)) 
    topP = float(input_payload.get('topP', 0.9)) 
    maxTokenCount = int(input_payload.get('maxTokenCount', 300)) 
    input_messages = input_payload['messages'] 
    
    # Prepare the request body with the input message 
    # NOTE: this is where changes might be required depending on your chosen model's inference parameters 
    body=json.dumps( 
        { 
            "anthropic_version": "bedrock-2023-05-31", 
            "max_tokens": maxTokenCount, 
            "messages": input_messages 
        } 
    ) 
    accept = "application/json" 
    contentType = "application/json" 
    # Invoke the Bedrock model 
    response = bedrock.invoke_model( 
        body=body, modelId=modelId, accept=accept, contentType=contentType 
    ) 

    # Process the response from the model 
    response_body = json.loads(response['body'].read().decode()) 

    # Return the model's response 
    return { 
        'statusCode': 200, 
        'body': json.dumps(response_body) 
    }

• in the lambda configuration, add an environment variable with your API key to prevent public access to the API once deployed. WARNING: this is not best practice for authentication especially if the AWS account is shared with many users.

• also in the configuration, go to the Permissions tab and click on the role name. This will take you to the IAM console, where you can add permissions to access bedrock. Use the aws-managed AmazonBedrockFullAccessPolicy, or use an inline policy if you want more fine-grained accesses defined.

• back in the lambda configuration page, under "General Configuration", increase the "Timeout" to ~1min+. The LLMs can take some time to generate responses, so this is required to avoid timing out.

API Gateway Configuration

Once the Lambda configuration is complete, it must be linked with an API in API Gateway

• In the API Gateway console, create a new HTTP API.

• create a new integration to the lambda function you just created, and give the API a name

• in Step 2, leave the method as ANY, or set to POST, if you wish to use other methods with the same resource path

• Step 3 is optional, add stages or skip the step
• Finally, if you do define stages you'll need to deploy the API, if you are using the default stage, it should be set to auto-deploy by default
• test your API in Postman; first, enter your "authorization" header with value "Bearer {YOUR_API_KEY}"

• then format the body of the request and send it to the endpoint you configured in API Gateway

• Now you can utilize this to access models in Bedrock from any of our AI tools by configuring them in the model configuration page. Here is an example of what your configuration might look like for an Anthropic Claude 3 model; although keep in mind that the input and output schemas will differ according to your models' request and response formats.

{
    "NAME": "bedrock",
    "ENDPOINT": "YOUR_NEW_ENDPOINT",
    "DESCRIPTION": "bedrock-completion",
    "HEADER": {"Content-Type":"application/json","Authorization":"Bearer ${secret}"},
    "SECRET": "YOUR_API_KEY",
    "HTTPMETHOD": "POST",
    "INPUTSCHEMA": {
        "payload": {
            "model": "anthropic.claude-3-sonnet-20240229-v1:0 | YOUR_MODEL",
            "messages": "${message_items}",
            "temperature": "0.0",
            "topP": "0.9",
            "maxTokenCount": "300"
        },
        "message_item": {
            "role": "${role}",
            "content": "${message}"
       }
    },
    "OUTPUTSCHEMA": {
        "type": "multi-result",
        "results": "/content",
        "text": "/text"
    },
    "METHODPARAM": "REQUEST_BODY"
}

Create an API key by going to this url The main difference from the standard model configuration is the inclusion of an image_url parameter in the input schema:

    {
        "type": "image_url",
        "image_url": {
          "url": "${image_url}"
        }
    }

• this addition allows you to send both a prompt and an image to the model
• Navigate to the model configuration page and fill out the following info to successfully configure the model:

{
    "NAME": "gpt-4-turbo",
    "ENDPOINT": "https://api.openai.com/v1/chat/completions",
    "DESCRIPTION": "gpt-4-turbo-completion",
    "HEADER": {"Content-Type":"application/json","Authorization":"Bearer ${secret}"},
    "SECRET": "YOUR_API_KEY",
    "HTTPMETHOD": "POST",
    "INPUTSCHEMA": {
        "payload": {
            "model": "gpt-4-turbo",
            "messages": "${message_items}",
            "temperature": 0.7
    },
    "message_item": {
        "role": "user",
        "content": [
        {
            "type": "text", 
            "text": "${message}"
        },
        {
            "type": "image_url",
            "image_url": {
            "url": "${image_url}"
            }
          } 
        ]
      }
    },
    "OUTPUTSCHEMA": {
       "type": "multi-result",
       "results": "/content",
       "text": "/text"
    },
    "METHODPARAM": "REQUEST_BODY"
}