Amazon Bedrock

Configure Amazon Bedrock as an LLM provider in agentgateway.

Before you begin

Set up an agentgateway proxy.
Make sure that your Amazon credentials have access to the Bedrock models that you want to use. You can alternatively use an AWS Bedrock API key.

Set up access to Amazon Bedrock

Store your credentials to access the AWS Bedrock API.

export AWS_ACCESS_KEY_ID="<aws-access-key-id>"
export AWS_SECRET_ACCESS_KEY="<aws-secret-access-key>"
export AWS_SESSION_TOKEN="<aws-session-token>"

Create a secret with your Bedrock API key. Optionally provide the session token.

kubectl create secret generic bedrock-secret \
  -n kgateway-system \
  --from-literal=accessKey="$AWS_ACCESS_KEY_ID" \
  --from-literal=secretKey="$AWS_SECRET_ACCESS_KEY" \
  --from-literal=sessionToken="$AWS_SESSION_TOKEN" \
  --type=Opaque \
  --dry-run=client -o yaml | kubectl apply -f -

Save the API key in an environment variable.

export BEDROCK_API_KEY=<insert your API key>

Create a Kubernetes secret to store your Amazon Bedrock API key.

kubectl apply -f- <<EOF
apiVersion: v1
kind: Secret
metadata:
  name: bedrock-secret
  namespace: kgateway-system
type: Opaque
stringData:
  Authorization: $BEDROCK_API_KEY
EOF

Create an resource to configure your LLM provider. Make sure to reference the secret that holds your credentials to access the LLM.

kubectl apply -f- <<EOF
apiVersion: agentgateway.dev/v1alpha1
kind: 
metadata:
  name: bedrock
  namespace: kgateway-system
spec:
  ai:
    provider:
      bedrock:
        model: "amazon.titan-text-lite-v1"
        region: "us-east-1"
  policies:
    auth:
      secretRef:
        name: bedrock-secret
EOF

Review the following table to understand this configuration. For more information, see the API reference.

Setting	Description
`ai.provider.bedrock`	Define the LLM provider that you want to use. The example uses Amazon Bedrock.
`bedrock.model`	The model to use to generate responses. In this example, you use the `amazon.titan-text-lite-v1` model. Keep in mind that some models support cross-region inference. These models begin with a `us.` prefix, such as `us.anthropic.claude-sonnet-4-20250514-v1:0`. For more models, see the AWS Bedrock docs.
`bedrock.region`	The AWS region where your Bedrock model is deployed. Multiple regions are not supported.
`policies.auth`	Provide the credentials to use to access the Amazon Bedrock API. The example refers to the secret that you previously created. To use IRSA, omit the `auth` settings.

Create an HTTPRoute resource to route requests through your agentgateway proxy to the Bedrock . Note that kgateway automatically rewrites the endpoint that you set up (such as /bedrock) to the appropriate chat completion endpoint of the LLM provider for you, based on the LLM provider that you set up in the resource.

kubectl apply -f- <<EOF                                             
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:       
  name: bedrock
  namespace: kgateway-system
spec:
  parentRefs:
    - name: agentgateway
      namespace: kgateway-system
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /bedrock
    backendRefs:
    - name: bedrock
      namespace: kgateway-system
      group: agentgateway.dev
      kind: 
EOF

Send a request to the LLM provider API. Verify that the request succeeds and that you get back a response from the chat completion API.

curl "$INGRESS_GW_ADDRESS/bedrock" -H content-type:application/json -d '{
    "model": "",
    "messages": [
      {
        "role": "user",
        "content": "You are a cloud native solutions architect, skilled in explaining complex technical concepts such as API Gateway, microservices, LLM operations, kubernetes, and advanced networking patterns. Write me a 20-word pitch on why I should use an AI gateway in my Kubernetes cluster."
      }
    ]
  }' | jq

curl "localhost:8080/bedrock" -H content-type:application/json  -d '{
    "model": "",
    "messages": [
      {
        "role": "user",
        "content": "You are a cloud native solutions architect, skilled in explaining complex technical concepts such as API Gateway, microservices, LLM operations, kubernetes, and advanced networking patterns. Write me a 20-word pitch on why I should use an AI gateway in my Kubernetes cluster."
      }
    ]
  }' | jq

Example output:

{
  "metrics": {
    "latencyMs": 2097
  },
  "output": {
    "message": {
      "content": [
        {
          "text": "\nAn AI gateway in your Kubernetes cluster can enhance performance, scalability, and security while simplifying complex operations. It provides a centralized entry point for AI workloads, automates deployment and management, and ensures high availability."
        }
      ],
      "role": "assistant"
    }
  },
  "stopReason": "end_turn",
  "usage": {
    "inputTokens": 60,
    "outputTokens": 47,
    "totalTokens": 107
  }
}

Next steps

Want to use other endpoints than chat completions, such as embeddings or models? Check out the multiple endpoints guide.

Explore other guides for LLM consumption, such as function calling, model failover, and prompt guards.

Anthropic