Gemini

Configure Google Gemini as an LLM provider in agentgateway.

Before you begin

Set up an agentgateway proxy.

Set up access to Gemini

1. Save your Gemini API key as an environment variable. To retrieve your API key, log in to the Google AI Studio and select API Keys.

   export GOOGLE_KEY=<your-api-key>

2. Create a secret to authenticate to Google.

   kubectl apply -f - <<EOF
   apiVersion: v1
   kind: Secret
   metadata:
     name: google-secret
     namespace: kgateway-system
   type: Opaque
   stringData:
     Authorization: $GOOGLE_KEY
   EOF

3. Create an AgentgatewayBackend resource to configure an LLM provider that references the AI API key secret.

   kubectl apply -f- <<EOF
   apiVersion: agentgateway.dev/v1alpha1
   kind: AgentgatewayBackend
   metadata:
     name: google
     namespace: kgateway-system
   spec:
     ai:
       provider:
         gemini:
           model: gemini-2.5-flash-lite
     policies:
       auth:
         secretRef:
           name: google-secret
   EOF

   Review the following table to understand this configuration. For more information, see the API reference.

   Setting	Description
   ai.provider.gemini	Define the Gemini provider.
   gemini.model	The model to use to generate responses. In this example, you use the gemini-2.5-flash-lite model. For more models, see the Google AI docs.
   policies.auth	The authentication token to use to authenticate to the LLM provider. The example refers to the secret that you created in the previous step.

4. Create an HTTPRoute resource that routes incoming traffic to the AgentgatewayBackend. The following example sets up a route on the /openai path to the AgentgatewayBackend that you previously created. The URLRewrite filter rewrites the path from /openai to the path of the API in the LLM provider that you want to use, /v1/chat/completions.

   kubectl apply -f- <<EOF
   apiVersion: gateway.networking.k8s.io/v1
   kind: HTTPRoute
   metadata:
     name: google
     namespace: kgateway-system
   spec:
     parentRefs:
       - name: agentgateway
     rules:
     - matches:
       - path:
           type: PathPrefix
           value: /gemini
       backendRefs:
       - name: google
         group: agentgateway.dev
         kind: AgentgatewayBackend
   EOF

6. Send a request to the LLM provider API. Verify that the request succeeds and that you get back a response from the chat completion API.

   Cloud Provider LoadBalancerPort-forward for local testing

   curl -vik "$INGRESS_GW_ADDRESS/gemini" -H content-type:application/json  -d '{
     "model": "",
     "messages": [
      {"role": "user", "content": "Explain how AI works in simple terms."}
    ]
   }'

   curl -vik "localhost:8080/gemini" -H content-type:application/json  -d '{
     "model": "",
     "messages": [
      {"role": "user", "content": "Explain how AI works in simple terms."}
    ]
   }'

   Example output:

   {"id":"aGLEaMjbLp6p_uMPopeAoAc",
   "choices":
     [{"index":0,"message":{
         "content":"Imagine teaching a dog a trick.  You show it what to do, reward it when it's right, and correct it when it's wrong.  Eventually, the dog learns.\n\nAI is similar.  We \"teach\" computers by showing them lots of examples.  For example, to recognize cats in pictures, we show it thousands of pictures of cats, labeling each one \"cat.\"  The AI learns patterns in these pictures – things like pointy ears, whiskers, and furry bodies – and eventually, it can identify a cat in a new picture it's never seen before.\n\nThis learning process uses math and algorithms (like a secret code of instructions) to find patterns and make predictions.  Some AI is more like a dog learning tricks (learning from examples), and some is more like following a very detailed recipe (following pre-programmed rules).\n\nSo, in short: AI is about teaching computers to learn from data and make decisions or predictions, just like we teach dogs tricks.\n",
         "role":"assistant"
         },
      "finish_reason":"stop"
      }],
    "created":1757700714,
    "model":"gemini-1.5-flash-latest",
    "object":"chat.completion",
    "usage":{
        "prompt_tokens":8,
        "completion_tokens":205,
        "total_tokens":213
        }
   }

Next steps

• Want to use other endpoints than chat completions, such as embeddings or models? Check out the multiple endpoints guide.

• Explore other guides for LLM consumption, such as function calling, model failover, and prompt guards.