Cloudflare Docs
AI Gateway
Cloudflare Docs
AI Gateway
Edit this page
Give us feedback
Set theme to dark (⇧+D)
  1. Products
  2. AI Gateway
  3. Supported providers
  4. Workers AI

Workers AI

Use AI Gateway for analytics, caching, and security on requests to Workers AI.

​​ REST API

To interact with a REST API, update the URL used for your request:

  • Previous: https://api.cloudflare.com/client/v4/accounts/{account_id}/ai/run/{model_id}
  • New: https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_slug}/workers-ai/{model_id}

For these parameters:

​​ Examples

First, generate an API token with Workers AI Read access and use it in your request.

Request to Workers AI llama model
curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_slug}/workers-ai/@cf/meta/llama-3-8b-instruct \
 --header 'Authorization: Bearer {cf_api_token}' \
 --header 'Content-Type: application/json' \
 --data '{"prompt": "What is Cloudflare?"}'

Request to Workers AI text classification model
curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_slug}/workers-ai/@cf/huggingface/distilbert-sst-2-int8 \
  --header 'Authorization: Bearer {cf_api_token}' \
  --header 'Content-Type: application/json' \
  --data '{ "text": "Cloudflare docs are amazing!" }'

​​ OpenAI compatible endpoints

Workers AI supports OpenAI compatible endpoints for text generation (/v1/chat/completions) and text embedding models (/v1/embeddings). This allows you to use the same code as you would for your OpenAI commands, but swap in Workers AI easily.

Request to OpenAI compatible endpoint
curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_slug}/workers-ai/v1/chat/completions \
 --header 'Authorization: Bearer {cf_api_token}' \
 --header 'Content-Type: application/json' \
 --data '{
      "model": "@cf/meta/llama-3-8b-instruct",
      "messages": [
        {
          "role": "user",
          "content": "What is Cloudflare?"
        }
      ]
    }
'

​​ Worker

To include an AI Gateway within your Worker, add the gateway as an object in your Workers AI request.

export interface Env {
  AI: Ai;

}



export default {
  async fetch(request: Request, env: Env): Promise<Response> {
    const response = await env.AI.run(
      "@cf/meta/llama-3-8b-instruct",
      {
        prompt: "Why should you use Cloudflare for your AI inference?"
      }
      {
        gateway: {
          id: "{gateway_slug}",
          skipCache: false,
          cacheTtl: 3360
        }
      }
    );
    return new Response(JSON.stringify(response));
  },

} satisfies ExportedHandler<Env>;

Workers AI supports the following parameters for AI gateways:

  • id string
    • Name of your existing AI Gateway. Must be in the same account as your Worker.
  • skipCache boolean(default: false)
  • cacheTtl number