Rate limiting best practices

The following sections cover typical rate limiting configurations for common use cases. You can combine the provided example rules and adjust them to your own scenario.

The main use cases for rate limiting are the following:

• Enforce granular access control to resources. Includes access control based on criteria such as user agent, IP address, referrer, host, country, and world region.
• Protect against credential stuffing and account takeover attacks.
• Limit the number of operations performed by individual clients. Includes preventing scraping by bots, accessing sensitive data, bulk creation of new accounts, and programmatic buying in ecommerce platforms.
• Protect REST APIs from resource exhaustion (targeted DDoS attacks) and resources from abuse in general.
• Protect GraphQL APIs by preventing server overload and limiting the number of operations.

​​ Enforcing granular access control

​​ Limit by user agent

A common use case is to limit the rate of requests performed by individual user agents. The following example rule allows a mobile app to perform a maximum of 100 requests in 10 minutes. You could also create a separate rule limiting the rate for desktop browsers.

​​ Allow specific IP addresses or ASNs

Another use case when controlling access to resources is to exclude or include IP addresses or Autonomous System Numbers (ASNs) from a rate limiting rule.

The following example rule allows up to 10 requests per minute from the same IP address doing a GET request for /status, as long as the visitor’s IP address is not included in the partner_ips IP list.

​​ Limit by referrer

Some applications receive requests originated by other sources (for example, used by advertisements linking to third-party pages). You may wish to limit the number of requests generated by individual referrer pages to manage quotas or avoid indirect DDoS attacks.

This example rule requires Advanced Rate Limiting.

​​ Limit by destination host

SaaS applications or customers using Cloudflare SSL for SaaS might have thousands of hosts under the same zone, which makes creating individual rules per host impractical. To overcome this, you can create a rate limiting rule that uses the host as a counting characteristic.

The following example rule will track the rate of requests to the /login endpoint for each host:

This example rule requires Advanced Rate Limiting.

​​ Protecting against credential stuffing

A typical use case of rate limiting is to protect a login endpoint. The following example contains three different rate limiting rules with increasing penalties to manage clients making too many requests.

Rule #1

Rule #2

Rule #3

Rule #1 allows up to four requests per minute, after which a Managed Challenge is triggered. This configuration allows legitimate customers a few attempts to remember their password. If an automated actor makes several requests, that client will likely be blocked by an unsolved Managed Challenge. On the other hand, if a human gets and passes the challenge when reaching rule #1’s rate limit, rule #2 will provide the next level of protection, allowing for up to 10 requests over the next 10 minutes. For clients exceeding this second threshold, rule #3 (the most severe) will apply, blocking the client for one day.

These three rules have a counting expression separate from the rule expression (also known as mitigation expression). When you configure a separate counting expression, the matching criteria will only be used when an action is triggered. In the counting expression you can include conditions based on the HTTP response status code and HTTP response headers, therefore integrating rate limiting with your backend logic.

You can also decide to have two different expressions — a counting expression and a rule/mitigation expression — to define:

1. The requests used to compute the rate.
2. The requests actually acted upon.

For example, rule #3 computes the rate considering POST requests to /login that returned a 401 or 403 HTTP status code. However, when the rate limit is exceeded, Cloudflare blocks every request to the example.com host generated by the same IP. For more information on counting expressions, refer to How Cloudflare determines the request rate.

​​ Limiting the number of operations

You can use rate limiting to limit the number of operations performed by a client. The exact rule providing this protection will depend on your application. The following examples address content scraping via query string parameters or JSON body.

​​ Prevent content scraping (via query string)

In this example, clients perform operations (such as looking up prices and adding to basket) on an ecommerce website using different query string parameters. For example, a typical request sent by a client could be similar to the following:

GET https://store.com/merchant?action=lookup_price&product_id=215
Cookie: session_id=12345

Your security team might want to consider setting up a limit on the number of times a client can lookup prices to prevent bots — which may have eluded Cloudflare Bot Management — from scraping the store’s entire catalog.

Rule #1

Rule #2

These two rate limiting rules match requests performing a selected action (look up price, in this example) and use IP as the counting characteristic. Similarly to the previous /login example, the two rules will help reduce false positives in case of persistent (but legitimate) visitors.

To limit the lookup of a specific product_id via query string parameter, you could add that specific query parameter as a counting characteristic, so that the rate is calculated based on all the requests, regardless of the client. The following example rule limits the number of lookups for each product_id to 50 requests in 10 seconds.

This example rule requires Advanced Rate Limiting.

You could follow the same pattern of rate limiting rules to protect applications handling reservations and bookings.

​​ Prevent content scraping (via body)

Consider an application that handles the operation and its parameters through the request body in JSON format. For example, the lookup_price operation could look like the following:

POST https://api.store.com/merchant
Cookie: session_id=12345

Body:
{
  "action": "lookup_price",
  "product_id": 215
}

In this scenario, you could write a rule to limit the number of actions from individual sessions:

This example rule requires Advanced Rate Limiting and payload inspection.

You could also limit the number of lookups of each product_id regardless of the client making the requests by deploying a rule like the following:

This example rule requires Advanced Rate Limiting and payload inspection.

​​ Limit requests from bots

A general approach to identify traffic from bots is to rate limit requests that trigger a large volume of 403 or 404 response status codes from the origin server. This usually indicates automated activity from scraping applications.

In this situation, you could configure a rule similar to the following:

To control the rate of actions performed by automated sources, consider use rate limiting rules together with Cloudflare Bot Management. With Bot Management, you can use the bot score as part of the matching criteria to apply the rule only to automated or likely automated traffic. For example, you can use a maximum score (or threshold) of 30 for likely automated traffic and 10 for automated traffic.

If your application tracks sessions using a cookie, you can use the cookie to set the rate limiting context (that is, use it as a counting characteristic). By setting the rate limiting characteristic to Cookie, the rule will group together requests from different IP addresses but belonging to the same session, which is a common scenario when dealing with a bot network performing a distributed attack.

Rule #1

Rule #2

These example rules require Advanced Rate Limiting and Bot Management.

If the application does not use a session cookie, you can use JA3 fingerprints to identify individual clients. A JA3 fingerprint is a unique identifier, available to customers with Bot Management, that allows Cloudflare to identify requests coming from the same client. All clients have an associated fingerprint, whether they are automated or not.

This example rule requires Advanced Rate Limiting and Bot Management.

​​ Protecting REST APIs

APIs can put significant strain on the application backend because API requests can be expensive to compute or serve. These requests may also require complex operations (such as data processing and large data lookups) that, if abused, can eventually bring down an origin server.

​​ Prevent volumetric attacks

Advanced Rate Limiting can mitigate many types of volumetric attacks, like DDoS attacks, mass assignment, and data exfiltration.

A common concern is to limit POST actions. For authenticated traffic, you can use API Discovery to identify a suitable rate of request per endpoint, and then create a rate limiting rule like the following:

This example rule requires Advanced Rate Limiting.

The counting characteristic can be any header, key, token, cookie, query parameter, or even JSON body field, since some APIs include a session ID or user ID as part of the JSON body. Refer to the following sections for additional information:

• If your unique identifier is in the URI path, refer to Protect resources.
• If your unique identifier is in the JSON body, refer to Prevent content scraping (via body).

​​ Protect resources

GET requests can also create excessive strain on an application or have an impact on costly resources, such as bandwidth. For example, consider an application with a large amount of stored files (such as images) where clients can download a file by accessing their specific URL:

GET https://api.store.com/files/<FILE_ID>
Header: x-api-key=9375

You probably wish to limit the number of downloads to avoid abuse, but you do not want to write individual rules for each file, given the size of the data storage. In this case, you could write a rule such as the following:

This example rule requires Advanced Rate Limiting.

The rule defines a limit of 10 downloads in 10 minutes for every file under https://api.store.com/files/*. By using Path as the rule characteristic, you avoid having to write a new rule every time there is a new uploaded file with a different <FILE_ID>. With this rule, the rate is computed on every request, regardless of their source IP or session identifier.

You could also combine Path with the x-api-key header (or IP, if you do not have a key or token) to set the maximum number of downloads that a specific client, as identified by x-api-key, can make of a given file:

This example rule requires Advanced Rate Limiting.

​​ Protecting GraphQL APIs

One of the biggest challenges posed by applications built on GraphQL is that a single path manages all queries to the server, and every request is usually a POST operation.

The purpose of the request is usually embedded in the body, which has information on what data the client wants to fetch or mutate (according to GraphQL’s terminology Open external link for server-side data modification), along with any additional data required to carry out the action.

The following examples are based on an application that accepts reviews for movies. A GraphQL request could look like the following:

POST https://moviereviews.example.com/graphql
Cookie: session_id=12345

Body:
{
  "data": {
    "createReview": {
      "stars": 5,
      "commentary": "This is a great movie!"
    }
  }
}

​​ Limit the number of operations

To limit the rate of actions, you could use the following rule:

This example rule requires Advanced Rate Limiting and payload inspection.

​​ Prevent server overload

The complexity necessary to handle a GraphQL request can vary significantly. Since the API uses a single endpoint, it is difficult to figure out the complexity of each request before it has been served.

To protect the origin server from resource exhaustion, rather than limiting the number of requests you need to limit the amount of complexity necessary to handle a single client over a period of time. Cloudflare Rate Limiting allows you to create rules that track complexity over time and block subsequent requests after reaching a complexity budget or limit.

This type of rate limiting requires that the server scores every served request according to the request’s complexity. Additionally, the server must add this score to the response as an HTTP header. Then, the rate limiting mechanism will use this information to update the budget for that specific client.

For example, the following rule defines a total complexity budget of 1,000 per hour:

This example rule requires Advanced Rate Limiting and payload inspection.

When the origin server processes a request, it adds a score HTTP header to the response with a value representing how much work the origin has performed to handle it — for example, 400. In the next hour, the same client can perform requests up to an additional budget of 600. As soon as this budget is exceeded, later requests will be blocked until the timeout expires.