If you’re building any system with many users, external APIs, or sensitive operations, you must protect your backend from sudden surges in requests that can cause overload or facilitate brute-force attacks. This is where Rate Limiting comes in.
What is Rate Limiting?
Rate Limiting is a technique used to control the number of requests a user, IP address, or even a specific endpoint can make within a defined time period.
You can set limits based on:
- Per User: Each authenticated user can make X requests per hour.
- Per IP Address: For unauthenticated users or to prevent IP-based attacks.
- Per Role: Different limits for different user roles (e.g., admins vs. regular users).
- Per Endpoint: Specific limits for sensitive or resource-intensive endpoints.
For example:
- “Each user can make 400 requests per hour.”
- “Each IP address can attempt 5 logins in 15 minutes.”
- “An instructor can upload a course once every 10 minutes.”
Rate limiting isn’t just about protection; it’s also about regulating resource consumption and improving overall system performance.
Naima’s Note: For AI-driven APIs, rate limiting is absolutely critical. Inference endpoints, especially for complex models, can be very resource-intensive. Without rate limiting, a single user or a malicious bot could overwhelm your AI service, leading to high latency, increased costs, and denial of service for legitimate users. It’s also essential for fair usage and preventing abuse of valuable AI resources.
Common Rate Limiting Algorithms
1. Fixed Window
This is the simplest and most commonly used method. You define a fixed time window (e.g., 1 hour) and a maximum number of requests allowed within that window.
Example: “100 requests per hour.” If the window starts at 01:00, the counter resets at 02:00.
The Problem: The “burst” problem at window boundaries. Imagine a user makes 100 requests at 01:59:50. The window resets at 02:00:00, and they can immediately make another 100 requests at 02:00:10. This means they effectively made 200 requests in a very short period (around 20 seconds), even though your limit was 100 per hour.
graph TD
A[Window 1: 01:00 - 02:00] --> B{100 Requests};
B -- at 01:59:50 --> C[Window Boundary];
C --> D[Window 2: 02:00 - 03:00];
D --> E{100 Requests};
E -- at 02:00:10 --> F[Total: 200 requests in ~20s];
When to use it:
- Public
GETendpoints. - Lightweight operations.
- Non-critical endpoints.
- Internal APIs where abuse is less likely.
2. Sliding Window
This method tracks the number of requests over a rolling time window. It’s more accurate in preventing bursts.
Example: “3 requests in any 10-second period.”
Let’s trace requests:
- Request 1: Time = 5s (Allowed - first request)
- Request 2: Time = 7s (Allowed - only 1 request in the last 10s (from 0s to 7s))
- Request 3: Time = 12s (Allowed - 2 requests in the last 10s (from 2s to 12s))
- Request 4: Time = 14s (BLOCKED - 3 requests in the last 10s (from 4s to 14s))
sequenceDiagram
participant User
participant RateLimiter
User->>RateLimiter: Request (Time: 5s)
RateLimiter-->>User: Allowed (Count: 1 in last 10s)
User->>RateLimiter: Request (Time: 7s)
RateLimiter-->>User: Allowed (Count: 2 in last 10s)
User->>RateLimiter: Request (Time: 12s)
RateLimiter-->>User: Allowed (Count: 3 in last 10s)
User->>RateLimiter: Request (Time: 14s)
RateLimiter--xUser: BLOCKED (Count: 3 in last 10s, limit reached)
When to use it:
- Sensitive APIs like:
- Login attempts (to prevent brute-force attacks).
- Email confirmation.
- Payment processing.
- File uploads.
- Any endpoint that needs robust protection against spam or abuse.
Naima’s Final Word: Rate limiting is a fundamental building block for robust and secure APIs. It’s not just about preventing malicious attacks; it’s also about ensuring fair access to resources and maintaining the stability of your system under varying loads. For AI services, where computational resources can be expensive and model inference can be slow, effective rate limiting is crucial for managing costs, preventing abuse, and guaranteeing a consistent quality of service for all users. This proactive approach to system health is a core tenet of the 10xdev.blog philosophy.