Implement rate limiting#25
Open
tishun wants to merge 1 commit into
Open
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #25 +/- ##
==========================================
- Coverage 98.29% 96.41% -1.89%
==========================================
Files 9 12 +3
Lines 764 1170 +406
==========================================
+ Hits 751 1128 +377
- Misses 13 42 +29
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Co-authored-by: MSC72m <msc72m@gmail.com>
tishun
force-pushed
the
feat/rate-limiting
branch
from
July 21, 2026 09:30
6c19a09 to
c0e46c3
Compare
tishun
marked this pull request as ready for review
July 21, 2026 10:13
|
Thanks a lot for including me and incorporating parts of my work into the new implementation. I really appreciate it! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
(This PR is co-authored by @MSC72m as part of his PR #20)
Add Redis-backed rate limiting
Adds a rate-limiting feature to the SDK, mirroring the existing caching design (DI-first, async-only, OpenTelemetry-aware).
What's new
rate_limit()DI dependency — per-route limits viaDepends(), keyed by a pluggable identifier (default: client IP). Scope defaults to the matched route template so path params share one bucket.RateLimitMiddleware— optional app-wide global limit +*-RateLimit-*/ IETFRateLimitheader injection. Wired viaFastAPIRedis(app).rate_limiting().RateLimitBackend— fixed-window counter with two atomic tiers:INCREX(Redis 8.8+) → LuaEVALfallback (Redis 2.6+). No non-atomic degrade; a scripting-less server takes the fail-open/closed path instead. Capability detection is memoized once per process on the shared pool state.rate.py):"100/minute",Rate(100, 60), or(100, 60).costweighting,skip_whenbypass, custom 429 builder, sync facade (SyncRateLimitBackend), and OTel spans/metrics.REDIS_RATE_LIMIT_*), docs, and unit + integration tests.Where to focus review
ratelimit_backend.py— correctness core. The Lua window-counter script, tier selection/memoization, and fail-open vs fail-closed semantics. This is the only place atomicity matters (counter read-modify-write).ratelimit.py::_apply_limit— shared helper extracted from the per-route dependency and the global middleware. The one intentional difference between the two callers is raise-RateLimitExceededvs. return-Response; everything else (skip → identify → hit → stash state → metric → 429) is unified. Confirm the two call sites stay behaviorally identical._build_key) — deliberately flat, no hash-tag, unlike the cache backend (which hash-tags eviction groups). Rationale: rate limiting only does single-key ops, so spreading counters across slots avoids a hot shard.Retry-After+ limit headers.Linked issues