Repository navigation
Description
Is your feature request related to a problem? Please describe.
The current Sliding Window Rate Limiter implementation in Nostream is non-performant under heavy load (thousands of clients). Because this method requires storing individual request metadata or counters within a specific timeframe, it leads to high memory consumption in Redis and increased latency during window calculations. As the relay scales, the memory overhead becomes a significant bottleneck.
Describe the solution you'd like
I would like to extend the rate limiter interface to include an Exponentially Weighted Moving Average (EWMA) implementation. EWMA is a "memoryless" rate limiting algorithm that only requires storing two values per key (the current rate and a timestamp), making it significantly more performant and memory-efficient than sliding windows.
The EWMA Formula
The rate is updated upon each new request using an exponential decay function. When a request arrives, the new rate is calculated as:
Where:
-
$R_{new}$ is the updated rate. -
$R_{old}$ is the rate stored from the previous request. -
$\Delta t$ is the time elapsed since the last request. -
$\lambda$ is the decay constant, which is determined by the user-configurable half-life ($t_{1/2}$ ):
Requirements
- Configurability: Users must be able to configure the
half-lifein the relay settings. - Default Behavior: Nostream should use the new EWMA rate limiter by default.
- Legacy Support: Relay operators should be able to switch back to the
sliding_windowvia a configuration setting (e.g.,LIMITER_STRATEGY). - Expected Outcomes: * A new EWMA rate limiter implementation.
- Full unit test coverage for the decay logic.
- Integration tests ensuring the relay correctly respects the chosen strategy and limits.
Describe alternatives you've considered
- Token Bucket: While effective, it typically requires more complex Redis Lua scripts to handle atomic updates of tokens and timestamps compared to the mathematical simplicity of EWMA.
- Fixed Window: Too inaccurate as it allows double the rate limit at window boundaries.
Additional context
This change is critical for high-traffic relays where Redis memory usage is currently a primary cost and performance driver. By moving to a mathematical decay model, we can handle thousands of concurrent clients with near-constant memory overhead per user.
Activity
saniddhyaDubey commentedon Apr 9, 2026
assign!
archief2910 commentedon Apr 9, 2026
i would like to work on this issue
saniddhyaDubey commentedon Apr 12, 2026
Hey @cameri , completed the implementation of EWMA based rate-limiting. Need your preference on few things:
per requestoperation in a custom lua script for atomicity. Is that okay or does that violates the code structure -- theICacheAdapterhas a new function,hitEWMA, which runs this script ?Would love your opinion on this one to complete this issue and open the PR.
feat: implement EWMA rate limiter with strategy support (#404) (#476)
cameri commentedon Apr 19, 2026
@saniddhyaDubey amazing work, thank you!
feat: implement EWMA rate limiter with strategy support (cameri#404) (c…