RateLimiter

All requests made through web interface or API are rate-limited - each user (or IP address when no user is logged in) is allowed to perform only a limited number of requests per time interval (by default 1 request per second with bursts of up to 60 requests at once).

Behavior

It's based on token bucket algorithm: each user has a bucket of tokens with limited capacity, each request consumes one token from the bucket (some resource-demanding requests may consume more). When there are not enough tokens in the bucket, request is refused (or delayed, see below). Tokens are continually refilled at a configured rate. By default, bucket size is 60 and refill rate is 1 token per second, but both numbers can be changed, even on per-user basis.

There's also a an automatic slow-down of requests by artificially delaying responses when there's not enough tokens (only works with sequential reuqests, when multiple requests are performed in parallel, error 426 Rate-limit exceeded is returned if no tokens are remaining).

Implementation

Rate limiting in NERD is implemented by RateLimiter class (/NERDweb/ratelimit.py). One RateLimiter object should be instantiated for each resource with limited access (currently used only for web/API access in NERD; it is therefore instantiated only once in nerd_main.py).

For each user, there is a bucket_size and token_per_sec parameters, which specify initial/maximum number of tokens and the rate at which tokens are added. Numbers of tokens per each active user are stored in Redis database.

The algorithm is not implemeted naively - tokens are not added to Redis counters regularly. Only when a request is made by the user, the current number of tokens is computed from the last number and time from last request.

Algorithm - when a request is attempted:

• Get bucket_size and token_per_sec of the user - may be set in user database, default values from global configuratios are used if not; the values are cached in Redis (keys <user>:bs and <user>:tps)
• Try to load number of tokens and time of last request for the user (Redis keys <user>:c and <user>:t)
  • If they don't exist, set the number of tokens to user's bucket size
  • If they do exist, compute how many token there should be, based on current time, time of last query and last number of tokens
• The use of tokens works in two modes:
  • If the parameter wait = True, then
    • If the current number of tokens > cost (1 by default), return True (success).
      If current number of tokens < cost and current number of tokens > 0, the cost is deducted from the tokens, the current number of tokens and the current time are saved in Redis. After that, the process goes into sleep mode until the value of the tokens is greater than or equal to 0. Return True (success). Otherwise, return False (rate-limit exceeded)
  • If the parameter wait = False, then
    • If current number of tokens > cost (1 by default), return True (success), otherwise return False (rate-limit exceeded) Store current number of tokens and current time into Redis

Rate-limiting is configured in nerdweb.yml, group rate-limit. It contains:

• Redis connection parameters (hostname, port, database index). It's recommended to use separate database (DB index or Redis instance) for these purposes.
• Default bucket_size and token_per_sec.

User-specific bucket_size and token_per_sec can be configured in user database - fields rl_bs and rl_tps.

To disable rate-limiting for some account, set rl_tps to infinity ('Infinity'::float in PSQL).