The Quota Keeper

The Quota Keeper: the same 20 requests through two different laws. A concurrency ceiling ("3 in flight") models connection limits; a windowed quota ("5 per 200ms") models what providers actually bill. They are different physics, and the admission timeline proves it.

source on github

a real captured run

QUOTA KEEPER: 20 requests, 10ms each, fired simultaneously

  law 1 - concurrency ceiling (3 in flight):
      0-200ms  ####################   20
    all admitted by 62ms - completion frees a slot, so short
    calls drain the queue as fast as they finish

  law 2 - windowed quota (5 per 200ms):
      0-200ms  #####                  5
    200-400ms  #####                  5
    400-600ms  #####                  5
    600-800ms  #####                  5
    last admitted at 604ms - finishing early buys NOTHING;
    the window admits five per period no matter how quick the calls

same requests, ~62ms versus ~604ms: pick the law your
provider actually enforces. connection pools are ceilings; billed
quotas are windows; production APIs are usually both at once -
which is why RateLimit lets you hold one of each.

source

# frozen_string_literal: true

# The Quota Keeper: the same 20 requests through two different laws.
# A concurrency ceiling ("3 in flight") models connection limits; a
# windowed quota ("5 per 200ms") models what providers actually bill.
# They are different physics, and the admission timeline proves it.
#
#   bundle exec ruby examples/quota_keeper.rb
#
# Runs offline; calls are 10ms of simulated IO.

require class="s">"bundler/setup"
require class="s">"agentic"
require class="s">"async"

REQUESTS = 20
CALL_TIME = 0.01

def fire_through(limit)
  admissions = []
  started = Process.clock_gettime(Process:class="y">:CLOCK_MONOTONIC)

  Sync do
    REQUESTS.times.map {
      Async do
        limit.acquire do
          admissions << Process.clock_gettime(Process:class="y">:CLOCK_MONOTONIC) - started
          sleep(CALL_TIME)
        end
      end
    }.each(&class="y">:wait)
  end

  admissions.sort
end

def admission_chart(admissions, bucket = 0.2)
  buckets = admissions.group_by { |t| (t / bucket).floor }
  (0..admissions.max / bucket).map { |i|
    format(class="s">"    %3d-%3dms  %-22s %d",
      i * bucket * 1000, (i + 1) * bucket * 1000, class="s">"#" * (buckets[i]&.size || 0), buckets[i]&.size || 0)
  }.join(class="s">"\n")
end

puts class="s">"QUOTA KEEPER: #{REQUESTS} requests, #{(CALL_TIME * 1000).round}ms each, fired simultaneously"
puts

concurrent = fire_through(Agentic:class="y">:RateLimit.new(3))
puts class="s">"  law 1 - concurrency ceiling (3 in flight):"
puts admission_chart(concurrent)
puts format(class="s">"    all admitted by %dms - completion frees a slot, so short", concurrent.last * 1000)
puts class="s">"    calls drain the queue as fast as they finish"
puts

windowed = fire_through(Agentic:class="y">:RateLimit.new(5, per: 0.2))
puts class="s">"  law 2 - windowed quota (5 per 200ms):"
puts admission_chart(windowed)
puts format(class="s">"    last admitted at %dms - finishing early buys NOTHING;", windowed.last * 1000)
puts class="s">"    the window admits five per period no matter how quick the calls"
puts

puts class="s">"same requests, ~#{(concurrent.last * 1000).round}ms versus ~#{(windowed.last * 1000).round}ms: pick the law your"
puts class="s">"provider actually enforces. connection pools are ceilings; billed"
puts class="s">"quotas are windows; production APIs are usually both at once -"
puts class="s">"which is why RateLimit lets you hold one of each."