The Burst Absorber

The Burst Absorber: three waves of requests slam a credential with a ceiling of 3 (Agentic::RateLimit - this round's release). The ceiling holds, the queue absorbs, and the per-request wait times show exactly what "absorbed" costs. Capacity planning is reading this table.

source on github

a real captured run

BURST ABSORBER: ceiling 3, calls take 50ms

  wave 1: 6 requests   wait p50  50ms   worst  50ms
  wave 2: 2 requests   wait p50   0ms   worst   0ms
  wave 3: 9 requests   wait p50  52ms   worst 102ms

  high-water mark: 3 of 3 - the ceiling held through every burst

wave 1 (6 into 3 slots) queues; wave 2 (2 requests) sails through;
wave 3 (9 at once) pays the real price. the ceiling converts
provider 429s into local queueing - visible, measurable, bounded.

source

# frozen_string_literal: true

# The Burst Absorber: three waves of requests slam a credential with a
# ceiling of 3 (Agentic::RateLimit - this round's release). The ceiling
# holds, the queue absorbs, and the per-request wait times show exactly
# what "absorbed" costs. Capacity planning is reading this table.
#
#   bundle exec ruby examples/burst_absorber.rb
#
# Runs offline; the upstream is sleep.

require class="s">"bundler/setup"
require class="s">"agentic"
require class="s">"async"

CEILING = 3
CALL_TIME = 0.05
WAVES = [6, 2, 9].freeze # requests arriving together, 120ms apart

limit = Agentic:class="y">:RateLimit.new(CEILING)
waits = Hash.new { |h, k| h[k] = [] }

puts class="s">"BURST ABSORBER: ceiling #{CEILING}, calls take #{(CALL_TIME * 1000).round}ms"
puts

Sync do |task|
  arrivals = []
  WAVES.each_with_index do |count, wave|
    arrivals << task.async do
      sleep(wave * 0.12) # the wave arrives
      count.times.map { |i|
        task.async do
          arrived = Process.clock_gettime(Process:class="y">:CLOCK_MONOTONIC)
          limit.acquire do
            waits[wave] << Process.clock_gettime(Process:class="y">:CLOCK_MONOTONIC) - arrived
            sleep(CALL_TIME)
          end
        end
      }.each(&class="y">:wait)
    end
  end
  arrivals.each(&class="y">:wait)
end

WAVES.each_with_index do |count, wave|
  wave_waits = waits[wave].sort
  puts format(class="s">"  wave %d: %d requests   wait p50 %3dms   worst %3dms",
    wave + 1, count, wave_waits[wave_waits.size / 2] * 1000, wave_waits.last * 1000)
end

puts
puts format(class="s">"  high-water mark: %d of %d - the ceiling held through every burst", limit.high_water, CEILING)
puts
puts class="s">"wave 1 (6 into 3 slots) queues; wave 2 (2 requests) sails through;"
puts class="s">"wave 3 (9 at once) pays the real price. the ceiling converts"
puts class="s">"provider 429s into local queueing - visible, measurable, bounded."