The Capacity Planner

The Capacity Planner: "how many workers do we need?" is not a feeling, it's Little's Law - L = lambda x W. The journal already holds W (task durations, as percentiles across runs); give the planner your target arrival rate and it computes the lanes, then checks the answer against every limit you've configured - because the binding constraint is usually not the one in the meeting.

source on github

a real captured run

CAPACITY PLANNER (target: 120 tickets/min at peak)

  task             p50        p95        lanes needed (p50/p95)
  fetch:ticket         83ms      158ms    1 / 1  #
  classify            364ms      751ms    1 / 2  ##
  draft:reply         993ms     2387ms    2 / 5  #####

  plan for p95, not p50: capacity sized to the median is capacity
  that queues every time latency has a bad day, and latency has a
  bad day 1 run in 8 in this journal. total lanes at p95: 8.

  the plan vs. what's actually configured:
    limit                                have     verdict at 120/min
    orchestrator concurrency_limit       8        holds
    provider quota (windowed)            90/min   BINDS - 90/min < 120/min arrivals, queues grow without bound
    connection pool ceiling              12       holds

  the meeting was about to argue concurrency_limit; the math says
  the provider QUOTA binds first - 90/min against 120/min arrivals
  isn't a slowdown, it's an unbounded queue (utilization > 1 has
  no steady state). fix the quota; the 8 lanes and the pool
  already hold. Little's Law plus a journal is a capacity plan;
  a dashboard plus a feeling is a postmortem.

source

# frozen_string_literal: true

# The Capacity Planner: "how many workers do we need?" is not a
# feeling, it's Little's Law - L = lambda x W. The journal already
# holds W (task durations, as percentiles across runs); give the
# planner your target arrival rate and it computes the lanes, then
# checks the answer against every limit you've configured - because
# the binding constraint is usually not the one in the meeting.
#
#   bundle exec ruby examples/capacity_planner.rb
#
# Runs offline; history is seeded into a journal first.

require class="s">"bundler/setup"
require class="s">"agentic"
require class="s">"tmpdir"

Agentic.logger.level = class="y">:fatal

# --- build history: 30 journaled runs of the pipeline ---------------------------
JOURNAL = File.join(Dir.tmpdir, class="s">"agentic_capacity.journal.jsonl")
File.delete(JOURNAL) if File.exist?(JOURNAL)
journal = Agentic:class="y">:ExecutionJournal.new(path: JOURNAL)
rng = Random.new(1123)

30.times do
  {class="s">"fetch:ticket" => 0.08, class="s">"classify" => 0.35, class="s">"draft:reply" => 0.9}.each do |name, base|
    # log-normal-ish: mostly base, occasionally 2-3x - like real latency
    duration = base * (0.8 + rng.rand(0.4)) * ((rng.rand < 0.12) ? (2 + rng.rand) : 1)
    journal.record(class="y">:task_succeeded, task_id: name, description: name, duration: duration.round(4), output: nil)
  end
end

state = Agentic:class="y">:ExecutionJournal.replay(path: JOURNAL)

TARGET_PER_MINUTE = 120 # tickets per minute at peak
CONFIGURED = {
  class="s">"orchestrator concurrency_limit" => 8,
  class="s">"provider quota (windowed)" => class="s">"90/min",
  class="s">"connection pool ceiling" => 12
}.freeze

puts class="s">"CAPACITY PLANNER (target: #{TARGET_PER_MINUTE} tickets/min at peak)"
puts
puts format(class="s">"  %-16s %-10s %-10s %-22s %s", class="s">"task", class="s">"p50", class="s">"p95", class="s">"lanes needed (p50/p95)", class="s">"")

lambda_per_sec = TARGET_PER_MINUTE / 60.0
total_p95_lanes = 0
state.duration_samples.keys.each do |task|
  p50 = state.duration_percentile(task, 50)
  p95 = state.duration_percentile(task, 95)
  # Little's Law: concurrent-in-service L = arrival rate x service time
  lanes_p50 = (lambda_per_sec * p50).ceil
  lanes_p95 = (lambda_per_sec * p95).ceil
  total_p95_lanes += lanes_p95
  puts format(class="s">"  %-16s %6.0fms   %6.0fms   %2d / %-2d %s",
    task, p50 * 1000, p95 * 1000, lanes_p50, lanes_p95, class="s">"#" * lanes_p95)
end

puts
puts class="s">"  plan for p95, not p50: capacity sized to the median is capacity"
puts class="s">"  that queues every time latency has a bad day, and latency has a"
puts class="s">"  bad day 1 run in 8 in this journal. total lanes at p95: #{total_p95_lanes}."
puts

# --- check the plan against every configured limit -------------------------------
puts class="s">"  the plan vs. what's actually configured:"
puts format(class="s">"    %-36s %-8s %s", class="s">"limit", class="s">"have", class="s">"verdict at #{TARGET_PER_MINUTE}/min")
verdicts = {
  class="s">"orchestrator concurrency_limit" => (total_p95_lanes <= 8) ? class="s">"holds" : class="s">"BINDS FIRST - raise to #{total_p95_lanes}",
  class="s">"provider quota (windowed)" => (TARGET_PER_MINUTE <= 90) ? class="s">"holds" : class="s">"BINDS - 90/min < #{TARGET_PER_MINUTE}/min arrivals, queues grow without bound",
  class="s">"connection pool ceiling" => (total_p95_lanes <= 12) ? class="s">"holds" : class="s">"BINDS - #{total_p95_lanes} lanes want connections"
}
CONFIGURED.each do |name, have|
  puts format(class="s">"    %-36s %-8s %s", name, have, verdicts[name])
end

puts
puts class="s">"  the meeting was about to argue concurrency_limit; the math says"
puts class="s">"  the provider QUOTA binds first - 90/min against 120/min arrivals"
puts class="s">"  isn't a slowdown, it's an unbounded queue (utilization > 1 has"
puts class="s">"  no steady state). fix the quota; the #{total_p95_lanes} lanes and the pool"
puts class="s">"  already hold. Little's Law plus a journal is a capacity plan;"
puts class="s">"  a dashboard plus a feeling is a postmortem."