The Cost Estimator

The Cost Estimator: price the plan BEFORE running it - per-task token estimates times a pricing table - gate on budget, then reconcile estimate against actuals afterward. LLM plans spend real money; nobody should learn the bill from the invoice.

source on github

a real captured run

COST ESTIMATOR (budget: 60c)

  pre-flight estimate:
    classify tickets     small  ~ 40000 tokens  ~  1.6c
    summarize threads    small  ~120000 tokens  ~  4.8c
    draft responses      large  ~ 60000 tokens  ~ 36.0c
    review drafts        large  ~ 25000 tokens  ~ 15.0c
    TOTAL                                              57.4c

  GATE: under budget, proceeding.

  reconciliation (estimate vs actual):
    classify tickets     est   1.6c  actual   2.1c  (+32%)
    summarize threads    est   4.8c  actual   6.1c  (+27%)
    draft responses      est  36.0c  actual  49.8c  (+38%)
    review drafts        est  15.0c  actual  13.7c  (-9%)
    TOTAL                est  57.4c  actual  71.7c

  feed actuals back into est_tokens and next month's estimates
  stop being folklore. budgets want feedback loops, not faith.

source

# frozen_string_literal: true

# The Cost Estimator: price the plan BEFORE running it - per-task token
# estimates times a pricing table - gate on budget, then reconcile
# estimate against actuals afterward. LLM plans spend real money;
# nobody should learn the bill from the invoice.
#
#   bundle exec ruby examples/cost_estimator.rb [budget_cents]
#
# Runs offline; actual token usage is seeded simulation.

require class="s">"bundler/setup"
require class="s">"agentic"

# $/1M tokens, input+output blended for the demo
PRICING = {
  class="s">"small" => 0.40,
  class="s">"large" => 6.00
}.freeze

JOBS = {
  class="s">"classify tickets" => {model: class="s">"small", est_tokens: 40_000},
  class="s">"summarize threads" => {model: class="s">"small", est_tokens: 120_000},
  class="s">"draft responses" => {model: class="s">"large", est_tokens: 60_000},
  class="s">"review drafts" => {model: class="s">"large", est_tokens: 25_000}
}.freeze

def cents(model, tokens)
  (PRICING.fetch(model) * tokens / 1_000_000 * 100)
end

budget_cents = (ARGV.first || 60).to_i
rng = Random.new(20260707)

orchestrator = Agentic:class="y">:PlanOrchestrator.new(concurrency_limit: 2)
tasks = JOBS.map do |name, job|
  task = Agentic:class="y">:Task.new(
    description: name,
    agent_spec: {class="s">"name" => name, class="s">"instructions" => class="s">"spend wisely"},
    payload: job
  )
  orchestrator.add_task(task, agent: ->(t) {
    sleep(0.01)
    # Actuals drift from estimates, as actuals do (+/- up to 40%)
    drift = 0.8 + rng.rand * 0.6
    {tokens: (t.payload[class="y">:est_tokens] * drift).round}
  })
  task
end

# --- pre-flight: price the graph before any token is spent -------------------
estimate = orchestrator.graph[class="y">:tasks].values.sum { |t| cents(t.payload[class="y">:model], t.payload[class="y">:est_tokens]) }

puts class="s">"COST ESTIMATOR (budget: #{budget_cents}c)"
puts
puts class="s">"  pre-flight estimate:"
JOBS.each do |name, job|
  puts format(class="s">"    %-20s %-6s ~%6d tokens  ~%5.1fc", name, job[class="y">:model], job[class="y">:est_tokens],
    cents(job[class="y">:model], job[class="y">:est_tokens]))
end
puts format(class="s">"    %-20s %28s %5.1fc", class="s">"TOTAL", class="s">"", estimate)
puts

if estimate > budget_cents
  puts class="s">"  GATE: estimate exceeds budget - plan refused before spending a cent."
  puts class="s">"  (raise the budget or downgrade 'draft responses' to the small model)"
  exit 1
end
puts class="s">"  GATE: under budget, proceeding."
puts

# --- the run, then the reconciliation ----------------------------------------
result = orchestrator.execute_plan

puts class="s">"  reconciliation (estimate vs actual):"
total_actual = 0.0
tasks.each do |task|
  job = task.payload
  actual_tokens = result.results[task.id].output[class="y">:tokens]
  actual = cents(job[class="y">:model], actual_tokens)
  total_actual += actual
  est = cents(job[class="y">:model], job[class="y">:est_tokens])
  drift_pct = 100.0 * (actual - est) / est
  puts format(class="s">"    %-20s est %5.1fc  actual %5.1fc  (%+.0f%%)", task.description, est, actual, drift_pct)
end
puts format(class="s">"    %-20s est %5.1fc  actual %5.1fc", class="s">"TOTAL", estimate, total_actual)
puts
puts class="s">"  feed actuals back into est_tokens and next month's estimates"
puts class="s">"  stop being folklore. budgets want feedback loops, not faith."