The Escalation Ladder

The Escalation Ladder: the pattern under every AI product that survives contact with customers. The machine does the whole job it can PROVE it can do, and hands the rest up a ladder - tier 0 auto-resolves from playbooks, tier 1 drafts for known-but-nuanced intents, and the human queue takes the rest. Two rules make it a product instead of a demo: confidence thresholds are BUSINESS POLICY as data (not vibes in a prompt), and sensitivity TRUMPS confidence - the machine can be …

source on github

a real captured run

THE ESCALATION LADDER (do what you can prove; hand up the rest with a dossier)

  tier 0 - auto-resolved from playbooks:
    T-1 (password_reset, conf 0.9): Sent manual reset link; advised checking spam filters.
    T-4 (feature_request, conf 0.95): Logged +1 for the feature; shared the public roadmap.

  tier 1 - specialist drafts:
    T-2 (refund, conf 0.9): Drafted refund of the duplicate charge pending payment-ops review.
    T-5 (data_export, conf 0.95): Reproduced: BOM missing for Excel; sent UTF-8 BOM re-export steps.

  the human queue:
    T-3 (refund, conf 0.9): sensitive (0.9 confident, and it does not matter)
         dossier: "Refund me today or my lawyer files the charg"... triage + attempts attached
    T-6 (unknown, conf 0.0): low confidence (0.0)
         dossier: "it just doesnt work anymore??? nothing loads"... triage + attempts attached

  the two load-bearing rules: thresholds are DATA (0.8 and 0.5 live
  in a POLICY hash a product manager can read, diff, and A/B), and
  the sensitivity check outranks the confidence check in the
  routing order itself - T-3 scored confident-refund and still went
  human, because being sure is not the same as being allowed.
  escalation is not failure; it's the product working as designed -
  and every handoff carries the full dossier, so the human starts
  from everything the machine learned instead of a blank screen.

source

# frozen_string_literal: true

# The Escalation Ladder: the pattern under every AI product that
# survives contact with customers. The machine does the whole job it
# can PROVE it can do, and hands the rest up a ladder - tier 0
# auto-resolves from playbooks, tier 1 drafts for known-but-nuanced
# intents, and the human queue takes the rest. Two rules make it a
# product instead of a demo: confidence thresholds are BUSINESS
# POLICY as data (not vibes in a prompt), and sensitivity TRUMPS
# confidence - the machine can be 95% sure about a legal threat and
# 100% wrong to touch it. Escalation hands over a dossier, never a
# shrug.
#
#   bundle exec ruby examples/support_escalation.rb
#
# Runs offline; six tickets ride the ladder, exit 1 if any lands
# on the wrong rung.

require class="s">"bundler/setup"
require class="s">"agentic"
require class="s">"tmpdir"

Agentic.logger.level = class="y">:fatal

POLICY = {auto_resolve_at: 0.8, draft_at: 0.5}.freeze

TICKETS = [
  {id: class="s">"T-1", text: class="s">"I forgot my password and the reset email never arrives"},
  {id: class="s">"T-2", text: class="s">"I was charged twice this month, please refund the duplicate"},
  {id: class="s">"T-3", text: class="s">"Refund me today or my lawyer files the chargeback and we sue"},
  {id: class="s">"T-4", text: class="s">"Would love a dark mode! Any plans?"},
  {id: class="s">"T-5", text: class="s">"The export finished but the CSV opens garbled in Excel"},
  {id: class="s">"T-6", text: class="s">"it just doesnt work anymore??? nothing loads. fix it"}
].freeze

# Offline stand-in for the triage LLM: keyword scoring with an
# honest confidence and a sensitivity flag (money+threats, legal)
TRIAGE = ->(text) {
  signals = {
    password_reset: [class="s">"password", class="s">"reset"], refund: [class="s">"charged", class="s">"refund", class="s">"chargeback"],
    feature_request: [class="s">"love", class="s">"plans", class="s">"mode"], data_export: [class="s">"export", class="s">"csv", class="s">"garbled"]
  }
  scores = signals.transform_values { |words| words.count { |w| text.downcase.include?(w) } }
  intent, hits = scores.max_by { |_, v| v }
  {intent: (hits.zero? ? class="y">:unknown : intent),
   confidence: [hits * 0.45, 0.95].min.round(2),
   sensitive: text.match?(/lawyer|sue|legal|lawsuit/i)}
}

PLAYBOOKS = {
  password_reset: class="s">"Sent manual reset link; advised checking spam filters.",
  feature_request: class="s">"Logged +1 for the feature; shared the public roadmap."
}.freeze

SPECIALIST = ->(ticket, triage) {
  case triage[class="y">:intent]
  when class="y">:refund then class="s">"Drafted refund of the duplicate charge pending payment-ops review."
  when class="y">:data_export then class="s">"Reproduced: BOM missing for Excel; sent UTF-8 BOM re-export steps."
  end
}

journal = Agentic:class="y">:ExecutionJournal.new(path: File.join(Dir.tmpdir, class="s">"agentic_ladder.jsonl"))
File.delete(journal.path) if File.exist?(journal.path)
orchestrator = Agentic:class="y">:PlanOrchestrator.new(concurrency_limit: 6, lifecycle_hooks: journal.lifecycle_hooks)

routed = {}
TICKETS.each do |ticket|
  triage = Agentic:class="y">:Task.new(description: class="s">"triage #{ticket[class="y">:id]}", agent_spec: {class="s">"name" => class="s">"t", class="s">"instructions" => class="s">"w"}, payload: ticket)
  resolve = Agentic:class="y">:Task.new(description: class="s">"resolve #{ticket[class="y">:id]}", agent_spec: {class="s">"name" => class="s">"r", class="s">"instructions" => class="s">"w"})
  orchestrator.add_task(triage, agent: ->(t) { {ticket: t.payload, triage: TRIAGE.call(t.payload[class="y">:text])} })
  orchestrator.add_task(resolve, [triage], agent: ->(t) {
    dossier = t.previous_output
    verdict = dossier[class="y">:triage]
    tier, outcome =
      if verdict[class="y">:sensitive]
        # sensitivity trumps confidence, always - this branch is FIRST
        [class="y">:human, class="s">"sensitive (#{verdict[class="y">:confidence]} confident, and it does not matter)"]
      elsif verdict[class="y">:confidence] >= POLICY[class="y">:auto_resolve_at] && PLAYBOOKS[verdict[class="y">:intent]]
        [class="y">:auto, PLAYBOOKS[verdict[class="y">:intent]]]
      elsif verdict[class="y">:confidence] >= POLICY[class="y">:draft_at] && (draft = SPECIALIST.call(dossier[class="y">:ticket], verdict))
        [class="y">:specialist, draft]
      else
        [class="y">:human, class="s">"low confidence (#{verdict[class="y">:confidence]})"]
      end
    dossier.merge(tier: tier, outcome: outcome)
  })
  routed[ticket[class="y">:id]] = resolve
end

result = orchestrator.execute_plan
resolutions = routed.transform_values { |task| result.task_result(task.id).output }

puts class="s">"THE ESCALATION LADDER (do what you can prove; hand up the rest with a dossier)"
puts
[[class="y">:auto, class="s">"tier 0 - auto-resolved from playbooks"], [class="y">:specialist, class="s">"tier 1 - specialist drafts"], [class="y">:human, class="s">"the human queue"]].each do |tier, label|
  puts class="s">"  #{label}:"
  resolutions.select { |_, r| r[class="y">:tier] == tier }.each do |id, r|
    puts class="s">"    #{id} (#{r[class="y">:triage][class="y">:intent]}, conf #{r[class="y">:triage][class="y">:confidence]}): #{r[class="y">:outcome]}"
    if tier == class="y">:human
      puts class="s">"         dossier: #{r[class="y">:ticket][class="y">:text][0, 44].inspect}... triage + attempts attached"
    end
  end
  puts
end

failures = []
failures << class="s">"the legal threat was touched by a machine" unless resolutions[class="s">"T-3"][class="y">:tier] == class="y">:human
failures << class="s">"gibberish was not sent to a human" unless resolutions[class="s">"T-6"][class="y">:tier] == class="y">:human
failures << class="s">"the password reset should have auto-resolved" unless resolutions[class="s">"T-1"][class="y">:tier] == class="y">:auto
failures << class="s">"a ticket fell off the ladder" unless resolutions.values.all? { |r| r[class="y">:tier] }
failures << class="s">"a human item arrived without a dossier" if resolutions.values.any? { |r| r[class="y">:tier] == class="y">:human && !(r[class="y">:ticket] && r[class="y">:triage]) }

puts class="s">"  the two load-bearing rules: thresholds are DATA (0.8 and 0.5 live"
puts class="s">"  in a POLICY hash a product manager can read, diff, and A/B), and"
puts class="s">"  the sensitivity check outranks the confidence check in the"
puts class="s">"  routing order itself - T-3 scored confident-refund and still went"
puts class="s">"  human, because being sure is not the same as being allowed."
puts class="s">"  escalation is not failure; it's the product working as designed -"
puts class="s">"  and every handoff carries the full dossier, so the human starts"
puts class="s">"  from everything the machine learned instead of a blank screen."
exit(failures.empty? ? 0 : 1)