The Telemetry Bus

The Telemetry Bus: lifecycle hooks are callbacks - one producer, one consumer, coupled at configuration time. A telemetry bus inverts that: the orchestrator emits NAMED EVENTS into a bus, and any number of handlers attach, detach, and crash independently. The producer never learns who is listening. This is the :telemetry pattern Elixir converged on, because every library inventing its own instrumentation callbacks was the worse world.

source on github

a real captured run

TELEMETRY BUS (three handlers, one bridge, zero coupling)

  run 1 - all handlers attached:
    [trace] SLOW: enrich took 82ms
    [bus] handler exporter crashed (IOError) - detached, plan unharmed
    [metrics] {:tasks=>3}

  run 2 - tracer detached at runtime (ops got tired of it):
    [metrics] {:tasks=>6}

  the orchestrator emitted the same events both runs - it cannot
  tell that the tracer left or that the exporter crashed, and
  that ignorance is the feature. hooks couple one producer to
  one consumer at configuration time; a bus decouples N handlers
  at RUNTIME, with isolation (the Friday exporter died alone).
  event names are namespaced tuples, measurements are separated
  from metadata - steal the whole :telemetry design; it was
  right. the framework's hooks made the bridge ten lines, which
  is exactly what hooks are for: being the floor a bus stands on.

source

# frozen_string_literal: true

# The Telemetry Bus: lifecycle hooks are callbacks - one producer,
# one consumer, coupled at configuration time. A telemetry bus
# inverts that: the orchestrator emits NAMED EVENTS into a bus, and
# any number of handlers attach, detach, and crash independently.
# The producer never learns who is listening. This is the :telemetry
# pattern Elixir converged on, because every library inventing its
# own instrumentation callbacks was the worse world.
#
#   bundle exec ruby examples/telemetry_bus.rb
#
# Runs offline; three handlers listen, one detaches mid-flight.

require class="s">"bundler/setup"
require class="s">"agentic"

Agentic.logger.level = class="y">:fatal

# The bus: names, payloads, and isolation. A crashing handler is
# detached and reported - it never takes the plan down with it.
class TelemetryBus
  def initialize
    @handlers = Hash.new { |h, k| h[k] = {} }
  end

  def attach(id, event, &handler)
    @handlers[event][id] = handler
  end

  def detach(id)
    @handlers.each_value { |hs| hs.delete(id) }
  end

  def execute(event, measurements, metadata = {})
    @handlers[event].each do |id, handler|
      handler.call(measurements, metadata)
    rescue => e
      detach(id)
      puts class="s">"    [bus] handler #{id} crashed (#{e.class}) - detached, plan unharmed"
    end
  end
end

BUS = TelemetryBus.new

# The bridge: hooks in, events out. This is the ONLY place the
# orchestrator and the bus know about each other.
def telemetry_hooks(bus)
  {
    after_task_success: ->(task_id:, task:, result:, duration:) {
      bus.execute([class="y">:agentic, class="y">:task, class="y">:success], {duration: duration}, {task: task.description})
    },
    after_task_failure: ->(task_id:, task:, failure:, duration:) {
      bus.execute([class="y">:agentic, class="y">:task, class="y">:failure], {duration: duration}, {task: task.description, type: failure.type})
    },
    plan_completed: ->(plan_id:, status:, execution_time:, tasks:, results:) {
      bus.execute([class="y">:agentic, class="y">:plan, class="y">:completed], {execution_time: execution_time}, {status: status})
    }
  }
end

# Handler 1: a metrics counter - knows nothing about logging or tracing
metrics = Hash.new(0)
BUS.attach(class="y">:metrics, [class="y">:agentic, class="y">:task, class="y">:success]) { |m, _| metrics[class="y">:tasks] += 1 }
BUS.attach(class="y">:metrics2, [class="y">:agentic, class="y">:task, class="y">:failure]) { |m, _| metrics[class="y">:failures] += 1 }

# Handler 2: a slow-task tracer - only speaks when something is worth saying
BUS.attach(class="y">:tracer, [class="y">:agentic, class="y">:task, class="y">:success]) do |measurements, metadata|
  puts class="s">"    [trace] SLOW: #{metadata[class="y">:task]} took #{(measurements[class="y">:duration] * 1000).round}ms" if measurements[class="y">:duration] > 0.05
end

# Handler 3: a fragile exporter someone deployed on a Friday
BUS.attach(class="y">:exporter, [class="y">:agentic, class="y">:task, class="y">:success]) do |_m, metadata|
  raise IOError, class="s">"export endpoint down" if metadata[class="y">:task] == class="s">"enrich"
end

def run_plan(bus)
  orchestrator = Agentic:class="y">:PlanOrchestrator.new(lifecycle_hooks: telemetry_hooks(bus))
  fetch = Agentic:class="y">:Task.new(description: class="s">"fetch", agent_spec: {class="s">"name" => class="s">"w", class="s">"instructions" => class="s">"w"})
  enrich = Agentic:class="y">:Task.new(description: class="s">"enrich", agent_spec: {class="s">"name" => class="s">"w", class="s">"instructions" => class="s">"w"})
  publish = Agentic:class="y">:Task.new(description: class="s">"publish", agent_spec: {class="s">"name" => class="s">"w", class="s">"instructions" => class="s">"w"})
  orchestrator.add_task(fetch, agent: ->(_t) { sleep(0.01) })
  orchestrator.add_task(enrich, [fetch], agent: ->(_t) { sleep(0.08) })
  orchestrator.add_task(publish, [enrich], agent: ->(_t) { sleep(0.01) })
  orchestrator.execute_plan
end

puts class="s">"TELEMETRY BUS (three handlers, one bridge, zero coupling)"
puts
puts class="s">"  run 1 - all handlers attached:"
run_plan(BUS)
puts class="s">"    [metrics] #{metrics.inspect}"
puts

puts class="s">"  run 2 - tracer detached at runtime (ops got tired of it):"
BUS.detach(class="y">:tracer)
run_plan(BUS)
puts class="s">"    [metrics] #{metrics.inspect}"
puts
puts class="s">"  the orchestrator emitted the same events both runs - it cannot"
puts class="s">"  tell that the tracer left or that the exporter crashed, and"
puts class="s">"  that ignorance is the feature. hooks couple one producer to"
puts class="s">"  one consumer at configuration time; a bus decouples N handlers"
puts class="s">"  at RUNTIME, with isolation (the Friday exporter died alone)."
puts class="s">"  event names are namespaced tuples, measurements are separated"
puts class="s">"  from metadata - steal the whole class="y">:telemetry design; it was"
puts class="s">"  right. the framework's hooks made the bridge ten lines, which"
puts class="s">"  is exactly what hooks are for: being the floor a bus stands on."