The Contract Fuzzer

The Contract Fuzzer: for every registered capability, generate inputs that SHOULD pass its declared contract and mutations that SHOULD fail it, then check the validator agrees. A contract that accepts garbage or rejects conforming data is a bug in the boundary - the worst place to have one.

source on github

a real captured run

CONTRACT FUZZ (seed 20260706)
  7 capabilities, 34 trials

  every contract accepted what it promised and rejected what it should.
  the boundary holds.

source

# frozen_string_literal: true

# The Contract Fuzzer: for every registered capability, generate inputs
# that SHOULD pass its declared contract and mutations that SHOULD fail
# it, then check the validator agrees. A contract that accepts garbage
# or rejects conforming data is a bug in the boundary - the worst place
# to have one.
#
#   bundle exec ruby examples/contract_fuzzer.rb [seed]
#
# Runs offline and deterministically: same seed, same verdicts.

require class="s">"bundler/setup"
require class="s">"agentic"

seed = (ARGV.first || 20260706).to_i
rng = Random.new(seed)

Agentic:class="y">:Capabilities.register_standard_capabilities
registry = Agentic:class="y">:AgentCapabilityRegistry.instance

# Generates a value conforming to a declared type
def conforming_value(type, rng)
  case type
  when class="s">"string" then %w[alpha beta gamma delta].sample(random: rng)
  when class="s">"number", class="s">"integer" then rng.rand(1..100)
  when class="s">"boolean" then [true, false].sample(random: rng)
  when class="s">"array" then Array.new(rng.rand(1..3)) { rng.rand(10) }
  when class="s">"object", class="s">"hash" then {key: rng.rand(10)}
  else class="s">"anything"
  end
end

# Generates a value that VIOLATES a declared type
def violating_value(type, rng)
  case type
  when class="s">"string" then rng.rand(1..100)
  when class="s">"number", class="s">"integer" then class="s">"not a number"
  when class="s">"boolean" then class="s">"yes"
  when class="s">"array" then class="s">"not an array"
  when class="s">"object", class="s">"hash" then 42
  end
end

def conforming_inputs(spec, rng)
  spec.inputs.to_h { |name, decl| [name, conforming_value(decl[class="y">:type], rng)] }
end

verdicts = []
trials = 0

registry.list.each_key do |name|
  spec = registry.get(name)
  validator = Agentic:class="y">:CapabilityValidator.new(spec)
  next if spec.inputs.empty?

  # Trial 1: conforming inputs must pass
  trials += 1
  begin
    validator.validate_inputs!(conforming_inputs(spec, rng))
  rescue Agentic:class="y">:Errors:class="y">:ValidationError => e
    verdicts << class="s">"#{name}: REJECTED conforming inputs (#{e.message})"
  end

  # Trial 2: each required key, dropped, must fail
  spec.inputs.select { |_, decl| decl[class="y">:required] }.each_key do |required|
    trials += 1
    inputs = conforming_inputs(spec, rng)
    inputs.delete(required)
    begin
      validator.validate_inputs!(inputs)
      verdicts << class="s">"#{name}: ACCEPTED inputs missing required :#{required}"
    rescue Agentic:class="y">:Errors:class="y">:ValidationError
      # correct rejection
    end
  end

  # Trial 3: each typed key, corrupted, must fail
  spec.inputs.each do |key, decl|
    corrupted = violating_value(decl[class="y">:type], rng)
    next if corrupted.nil?

    trials += 1
    inputs = conforming_inputs(spec, rng).merge(key => corrupted)
    begin
      validator.validate_inputs!(inputs)
      verdicts << class="s">"#{name}: ACCEPTED #{decl[class="y">:type]} key :#{key} holding #{corrupted.inspect}"
    rescue Agentic:class="y">:Errors:class="y">:ValidationError
      # correct rejection
    end
  end
end

puts class="s">"CONTRACT FUZZ (seed #{seed})"
puts class="s">"  #{registry.list.size} capabilities, #{trials} trials"
puts
if verdicts.empty?
  puts class="s">"  every contract accepted what it promised and rejected what it should."
  puts class="s">"  the boundary holds."
else
  puts class="s">"  BOUNDARY DEFECTS (#{verdicts.size}):"
  verdicts.each { |v| puts class="s">"  - #{v}" }
  exit 1
end