The Behavior Spec
The Behavior Spec: ruby/spec exists because "MRI does X" is not a specification - it's an implementation detail wearing one. When TruffleRuby and JRuby needed to know what Ruby MEANS, the answer had to be executable, implementation-neutral, and phrased as behavior. Same medicine here: a compliance file for the framework's subtlest semantics, in a 30-line mspec so the spec depends on nothing it's specifying.
Testing & Verification
Round 14
Benoit Daloze
exit 0
bundle exec ruby examples/behavior_spec.rb
a real captured run
THE BEHAVIOR SPEC (executable semantics, mspec-style) ok RateLimit windowed admission: admits exactly ceiling acquisitions, then refuses (boundary is closed) ok RateLimit windowed admission: try_acquire without a block still consumes a window slot ok RateLimit windowed admission: resize applies to the NEXT admission decision ok RelationRules presence semantics: presence means key-given-and-non-nil ok RelationRules presence semantics: sum_lte treats missing fields as zero, and the boundary as closed ok ExecutionJournal replay semantics: later events win: a success erases an earlier failure, not vice versa 6 behaviors pinned, 0 drifted. why this file exists when the rspec suite already does: the suite tests THIS implementation; this file specifies WHAT ANY implementation must do - the boundary conditions someone porting the limiter to a Ractor, a different VM, or another language needs answered precisely. note what's pinned: the ceiling-th+1 refuses (closed boundary), resize counts OLD stamps against the NEW ceiling, nil triggers don't engage requires, and a success erases an earlier failure. every one of those is a choice that could have gone the other way - which is exactly what a spec is: the choices, written down, executable, so 'what the code happens to do' and 'what the code means' stop being the same sentence. (this file's own round-14 ask was delivered in round 15: the fiber-vs-thread guarantees are now pinned per method in spec/agentic/concurrency_contract_spec.rb and documented as @note Concurrency contract: on the methods themselves.)
source
# frozen_string_literal: true # The Behavior Spec: ruby/spec exists because "MRI does X" is not a # specification - it's an implementation detail wearing one. When # TruffleRuby and JRuby needed to know what Ruby MEANS, the answer # had to be executable, implementation-neutral, and phrased as # behavior. Same medicine here: a compliance file for the framework's # subtlest semantics, in a 30-line mspec so the spec depends on # nothing it's specifying. # # bundle exec ruby examples/behavior_spec.rb # # Runs offline; exits 1 if any pinned behavior drifts. require class="s">"bundler/setup" require class="s">"agentic" Agentic.logger.level = class="y">:fatal # --- a 30-line mspec: describe/it/should, no dependencies ----------------------- module MSpec RESULTS = [] def self.describe(subject) @subject = subject yield end def self.it(behavior) yield RESULTS << [@subject, behavior, class="y">:pass, nil] rescue => e RESULTS << [@subject, behavior, class="y">:FAIL, e.message[0, 50]] end def self.expect(actual, expected, note = class="s">"") raise class="s">"expected #{expected.inspect}, got #{actual.inspect} #{note}" unless actual == expected end end # --- the compliance file --------------------------------------------------------- MSpec.describe class="s">"RateLimit windowed admission" do MSpec.it class="s">"admits exactly ceiling acquisitions, then refuses (boundary is closed)" do limit = Agentic:class="y">:RateLimit.new(3, per: 60) MSpec.expect(3.times.count { limit.try_acquire }, 3) MSpec.expect(limit.try_acquire, false, class="s">"(the ceiling-th+1 must refuse, not queue)") end MSpec.it class="s">"try_acquire without a block still consumes a window slot" do limit = Agentic:class="y">:RateLimit.new(1, per: 60) limit.try_acquire MSpec.expect(limit.try_acquire, false) end MSpec.it class="s">"resize applies to the NEXT admission decision" do limit = Agentic:class="y">:RateLimit.new(1, per: 60) limit.try_acquire limit.resize(2) MSpec.expect(limit.try_acquire, true, class="s">"(old stamps count against the new ceiling)") MSpec.expect(limit.try_acquire, false) end end MSpec.describe class="s">"RelationRules presence semantics" do MSpec.it class="s">"presence means key-given-and-non-nil" do check = Agentic:class="y">:RelationRules.check(relation: class="y">:requires, fields: [class="y">:a, class="y">:b]) MSpec.expect(check.call({a: 1, b: 2}), true) MSpec.expect(check.call({a: 1}), false) MSpec.expect(check.call({a: nil, b: nil}), true, class="s">"(nil trigger = absent, rule not engaged)") end MSpec.it class="s">"sum_lte treats missing fields as zero, and the boundary as closed" do check = Agentic:class="y">:RelationRules.check(relation: class="y">:sum_lte, fields: [class="y">:a, class="y">:b], limit: 10) MSpec.expect(check.call({a: 10}), true, class="s">"(missing b contributes 0; 10 <= 10)") MSpec.expect(check.call({a: 10, b: 1}), false) end end MSpec.describe class="s">"ExecutionJournal replay semantics" do MSpec.it class="s">"later events win: a success erases an earlier failure, not vice versa" do require class="s">"tmpdir" path = File.join(Dir.mktmpdir, class="s">"j.jsonl") j = Agentic:class="y">:ExecutionJournal.new(path: path) j.record(class="y">:task_failed, task_id: class="s">"t", description: class="s">"t", duration: 0.1, error: class="s">"x", error_type: class="s">"E", retryable: true) j.record(class="y">:task_succeeded, task_id: class="s">"t", description: class="s">"t", duration: 0.1, output: nil) state = Agentic:class="y">:ExecutionJournal.replay(path: path) MSpec.expect(state.completed_task_ids, [class="s">"t"]) MSpec.expect(state.failed_task_ids, [], class="s">"(recovery must clear the failure ledger)") end end puts class="s">"THE BEHAVIOR SPEC (executable semantics, mspec-style)" puts MSpec:class="y">:RESULTS.each do |subject, behavior, status, err| puts format(class="s">" %-4s %s: %s%s", (status == class="y">:pass) ? class="s">"ok" : class="s">"FAIL", subject, behavior, err ? class="s">" - #{err}" : class="s">"") end failures = MSpec:class="y">:RESULTS.count { |r| r[2] == class="y">:FAIL } puts puts class="s">" #{MSpec:class="y">:RESULTS.size} behaviors pinned, #{failures} drifted." puts puts class="s">" why this file exists when the rspec suite already does: the suite" puts class="s">" tests THIS implementation; this file specifies WHAT ANY" puts class="s">" implementation must do - the boundary conditions someone porting" puts class="s">" the limiter to a Ractor, a different VM, or another language" puts class="s">" needs answered precisely. note what's pinned: the ceiling-th+1" puts class="s">" refuses (closed boundary), resize counts OLD stamps against the" puts class="s">" NEW ceiling, nil triggers don't engage requires, and a success" puts class="s">" erases an earlier failure. every one of those is a choice that" puts class="s">" could have gone the other way - which is exactly what a spec is:" puts class="s">" the choices, written down, executable, so 'what the code happens" puts class="s">" to do' and 'what the code means' stop being the same sentence." puts class="s">" (this file's own round-14 ask was delivered in round 15: the" puts class="s">" fiber-vs-thread guarantees are now pinned per method in" puts class="s">" spec/agentic/concurrency_contract_spec.rb and documented as" puts class="s">" @note Concurrency contract: on the methods themselves.)" exit(failures.zero? ? 0 : 1)