Fixrb

fix

Specifications in the words of RFC 2119, tested against a subject.

A specification says what a subject MUST, SHOULD or MAY do, in the words of RFC 2119, and nothing about how it is made: it names no class, builds no object, and holds no logic. Any subject is then tested against it, each context in a child process, and every expectation gets a result. One specification can test several implementations, a library you depend on, or the same object at every release; and because the levels are three, it can tell what it requires from what it recommends and what it leaves optional.

Version
1.0.0
Ruby
3.3 or later, with fork
Dependencies
aw, defi, matchi, spectus
License
MIT

Install

Add gem "fix" to your Gemfile, or run gem install fix.

In one file

A file writes a specification, then tests a subject against it. Fix :Duck do … end builds the specification and registers it under its name; Fix[:Duck] finds it again. test builds a new subject with the block for every expectation, sends it the methods of the on around it, judges what comes out, writes one line per expectation and returns whether all of them pass; exit turns that into the status of the process.

require "fix"

class Duck
  def walk = "Klop klop!"
  def swim = "Swoosh..."
end

Fix :Duck do
  on :walk do
    it MUST eq "Klop klop!"
  end

  on :swim do
    it MUST eq "Swoosh..."
  end

  on :speak do
    it MAY eq "Quaaaaaack!"
  end
end

exit(Fix[:Duck].test { Duck.new })

ruby duck.rb
duck.rb:10: Success: expected "Klop klop!" to eq "Klop klop!"
duck.rb:14: Success: expected "Swoosh..." to eq "Swoosh..."
duck.rb:18: Info: NoMethodError: undefined method 'speak' for an instance of Duck

The duck has no speak: under MAY, a method the subject does not have is an info, which passes. The specification could sit in a file of its own and the test in another; the class could be anything with a walk and a swim. Keep the parentheses of exit(…): exit Fix[:Duck].test do … end would give the block to exit.

What a specification is for

One specification, as many subjects as you like

A specification is written once and tested against anything: the subject is chosen by the block of test, not by the specification. Here a stack is specified by what it does, then tested against Array and against a linked list. Nested on form a method chain, each one sent to what the one above returned, so push has to return the stack for pop to follow.

require "fix"

Fix :Stack do
  on :empty? do
    it MUST be_true
  end

  on :push, 1 do
    on :push, 2 do
      on :pop do
        it MUST eq 2
      end

      on :size do
        it MUST eq 2
      end
    end
  end

  on :pop do
    it MUST be_nil
  end
end

class LinkedStack
  Node = Data.define(:value, :below)

  def initialize = @top = nil
  def empty? = @top.nil?
  def size = each_node.count
  def push(value) = tap { @top = Node.new(value, @top) }

  def pop
    value = @top&.value
    @top = @top&.below
    value
  end

  private

  def each_node
    return to_enum(:each_node) unless block_given?

    node = @top
    while node
      yield node
      node = node.below
    end
  end
end

Fix[:Stack].test { [] }               # => true
Fix[:Stack].test { LinkedStack.new } # => true

ruby stack.rb
stack.rb:5: Success: expected true to be true
stack.rb:11: Success: expected 2 to eq 2
stack.rb:15: Success: expected 2 to eq 2
stack.rb:21: Success: expected nil to be nil
stack.rb:5: Success: expected true to be true
stack.rb:11: Success: expected 2 to eq 2
stack.rb:15: Success: expected 2 to eq 2
stack.rb:21: Success: expected nil to be nil

A contract in three levels

The levels of RFC 2119 say what an adapter has to do, what it had better do, and what it may leave out. A MUST not met fails the build; a SHOULD not met is a warning, and a MAY on a method the subject does not have is an info: both pass, and both are written, so the gap is known without blocking anyone. let declares a parameter of the subject, and with opens a context where it has another value: the block of test reads it to build the subject.

require "fix"

# What the application needs from a cache: a contract, in three levels.
Fix :Cache do
  let(:entries) { {} }

  on :read, :answer do
    it MUST be_nil
  end

  with entries: { answer: 42 } do
    on :read, :answer do
      it MUST eq 42
    end
  end

  on :write, :answer, 42 do
    it SHOULD be_true
  end

  on :clear do
    it MAY be_nil
  end
end

class MemoryCache
  def initialize = @store = {}
  def read(key) = @store[key]
  def write(key, value) = @store[key] = value
end

exit(Fix[:Cache].test { entries.each_with_object(MemoryCache.new) { |(key, value), cache| cache.write(key, value) } })

ruby cache.rb
cache.rb:8: Success: expected nil to be nil
cache.rb:13: Success: expected 42 to eq 42
cache.rb:18: Warning: expected 42 to be true
cache.rb:22: Info: NoMethodError: undefined method 'clear' for an instance of MemoryCache

What you expect of a dependency

The subject does not have to be yours. A specification can hold what the application relies on in a library, and run at each upgrade of that library; here the standard JSON module. An exception the method raises is judged as such: raise_exception expects it, any other matcher reports an error.

require "fix"
require "json"

# What the application expects of a library it depends on, tested at each
# upgrade of that library.
Fix :JSON do
  on :generate, [1, 2] do
    it MUST eq "[1,2]"
  end

  on :parse, "[1,2]" do
    it MUST eq [1, 2]
  end

  on :parse, "nope" do
    it MUST raise_exception JSON::ParserError
  end

  on :pretty_generate, [1] do
    it SHOULD eq "[\n  1\n]"
  end
end

exit(Fix[:JSON].test { JSON })

ruby json.rb
json.rb:8: Success: expected "[1,2]" to eq "[1,2]"
json.rb:12: Success: expected [1, 2] to eq [1, 2]
json.rb:16: Success: expected #<JSON::ParserError: unexpected token 'nope' at line 1 column 1> to raise exception JSON::ParserError
json.rb:20: Success: expected "[\n  1\n]" to eq "[\n  1\n]"

Specifications that compose

A specification is a matcher: fix(:Name) requires a registered one, fix { … } one written on the spot, and a level applies to it as to any matcher. A failed fix says that the specification is not met, not which of its expectations failed: test that specification on its own for the detail.

require "fix"

Fix :Positive do
  on :positive? do
    it MUST be_true
  end
end

Fix :Score do
  it MUST be_a_kind_of Numeric
  it MUST fix(:Positive)
  it MUST_NOT fix { on(:nan?) { it MUST be_true } }
end

exit(Fix[:Score].test { 7.5 })

ruby compose.rb
compose.rb:10: Success: expected 7.5 to be a kind of Numeric
compose.rb:11: Success: expected 7.5 to fix Positive
compose.rb:12: Success: expected 7.5 not to fix an anonymous specification

State that cannot leak

Each context with expectations runs in a child process forked from the one that tests, with aw. A subject that writes to a global, a constant, ENV or the current directory does so in its child alone: every context starts from the state the file was loaded with, and the order of the contexts cannot change a result.

require "fix"

# A subject with a global state: every context starts from the state the
# file was loaded with, since each one runs in a child process.
module Registry
  NAMES = []

  def self.add(name) = tap { NAMES << name }
  def self.names = NAMES
end

Fix :Registry do
  on :add, "Alice" do
    on :names do
      it MUST eq ["Alice"]
    end
  end

  on :add, "Bob" do
    on :names do
      it MUST eq ["Bob"]
    end
  end

  on :names do
    it MUST eq []
  end
end

exit(Fix[:Registry].test { Registry })

ruby registry.rb
registry.rb:15: Success: expected ["Alice"] to eq ["Alice"]
registry.rb:21: Success: expected ["Bob"] to eq ["Bob"]
registry.rb:26: Success: expected [] to eq []

The language

The block of Fix is a document, read once when the file is loaded, with four words. on :name, *arguments sends a method to the subject, and the expectations inside judge what it returns. let(:name) { … } is a parameter of the subject, computed once per expectation, the first time the subject block calls it; with name: value opens a context where it has that value. it takes a requirement: a level, MUST, MUST_NOT, SHOULD, SHOULD_NOT or MAY, then a matcher.

require "fix"

Fix :Greeting do
  let(:name) { "Bob" }

  it MUST eq "Hello, Bob!"
  it MUST_NOT be_nil

  on :size do
    it MUST eq 11
  end

  with name: "Alice" do
    it MUST eq "Hello, Alice!"
  end
end

exit(Fix[:Greeting].test { "Hello, #{name}!" })

ruby greeting.rb
greeting.rb:6: Success: expected "Hello, Bob!" to eq "Hello, Bob!"
greeting.rb:7: Success: expected "Hello, Bob!" not to be nil
greeting.rb:10: Success: expected 11 to eq 11
greeting.rb:14: Success: expected "Hello, Alice!" to eq "Hello, Alice!"

The matchers are those of matchi, as words: eq, be, be_true, be_false, be_nil, be_within(delta).of, match, be_an_instance_of, be_a_kind_of, raise_exception, change and satisfy, plus fix; any object with match? and to_s is one too. There is no predicate helper: on(:empty?) { it MUST be_true } asks the question.

A let is visible from the subject block and from the other lets only; named in an expectation, it raises NameError at load, with the remedy. Its name cannot be a method of the object the subject is built on, nor a word of the language, so that nothing is redefined in silence. Everything that is wrong in the document raises when the file is loaded, before any test: an it with a block or without a requirement, a let or a specification named twice, a matcher without match?. A block that raises registers nothing.

Outcomes

Two questions tell the levels apart. Does the method exist? MUST and SHOULD require it, MAY does not. Does it do what is specified? MUST and MAY require it, SHOULD recommends it. Hence five outcomes, of which a success, a warning and an info pass:

LevelMatch is trueMatch is falseMissing methodOther exception
MUSTsuccessfailureerrorerror
SHOULDsuccesswarningerrorerror
MAYsuccessfailureinfoerror

A negation swaps the first two columns. The missing method is the one named by the on of the context, when the object it is sent to raises NoMethodError for it. Below an error, test writes the frames of its backtrace that are not those of Fix and its gems, so that the line of the subject is there:

require "fix"

class Account
  def initialize(balance)
    @balance = balance
  end

  attr_reader :balance

  def withdraw(amount)
    raise ArgumentError, "insufficient funds" if amount > balance

    @balance -= amount
  end
end

Fix :Account do
  on :balance do
    it MUST eq 100
    it MUST eq 200
    it SHOULD eq 200
  end

  on :currency do
    it MAY eq "EUR"
  end

  on :withdraw, 500 do
    it MUST eq(-400)
  end
end

Fix[:Account].test { Account.new(100) } # => false

ruby account.rb
account.rb:19: Success: expected 100 to eq 100
account.rb:20: Failure: expected 100 to eq 200
account.rb:21: Warning: expected 100 to eq 200
account.rb:25: Info: NoMethodError: undefined method 'currency' for an instance of Account
account.rb:29: Error: ArgumentError: insufficient funds
  account.rb:11:in 'Account#withdraw'
  account.rb:33:in '<main>'

Results and runners

test writes, for each expectation, one line, LOCATION: OUTCOME: SUMMARY, the summary being that of spectus, and nothing else: no letter, no colour, no count. A file is run with ruby, and its status is 0 when the specification is met: that is all a continuous integration, a Rakefile or a runner needs, and the grammar of the lines is what a runner reads to recap or colour them. run gives the same results as data, in the order of the contexts, and match? tests a value as a matcher does:

require "fix"

Fix :Size do
  it MUST be_an_instance_of String

  on :size do
    it MUST eq 3
  end
end

results = Fix[:Size].run { "abcd" }
results.map(&:outcome) # => [:success, :failure]
results.map(&:passed?) # => [true, false]

result = results.last
result.location # => "examples/fix/results.rb:7"
result.chain    # => ".size"
result.summary  # => "expected 4 to eq 3"
result.to_s     # => "examples/fix/results.rb:7: Failure: expected 4 to eq 3"
result.result.class # => Spectus::Result

Fix[:Size].match? { "abc" } # => true

Processes

The contexts run one after the other, each in its child, and the parent writes the results of a context as its child returns them. What the subject raises, an exit included, is an error of its expectation alone. Nothing is rescued around a child: an Interrupt stops the run, and so does a child that died, with Aw::ChildError. Fix never writes a result that did not come from an expectation. test(fork: false) runs the contexts in the process itself, for a debugger. The cost of a context is one fork, a few milliseconds.

What Fix leaves out

No mocks, no before and after, no shared examples, no random order: a specification holds expectations and nothing else, and what a test needs to set up belongs to the subject block, where the lets live. The same specification tests a class, a module, an instance or a library, and gives the same results on every run. Fix needs fork, so it runs on Linux, macOS and the BSDs, not on Windows or JRuby. A test suite that needs fixtures, doubles or an application framework keeps its framework; Fix specifies the behaviour the pieces agree on, and matchi-fix brings such a specification into that framework as a matcher.

Extensions

fix-its adds the word its, a method and its expectation on one line. matchi-fix brings a specification of Fix to any framework that takes a matcher of matchi. The block of Fix runs on an instance of Fix::Dsl, where a gem may add a word of its own.

Documentation