Unit Testing framework in Scheme

Unit testing has become a standard tool for testing an piece of code. Implementing a Scheme solution for unit testing is simple and fun. You define a domain specific language for describing unit tests. This is completly done with macros. It looks something like this:

(define-unit-test testing-string=?
  "testing simple cases of string=? procedure"
  (assert-true! (string=? "Hello" "Hello"))
  (assert-false! (string=? "Hello" "world")))

Basically you specify a unit test name, testing-string=? here, together with a documentation string. Then comes a Scheme code performing the tests. You use the different flavours of the assert-xxx! procedure.

The implementation looks as follow. The main datastructure is the unit-test record type.

(define-record-type 
  (make-unit-test name doc thunk)
  unit-test?
  (name unit-test-name set-unit-test-name!)
  (doc  unit-test-doc  set-unit-test-doc!)
  (thunk unit-test-thunk set-unit-test-thunk!))

You should recognize here the different parts of the define-unit-test macro. In fact the macro basically creates instances of the <unit-test> record and store them in a table. Here is the macro definition:

(define-syntax define-unit-test
  (syntax-rules ()
    ((define-unit-test ?name ?doc . ?body)
     (add-unit-test! (make-unit-test '?name ?doc (lambda () . ?body))))))

All the unit tests defined are stored in a global variable.

(define *unit-tests* '())

(define (add-unit-test! test)
  (set! *unit-tests* (cons test *unit-tests*)))

Running all the unit tests defined so far is quite simple:

(define (run-all)
  (for-each (lambda (t) ((unit-test-thunk t))) *unit-tests*))

Writing unit tests requires the different assert procedures. They are simply implemented as procedures.

(define (assert-true! value)
  (if (not value) (display "error: assert-true! violation")))

(define (assert-false! value)
  (if value (display "error: assert-false! violation")))

(define (assert! expected effective)
  (if (not (equal? expected effective))
      (display "error: assert violation!")))

That's it for the implementation. In under 50 lines of Scheme code you have a basic unit testing framework: that's fun or not ? Moreover this is a solution that runs on any R5RS Scheme system with SRFI-9 record type definition: Scheme48, PLT Scheme, MIT-Scheme, etc...

This implementation has some limitations though. You may want some of the following features if you want a solid unit testing framework:

• Some kind of output/trace/statistics when running all unit tests. This makes running all tests more appealing. Having the number of run tests displayed, togehter with the currently running tests and its result (passed/failed) would be nice. Some basic modifications of the run-all procedure may go a long way toward this goal.
• When some assertions are violated, some kind of debugging output in order to better understand what happened. Here you may transform the different assert procedure into macros, so that in case of failure you can display the failing expression. Something like: (string=? "Hello" "World") failed, expected #f, evaluated to #t.
• An ability to define some test suites. Test suites are subset of all unit tests. When doing development you may not be interested in running all unit tests, because only some unit tests are necessary to make sure your modification are safe. Also running all unit tests make take some time. What good it is to unit test a web server, when changing a GUI library. Thus having a GUI tests suite and web server test suite may be quite usefull.
• Fixtures. When running a unit test, you may need to set your environment in a specific state so that your test run successfully. When defining unit tests, you may want to create some entries in a database or starting a web server, in order for your tests run properly.