Implementing a unit testing framework is fun to implement in Scheme. With a 100 lines of Scheme code, you are basically up and running. See my previous blog about unit testing in Scheme
One of the improvement is to provide access to the currently running test case. As naive this feature seems to be, it brings some interesting implementation discussion. So how do you go about implementing this ?
One simple solution is to have the currently running unit test in a global variable. The code looks like this:
(define *unit-test* #f)
(define (the-unit-test) *unit-test*)
(define (run-all-test!)
(for-each (lambda (t) (set! *unit-test* t) ((unit-test-thunk t)))
*unit-tests*))
In the running unit test, you simply call (the-unit-test). All fun, but what's wrong with this solution ? We have a global variable ! Everybody knows that global state is bad and you are better without it.
Now that you have multi-core processors, you may be tempted to run your unit tests suite in parallel. But how do you go about it if you have a global variable ? You can't.
So the standard solution is to pass around the running unit test instance around. The implementation goes like this:
(define (run-all-tests!)
(for-each (lambda (t) ((unit-test-thunk t) t)) *unit-tests*))
(define-syntax define-unit-test
(syntax-rules ()
((define-unit-test ?name ?doc ?body)
(add-unit-test! (make-unit-test '?name ?doc ?body)))))
(define-unit-test test-string=?
"Tests the string=?"
(lambda (unit-test)
(assert-true! unit-test (string=? "Hello" "Hello"))))
Some comments are required in order to fully grasp this design. The first change is that the running test is passed as the first parameter to the code performing the test cases.
This leads to the second change: the definition of the define-unit-test macro. The code for the unit test is now a lambda expression in itself. This is necessary because syntax-rules only supports hygienic macro definition. Having the name of the unit test variable defined implicitly requires a non-hygienic macro. Thus you should use a syntax-case macro facility or the explicit renaming macros.
The last change is that now the assert-xxx procedure takes an additional parameter (the currently running unit test). This makes it possible to print the currently running unit test name when some assertion fails !
Now you have a nice implementation which makes running more than one unit test concurrently possible. So you may take advantage of the multiple cores of your processor.
Let say that you want some statistics about your unit tests, like how many assertions have been done, the number of unit tests run and the ratio of failed/passed unit tests. If you think about it, you should recognize the statistic collector as another global object that should be accessible when running your unit tests.
You use a global variable for this. But as we have seen, this is not a good solution. You add another parameter to the running unit test and you have 2 parameters to pass around. To simplify the implementation, you may define a combo datastructure containing the running unit test and the collected statistics.
This principle is fine, but scales poorly. Maybe in the future, you want to provide another kind of information within the running tests. So you add another parameter or another entry in your combo datastructure. Finally you have thousands of parameters/entries in your solution. Isn't there a more simple and scalable solution ? In a following blog entry, you will know about a unique feature (almost) of the lisp family of language: dynamic varables.