Organising unit tests in Haskell

I’ve had a lot on my plate recently so I’ve gone a bit underground. The DTubes stuff is still limping along, but it turns out that I wasn’t the only one with that idea and that others seem to be working on it with much more gusto. Banshee now supports sharing your playlist over DTubes i.e. with all your IM contacts. I don’t use Banshee, but this feature could convert me to their cause. My timelapse patch for Cheese will hopefully be finished today, or at least early next week.

What has been occupying most of my time (apart from Monster and research) is getting into this Haskell thing. I’m coming in through the back door I suppose. I’ve taken over maintenance of a Haskell library for automata theory. I’ve largely given up on libAMoRE, the codebase is featurefull but a nightmare to maintain. It’s time to start over, and in a higher level language than C. Luckily Prof. Simon Thompson from the University of Kent already had an automata library. It’s the minix of automata libraries as it strives for understandable code over micro-optimisation. It is also written in Haskell. He was even happy to relicence it to me under the BSD-new. You can find the code for Mira here.

Which leads me to unit testing. I’ve got a thing for unit testing. Eric has previously blogged about it. However, it didn’t answer one or two of my questions. Actually, it’s more to do with my testing style.

Say I have a file called BuildNfa.hs or alternatively BuildNfa.java, or BuildNfa.py. I like to then have my test file for that unit in a subdirectory called tests with filename TestBuildNfa.hs. Then I want one main file that will run all the tests for the entire project. Furthermore, I tend to just use the xUnit clone for the particular language I’m working with, and the native build system. So for Haskell this means using HUnit as the testing framework and Cabal for the build system.

My source files are in a directory called Mira and the tests are in a subdirectory Mira/Tests. For some reason Haskell insisted that the subdirectory start with a capital letter. Now I’m a total Haskell n00b, so this may simply have been a mistake or a lack of understanding on my part.

I stole/borrowed/attributed the Cabal Setup.hs from some happy hacker called Holden. Thanks Holden. My entire Setup.hs file looks like

import Distribution.Simple
import Distribution.PackageDescription(PackageDescription)
import Distribution.Simple.LocalBuildInfo(LocalBuildInfo)
import System.Cmd(system)
import Distribution.Simple.LocalBuildInfo

main = defaultMainWithHooks (simpleUserHooks {runTests = runzeTests})

runzeTests:: Args -> Bool -> PackageDescription -> LocalBuildInfo -> IO ()
runzeTests a b pd lb = system ( "runhaskell ./Tests/RunTests.hs") >> return()

Which implies that I have the file Tests/RunTests.hs as my mainline for running all my project tests.

Each test in the project has the following form

module Tests.TestBuildNfa
where

import Test.HUnit

.
.
.

suite =
TestList [TestLabel "Simple Nfa Or construction" test_simple_or]

So each Unit test has a function called suite which builds a list of tests.

Thus the runner file

module RunTests
where

import Test.HUnit
import Tests.TestBuildNfa as TestBuildNfa

main =
runTestTT TestBuildNfa.suite

need only import each “test suite” and call the HUnit function runTestTT on it.

Now I’m unsure if this is a good Haskellish way of doing unit testing, but it chimes with my experience from C, C++, Java, Python, Perl and a few other languages. It also sets up a structure for organising unit tests that is a little more extensible than other setups I’ve seen on the Interwebs. Many other people lob all their tests into one file. I simply don’t like that approach. Using this approach a simple runhaskell Setup.hs test will build and run all the tests for your project.

So, still hacking and am interested in two short term code related goals

  1. looking at literal Haskell, and
  2. getting my cheese timelapse feature finished.

And in the more medium term (i.e. over the summer) I have the following goals:

  • Have a look at how C# and F# can be integrated so that C# does the UI and F# the logic.
  • Repackage my spu-gcc, spu-newlib and update my gallium3d package for Fedora 11 on PS3.
  • Package OpenSG for Fedora.
  • That’s on top of trying to finish my Ph.D. and actually have a holiday this year. I’m visiting Vienna with my wife but intend to take my netbook along to hack on thesis.

4 Responses to “Organising unit tests in Haskell”

  1. Eric Kow says:

    Module names in Haskell start with a capital letter.

    It looks fine to me, although running a Haskell program from the shell (runzeTests) seems a bit complicated, when you could simply just runTestTT TestBuildNfa.suite directly.

    Also, with a couple of superficial modifications, you could switch to a proper test framework, namely test-framework (this supesedes the testrunner package I previously mentioned). This buys you some nice reporting, the ability to group your tests, run them in parallel and eventually integrate QuickCheck tests when you work out a way to write them.

  2. Eric Kow says:

    Also, if your tests are for one specific module (e.g, BuildNfa), you might consider just putting them in that same module. Then they can double as documentation and keep your tree simple.

  3. balor says:

    I don’t like to put my tests in the same module. Maybe it’s an OO hangover. But I generally don’t want to ship a test suite with the eventual product. So keeping the suite in a subdirectory means I can release the “product”, rather than the development code, more easily.

Leave a Reply