I project I happened across a while ago is starting to intrigue me more and more: Specs
It’s a framework written in Scala designed to support Behavior-Driven design and testing. This style of testing encourages tests to describe the be “behavior” of the class or component under test, ideally in way that is easy to understand by both developers and non-developers alike.
How does this differ from other forms of testing? Mostly in the way functionality is described – a feature is expressed in terms of a series of interactions with the system under test, describing functionally in a narrative form. We don’t describe a functional technically, as in “passing 43 should result in a response of ‘ABC’”, but in terms of valuable feature interactions, such as “when a valid login is entered, the foo link becomes available to be clicked”, for example.
Behaviour-driven design and development make sure we’re developing code that implements the right features, as opposed to test-driven development, which focuses more on developing the code correctly and accurately. BDD and BDT is more about the “what” as opposed to the “how”, in other words.
If the subject-area expert for the system under development can understand the language of the test, then it becomes more of an executable specification than just a test – it may even form the acceptance criteria for a story or feature, especially when multiple such specifications are chained into a narrative.
Flowing from the oft-described “As a… I want… So that…” way of describing a story, a narrative describes a sequence of flow through an application, describing and verifying the behavior of the system at each step in the narrative.
Specs supports this very nicely, allowing you to write in a DSL (Domain-Specific Language) which is nonetheless a fully capable programming language at the same time (in this case, Scala).
The basic unit of a Specs test is the “specification”, implying, appropriately, that you literally write executable specifications – documentation that actually verifies that it properly describes the system being described!
Scala is, by design, an extremely extensible language, so it’s support for DSL’s such as specs is very good – it doesn’t burden you with a lot of unnecessary syntax or verbosity, so the resulting specification is highly readable and relatively english-like. So much so that it’s even possible for a domain expert non-developer to write, much less read, the executable specifications – especially if they have examples to go from.
Scala fits seamlessly into any JVM-based environment, plays nicely with IDEs, Ant, Junit, Maven, and all the tools you might already be using to build your systems, so it’s adoption doesn’t cause much of a ripple effect. Scala files simply live in a directory structure inside an otherwise all-Java project, for instance, and are compiled to .class files, just like Java.
Specs also leverages other powerful test frameworks, such as Mockito, JMock and Scalacheck, among several others.
My team is using Specs in two modes, so far: One is as a part of the primary build process, just like you’d run your JUnit or SpecsTest tests. The Scala tests in this case are used as either actual unit tests (although expressed in a behavior-driven way) or in-process functional tests (if they include several classes or a whole subsystem under test).
The other way we use Specs is as a set of stand-alone acceptance tests. In this mode we run Specs externally from the system under test. This is especially useful for our systems that rely heavily on REST web services. In these cases we write our narratives to assert various responses from the REST services found at various URLs, while launching Specs from Ant in it’s own VM on another box entirely. These are true “black box” tests, where the tests have no knowledge of or access to the code under test – they must interact with the running system in a very “real world” way. This provides an ideal environment for acceptance tests, especially given the readability of the Specs specificiations. Our user-proxies also like the fact that the test itself can’t be influencing the code “inside the box”, as it’s easy for programmers to write “happy path” tests
Of course, once you’ve used specs and Scala for your testing, you may be tempted to infiltrate it into your other development, but that’s another post