Multilingual Testing

A technique I’ve seen a lot in recent years, and I believe for good reason, is using one language (programming language, that is!) to test code written in another.

There are some non-obvious advantages to this that I’d like to explore a bit below, as well as some obvious (but perhaps less important) disadvantages.

On the downside, of course, you need a developer or team conversant in at least two different languages. This is more rare than you might think, although I think more experienced developers have the edge in that they’ve likely been working in more than one language if they’ve been in the industry a while. Also, many developers like to tinker, and will often pick up one of the “low ceremony” languages to do that tinkering with, and end up gaining at least a passing familiarity with a new language as a result.

Another downside (and often the reason this isn’t done more) is a lack of common tool support between the two languages. If you’re working in an IDE that only speaks Java fluently, for instance, you’re less inclined to use another language for your tests, as then you’ve not only got to switch languages, but endure the “context switch” overhead of changing IDE’s midstream as well. If you’re doing real TDD, that’s a lot of context switches.

On the upside, though, there are a number of good reasons you might want to consider using a different language for your tests:

  • Intentional context switch: The same reason some people avoid switching languages for testing can actually be an advantage. When you’re tests are in another language, you’ve got a built-in mental gear-change between the code and the tests
  • Decoupling: If you’re changing language, you’re more inclined (maybe even required) to limit the touch-points between the tests and the actual code – e.g. you might isolate the interface to a single class that’s made visible in the test. This can be a good thing, as a too-tightly coupled test is a brittle test (and often not a unit test, but that’s a whole ‘nother topic)
  • Better test framework: While xUnit-like frameworks abound in just about every language, some of the higher-level and BDD-type frameworks are best in one specific language. RSpec is an excellent example: It’s originally in Ruby (although I understand some attempts have been made to port it), so it might be that you’d consider using Ruby as a test language in order to get the goodness of RSpec. Ditto for easyb, which is written in Groovy (a very close relative of Java, in many ways). The reverse could be true: You could write JUnit unit tests for your Scala application, for instance, taking advantage of JUnit 4 and friends being on the Java platform. </li>
    • Better language suitability: Some languages are much “lower ceremony” than other – e.g. compare Ruby and Java. You need a lot of scaffolding and fuss around a large Java application, but Ruby is a bit more nimble and in some cases, more expressive. Expressiveness being a key quality of a good test, this might mean testing a Java app with Ruby (or perhaps JRuby, to be specific) is a good plan, especially with the existence of Test::Unit and RSpec and such
    • Multi-Lingual project: It’s becoming more and more common to see a project use more than one development language in any case (e.g. for the ‘real’ code, not just the tests). In this case, there’s even less of a barrier to doing your tests in one of those languages – you’re only helping to maintain your fluency in both language as well, which can’t be a bad thing.</ul> So, how about it? Anyone out there have good or bad stories about multi-lingual testing to share?

    Principles and Practices

    Tired of the Software Development Grind? Know it can be done better? Check out my book: Principles and Practices of Software Craftsmanship or sign up for my Craftsmanship Dispatches newsletter.