Tag Archives: testing

Reusing Java’s tools: Cobertura and Scala

I’ve recently been writing some collection implementations for Scala which will hopefully go in the standard library in the upcoming release. Naturally I want to make sure that I tested the hell out of them, so I’ve been writing a reasonably large number of ScalaCheck tests to test the code and make sure it does what I want it to. Earlier I realised I should probably be doing some sort of coverage checking to make sure I had tested things adequately so thought I’d see how easy it would be to plug Cobertura into a Scala project. And I have good news! The answer is: Essentially trivial.

There were a few minor issues that cropped up while doing it:

  • Cobertura strips scala specific metadata out of the class files. This means that you must compile everything first, including your unit tests, before instrumenting code.
  • Scala does a lot of name mangling and generates a lot of classes with weird names. Cobertura naturally doesn’t understand the mapping back to Scala code, so you’ll see the mangled names. They’re not usually that hard to figure out.
  • Cobertura won’t be able to find your source files if you don’t follow Java directory conventions for packages.

Other than that, it all works fine. You get per line coverage information for your source files, decent per package and per class reports, etc. No problem.

Unfortunately my coverage wasn’t nearly as good as I hoped, so I’ll probably have to write a bunch more tests before I submit a patch.

Update: Actually, the line number reporting for Cobertura + Scala appears to be really bad, and often really hard to decipher. But it’s still a nice tool, and actually helped me catch some bugs.

This entry was posted in programming and tagged , , on by .

Testing, testing

Not the blog, the code.

Among the many weird and wonderful iterators I have in my personal JGreenspun toolkit is one called the FlatteningIterator. It’s a useful little widget for recursively digging through mixed collections of objects and collections. Completely impossible to give a decent type signature to unfortunately, but what can you do? (I mean, I can conceive of a type system in which you can, but it’s basically going to have to be dependent typing). It forms a major chunk of the functionality for my recent WebCrawler implementation.

Indeed, that implementation is exactly what I want to talk about – the very first time I came to run the webcrawler it failed horribly. Rather than, as one might expect, being a failure in fetching or parsing the web page, the guilty party was my nice little utility iterator.

In my efforts to preserve laziness (that is, an object would never be fetched until it was needed), I’d basically screwed up. But I’d done it in a mildly subtle way so I never even thought to unit test it. The iterator’s hasNext() method returned true iff there was a non-empty iterator available to it.

Can you spot the problem? If the iterator available is an iterator which only returns empty iterators then the flattened iterator will falsely report that it has a next element, so using the class while(iter.hasNext() { iter.next(); } loop will throw an unfriendly exception. Unfortunately I completely failed to spot this until it came up in actual practice.

Thus, even though I had extensive unit tests for this class, they missed the one way in which it was actually broken. (The class passed all the other unit tests first time). This is because logic errors in the implementation often correspond to failures to spot an edge case. So if the same person doing the implementation is writing the unit tests, the unit tests probably won’t cover that edge case either.

Fortunately it only took a little time to figure this out, and once I had the unit test was updated to cover this case. So, no real harm done. But I’d have been really annoyed if it had taken me a long time to track down a bug and it turned out to be in code I thought was safe due to unit testing.

This entry was posted in programming and tagged on by .