Do you know what the best design decision I made in Hypothesis was?

It wasn’t well thought out. It wasn’t anything clever or theoretically interesting. It was done purely because I was lazy, not for any principled reason. It’s still the best design decision.

It’s this: Hypothesis does not have its own test runner. It just uses whichever one you’re already using.

This turns out to be a great idea, because it means that there’s basically zero cost to using Hypothesis: Install an extra library, write some code that uses @given to feed some data to a function. Done. Now you’re using Hypothesis.

As far as difficult sells go, this this barely even registers.

Compare this to, say, adding static analysis to your process (I’ve never got around to adding pylint to Hypothesis and I care about this stuff, because it gives too many false positives so it’s never been quite worth the up front investment), or rewriting everything in Haskell.

It’s certainly possible that both of those will produce enough of a return on investment from their correctness benefits that they’re worth the up front cost, but unless you’re really confident that they will and can afford to take the time to do it right now they’re things that you can put off basically indefinitely.

Hypothesis on the other hand is so cheap to get started with that you can start benefiting almost immediately, and then incremental amounts more work will produce incremental amounts more improvement.

This turns out to be incredibly valuable if what you actually want to produce better software.

One of the responses to the economics of software development post is that doing everything right up front can be cheaper than the long term cost of the unprincipled hacks that people often do instead.

This is certainly true. If you are lucky enough to find yourself in a position where you have the option of doing everything right up front, I recommend you make a reasonable effort towards doing so. You’ll probably fail (getting everything right is really rather hard), but in failing you will probably still end up in a better position than in not trying.

But, well, this is rarely the situation in which we find ourselves. Even if we got everything right at the beginning, at some point in the project we were rushed, or made a decision that later turned out to be a bad call, or in some other way failed to be perfect, because nobody can be perfect all the time.

If this is not the situation you find yourself in and you are working on a mature, high quality project that has consistently got most things right throughout its lifespan and has fixed the things it does not, good for you. Say hi to the unicorns for me while you’re there.

Most of us are not only not in that situation but that situation is almost entirely unreachable. Saying that if you did everything right at the beginning you wouldn’t have this problem is just another way of saying it sucks to be you.

So what we need are not tools that work great if you use them from the beginning, or tools that work great if you embrace them whole-heartedly, but tools that make it easy to get started and easy to move in the right direction.

This means that “rewrite it in Haskell” is out, but “we can write this microservice in Haskell” might be in. It means “Always have 100% branch coverage” is out, but “merges must never decrease branch coverage” is in. Failing the build if you have any static analysis warnings is out, but failing a merge if the touched lines have any static analysis failures is in.

Incremental quality improvements allow you to move your project to a higher quality state without requiring the huge up front investment that getting everything right from the beginning requires, and they don’t punish you for the fact that you didn’t get everything right before you had this tool to help you get things right.

This is of course often a harder problem, but I think it’s one that’s important to solve. If we want higher quality software we don’t get to live in a fantasy world where everything magically works, we have instead to think about how to move from the world we live in to a reachable world in which everything works better, and we need to think about what the steps along the way look like.