This is a response to a good piece called “Infrastructure as code would be easy if we cared”. It’s not required reading for this post, but I recommend it anyway, and I mostly agree with its message.

But there’s one point in it that I feel the need to quibble with, partly because I spent so long making it myself before I realised it was bogus:

But any real business:

• Knows what their acceptable defect rate is
• Is already operating at it

This isn’t true.

I’m not going to quibble about “any real business” or “know”. I could, but it would be needless pedantry.

What I want to quibble with is the idea that businesses operate at their acceptable defect rate.

The defect rate a business operates is not a single number that is magically acceptable when all others aren’t. It’s a rate they can’t afford to improve: When making improvements to your defect rate costs more than the reduced rate of defects would save you, you stop trying to improve the rate.

You might call that the acceptable defect rate if you like, but if you do the point is that the acceptable defect rate is inherently unstable and the idea that it would remain the same under changes to your work flow is just untrue.

This is not a trivial point: It means that you can reduce the defect rate and get more money if you change the economics. A reduction in defects that is currently non-viable because it would cost you 100 person hours could suddenly become viable if it cost you 50 person hours instead (it could also become viable if e.g. new regulations come in that make some of those defects really expensive).

And some things do change the costs of reducing defects. Obviously I think Hypothesis is one of those things, because it increases your ability to find defects much more quickly, thus decreasing the cost of fixing them. This is a comparatively rare example of something that changes the cost of finding and fixing defects while keeping most other things fixed, and I genuinely believe that this reduces the defect rate in software.

It’s always tempting to think of the world as immutable and hard to change, but the reality is that it’s mostly just the result of large systems responding to costs and incentives, and small changes in those costs incentives can produce remarkably large effects if you give them time to work.