When asked what’s genuinely novel in Hypothesis as opposed to mere optimisation, I usually cite three things:
- Strategies carry simplification and serialization with them but still compose monadically.
- The data generation can hit corners of the search space that standard Quickcheck can not, and adapt dynamically to requirements.
- The simplification is much more sophisticated than the standard Quickcheck interface and can adapt to the structure of the problem much better.
It took me a surprisingly long time to notice that all three of these rely on essentially the same tweak to the standard API.
The tweak is this: Introduce an intermediate representation.
- By introducing an intermediate template type which is then reified to the final result, you can simplify a mapped object by simplifying the argument.
- Instead of generating a random value, you first generate a random parameter and then draw a conditional value given that parameter. As well as producing more interesting results this way (because of how you compose parameters for e.g. lists) this also lets you reuse parameters which are more likely to give you a useful result.
- Instead of having a single simplify function, you have a list of simplifiers which you apply in turn.
Each of these have taken what was a single step and broken it up into two steps which you can naturally compose to give the original. We then don’t compose these in the obvious way, but instead change things around just enough to give something strictly more powerful while still mostly retaining the original characteristics.
I’m not sure what the broader significance of this is yet, but it seems likely that this approach has other areas to which you could apply it.