Author Archives: david

On Haskell, Ruby, and Cards Against Humanity

I really hate Cards Against Humanity.

I don’t particularly hate it for the obvious reasons of middle class white kids sitting around a table giggling at how transgressive they are being with their jokes about race. I don’t approve, and I would probably avoid it for that reason alone, but it’s frankly so banal in this regard that it probably wouldn’t be enough to raise my ire above lukewarm. What I really hate about it is that it’s a game about convincing yourself that you’re much funnier than you actually are.

Cards Against Humanity is funny, but it’s not that funny. It relies on shock value to inflate its humour, and by making a game of that draws you in and makes you complicit. Your joke about Vladimir Putin and Altar Boys? Not actually very funny, but by giving you permission to be shocking, Cards Against Humanity lets you feel like the world’s best comedian for an hour.

There’s nothing intrinsically wrong with enjoying pretending you’re funnier than you actually are. I’ve intensely enjoyed games where I got to spend an hour pretending I was an elf, and no harm was done to any parties involved (except for some imaginary orcs who found themselves full of imaginary lightning bolts). The difference is, I don’t walk away from the latter with any doubt in my head about whether I’m an elf (I’m not, in case you were wondering). People walk away from a game of Cards Against Humanity thinking that they’re hilarious.

Is this objectively a problem? I’m not sure I’d go that far. It would require too much of a digression into systems of ethics and theories of value.

Personally though, I hate it. I hate it a lot. Humour and accurate self-knowledge are both things I value, and the intersection is especially important because inaccurate self-knowledge about something you want to become better at is a great way of becoming worse at it instead. If you learn the lesson that being crass is a great way to be hilarious, you won’t become better at humour you’ll just become better at being crass.

Which brings me on to Ruby and Haskell.

These two programming languages are about as far away from each other on the field of programming languages as you can get. But one thing they have in common, other than garbage collection and list syntax, is their similarity to Cards Against Humanity.

Oh, they generally don’t give you the same sense of being way funnier than you actually are (_why’s poignant guide and learn you a haskell aside), but they do have community norms that have similar effects of inflating your sense of how good you are at something important.

A thing I value probably at least as much as humour is good API design. It’s an essential weapon in the war on shitty software. It’s almost completely absent in Ruby. There are some exceptions (Sequel and Sinatra for example), but by and large Ruby APIs seem almost actively designed to promote bad software.

Ruby is full of people who think they’re doing good API design, but they’re not. The problem is that in Ruby everyone is designing “DSLs”. The focus is on making the examples look pretty. I’m all for making examples look pretty, but when you focus on that to the exclusion of the underlying semantics, what you get is the Cards Against Humanity effect all over again. You’re not doing good API design – you’re doing something that has the appearance of good API design, but is in reality horrifically non-modular and will break in catastrophic ways the first time something unexpected happens, but the lovely syntax lets you pat yourself on the back and tell yourself what a good API designer you are.

And then there’s Haskell.

The way this problem manifests in Haskell is in how incredibly clever it makes you feel to get something done in it. Haskell is different enough from most languages that everything feels like an achievement when writing it. “Look, I used a monad! And defined my own type class for custom folding of data! Isn’t that amazing?“. “What does it do?” “It’s a CRUD app”.

To a degree I’m sure this passes once you’re more familiar with the language and are using it for real things (although for some people familiarity just spurs them on to even greater heights), but a large enough subset of the Haskell community never reaches that stage and just continues to delight in how clever they are for writing Haskell. In the same way that Cards Against Humanity gives you a false sense of being funny, and Ruby gives you a false sense of designing good APIs, Haskell gives you a false sense of solving important problems because it gives you all these exciting problems to solve that have nothing to do with what you’re actually trying to achieve.

I very much do not hate Haskell or Ruby in the same way that I hate Cards Against Humanity, because unlike Cards Against Humanity these features are not the point of the languages, and the languages have myriad virtues to counteract them.

But I do find a fairly large subset of their communities really annoying in how much they exemplify this behaviour. They’re too caught up in congratulating themselves on how good they are at something to notice that they’re terrible at it, and it makes for some extremely frustrating interactions.

This entry was posted in programming on by .

The #1 reason I don’t write Haskell

I generally can write Haskell. I’m not especially good at it, I’m definitely not able to tell you how to use a ZygoHistoPremorphism in your code, but fundamentally I find Haskell code relatively comprehensible and there aren’t any blockers to my getting better at it other than time and effort. There’s a lot I like about the language, and I often steal ideas from it for use elsewhere.

And yet I have almost zero inclination to write real code in Haskell, and every time I get a slight hankering to do so this is quickly fixed.

Because the first thing I have to interact with every time I write some Haskell is cabal, and this is enough to cure me of any desire to continue the interaction all on its own.

There are a number of little irritations I have with Cabal. I could get over them. There’s no such thing as a good package manager, only less bad package managers.

But there’s one big thing that I simply cannot get over and prevents me from ever taking Haskell seriously as a platform until it has been fixed.

$ cabal uninstall somepackage
cabal: unrecognised command: uninstall (try --help)

This is not just that it’s called one of the many other things people bewilderingly decide is the appropriate name for the uninstall command. cabal genuinely cannot uninstall packages. Further, having the wrong packages installed can leave you in a stuck state where you are unable to develop.

At which point the only fix is “trash your cabal install, rebuild cabal and cabal-install from source” (a process which takes nearly half an hour on my machine, and that’s before I’ve also rebuilt all my dependencies from source).

I understand that this is a hard problem which results from the underlying ghc-pkg infrastructure. I don’t really care.

I also understand that cabal sandboxes are a thing. I don’t care about that either. It helps for development, but you only have to make one mistake or give in to any of the myriad things that are pushing you to do a global install just this one time (binaries, incredibly slow build times, etc) to accidentally destroy your system.

The simple fact is that if the default mode of operation for the primary way of installing packages for your language is “oops, you did the wrong thing. Let me put your system in an inoperable state while you wait for everything to recompile from scratch” then what you  have here is not a package manager, it’s a waste of time, and I have enough broken software wasting my time already.

I stand by everything I said in it’s OK for your open source to be shitty. There is no obligation on the part of the maintainers of Cabal, or of GHC, to fix these problems for free. But the fact that it’s OK for software to be shitty doesn’t make it not shitty, and the fact that more than 50% of the time I interact with Cabal I end up in a screaming rage means that I’m just going to not use it unless absolutely forced to, which in turn means that there’s basically zero chance of my ever treating Haskell as a serious development environment.

Edit to add: Brian Mckenna suggested this fix, which is to globally disable the ability to install outside a sandbox. This actually does make sandboxes a viable solution to the problem and is the only thing anyone has suggested that I consider an acceptable workaround. So although I stand by the claim that the cabal infrastructure is terrible, it may now at least be manageably terrible because it can be used without accidentally trashing your system.

This entry was posted in Uncategorized on by .

Getting Google Analytics on readthedocs

If you’re like me, you like the following things:

  1. Knowing about people who use your software
  2. Not hosting your own documentation

As a result, you want some sort of analytics on readthedocs!

If you’re very like me this will have resulted in you spending a fruitless morning looking through the results of google and finding none of the solutions work at all.

Well, allow me to fast forward this process for you. The correct way to add google analytics to your readthedocs instance is as follows:

  1. Go to the Admin page for your project
  2. Click on “Advanced settings”
  3. Enter your Google Analytics Tracking ID in the box at the bottom that says “Analytics ID”

That’s it.

I have literally no idea how you are supposed to discover this. I found it by accident while messing around with build settings to try to get one of the other solutions to build. Hopefully this is of use to you.

This entry was posted in Uncategorized on by .

How to improve your Quickcheck implementation with this one weird trick

When asked what’s genuinely novel in Hypothesis as opposed to mere optimisation, I usually cite three things:

  1. Strategies carry simplification and serialization with them but still compose monadically.
  2. The data generation can hit corners of the search space that standard Quickcheck can not, and adapt dynamically to requirements.
  3. The simplification is much more sophisticated than the standard Quickcheck interface and can adapt to the structure of the problem much better.

It took me a surprisingly long time to notice that all three of these rely on essentially the same tweak to the standard API.

The tweak is this: Introduce an intermediate representation.

  1. By introducing an intermediate template type which is then reified to the final result, you can simplify a mapped object by simplifying the argument.
  2. Instead of generating a random value, you first generate a random parameter and then draw a conditional value given that parameter. As well as producing more interesting results this way (because of how you compose parameters for e.g. lists) this also lets you reuse parameters which are more likely to give you a useful result.
  3. Instead of having a single simplify function, you have a list of simplifiers which you apply in turn.

Each of these have taken what was a single step and broken it up into two steps which you can naturally compose to give the original. We then don’t compose these in the obvious way, but instead change things around just enough to give something strictly more powerful while still mostly retaining the original characteristics.

I’m not sure what the broader significance of this is yet, but it seems likely that this approach has other areas to which you could apply it.

This entry was posted in Uncategorized on by .

Some empirically derived testing principles

Here are some principles I have found useful when testing Hypothesis. I don’t promise any of these are universally good, all I promise is that all of them have resulted in my finding bugs that I would otherwise have missed, and that together they seem to give me a lot more confidence in my software than I otherwise would have.

1. 100% coverage is mandatory.

This is a slight lie in that I do have the occasionally nocover or ignore pragma sprinkled around the code, but those generally occur in places that I’ve tried really hard to cover and it’s just not possible to hit reliably.
People seem to think that coverage is a bad metric and doesn’t really tell you a lot about your testing. There’s a line I have here which I believe is stolen from someone from the sqlite project:

100% coverage tells you nothing, but less than 100% coverage tells you something.

Code that is not tested almost certainly contains bugs. Code that is tested probably still contains bugs, but the probability is lower. Therefore, all other things being equal, code with 100% coverage is likely to contain fewer bugs.
This is essentially operating on the principle that metrics make great constraints and lousy goals. The goal is not to maximize coverage, the goal is to test our project well. 100% coverage the starting point, not the finish line.
(“Coverage” here means “branch coverage”. In reality it means “The finest grained coverage metric that you can get out of your tooling and that it’s meaningful to claim 100% coverage on”)

2. All bugs result in tests.

The core principle of this is hopefully both self-explanatory and uncontroversial. Step 1 to fixing a bug is to write a test for it. Otherwise the bug might reoccur.
But it goes deeper. Every bug is actually two bugs: A bug in your software, and a bug in your test suite.
The former is obvious. A bug in your software is a bug in your software. The latter is more subtle: It’s a bug in your test suite because your test suite didn’t find this bug, and that indicates a failure of your testing.
So. You’ve found a bug. Why didn’t your test suite catch it? What general failure of testing this indicates, and can you fix that failure in a way that would catch other instances of similar bugs?
As a concrete example: In Hypothesis’s data serialization layer, one of the invariants is that an attempt to deserialize bad data can only raise a BadData exception. Any other exception is a bug. This invariant is used to guarantee that Hypothesis can always read from old databases – the worst case scenario is that you can’t reuse that data, not that it crashes your tests.
In preparing for the 1.1 release I found an instance where this wasn’t the case. This then caused me to write some generic tests that tried fuzzing the data Hypothesis reads in order to find exceptions that weren’t BadData. I found 5 more.

3. Builds should fail fast.

There’s nothing more frustrating than getting to the end of a long build and then having the trivial thing that you could have found out 30 seconds into the build failing everything.
Linting is a major culprit here. lint should run at the beginning of builds, not at the end. I also have a separate entry in the build matrix which runs only a fast subset of the tests and checks the results for 100% coverage. This means that if I’ve forgot to test some area of the code I find out fast rather than at the end.

4. Principles should never be satisfied by accident.

Suppose your test suite catches a bug in your change with a failing test. Is that test actually the one that should have caught it? This particularly comes up with internal APIs I find. Testing internals is important, and if a bug in an internal was only caught because of a bug it caused in the public API, that internal could use a more specific test.
This also plays well with the faster build step for coverage. By forcing coverage to happen on simple examples it ensures that code is covered deliberately rather than as a result of testing other things. This helps offset the problem that you can often get to 100% coverage without ever really testing anything.
Flaky tests are often an example of something happening by accident. Sometimes it’s just that the test is bad, but often it’s that there’s some hard to trigger condition that could use a dedicated test that hits it reliably.

5. You need both very general and very specific tests.

Tests need both breadth and depth. Broad tests are things like Hypothesis which ensure that your code is well behaved in every scenario (or at least as close to every scenario as they can), but they tend to have very shallow definitions of “well behaved”. Highly specific tests can assert a lot more about what the code should do and as a result are true in a much narrower set of environments.
These tend to catch different classes of bugs.
So far to the best of my knowledge nobody has come up with a way of testing things that achieves both breadth and depth at the same time other than “do a massive amount of work manually writing tests”, but you can do both by writing both kinds of tests, and this is worth doing.

6. Keep trying new ways of testing things.

It’s been my experience that every time I try to come up with a new way of testing Hypothesis I find at least one new bug in it. Keeping on coming up with new and creative ways of testing things is a great way to keep ahead of your users in finding bugs.

7. There is no substitute for hard work.

Unfortunately, quality software is hard. I spend at least as much effort testing Hypothesis as I do writing it, and there’s probably no way around that. If you’re taking quality seriously, you need to be spending as much work on quality as functionality.

This entry was posted in Uncategorized on by .