Why I am no longer #YesToAV

As you might recall, there was a referendum four years ago that would have significantly affected the results of the upcoming general election.

Entrenched power very successfully persuaded the British public that electoral form was dangerous and scary and might cause the wrong person to win, and they correspondingly voted against it.

I was, and am, very angry about this.

What I didn’t notice at the time was that the entire referendum was a massive con, that every option was a bad one, and AV would quite possibly have been even worse than the status quo.

Viewed as an alternative to FPTP in the abstract, AV is undoubtedly the better system. It’s not a great system but say we were to want to e.g. elect a mayor then AV is unambiguously preferable to FPTP (indeed, the Mayor of London is elected with a slightly weird and limited form of AV – it’s AV where you can only choose a first and second preference).

The problem is that electing a parliamentary body like the house of commons is not like electing a mayor, because you’re performing the same class of vote many times. This is what gives rise to things like Gerrymandering, but it also means that choosing apparently better electoral systems can give you globally worse results.

AV was sold as reducing tactical voting by allowing you to vote for your actual first choice preference first and not “waste” your vote. The problem is that in reality your first choice vote ends up being a red herring if it’s not for one of the two majority parties, because your candidate will drop out of the race anyway. Sure, you nicely signal your support, but ultimately the only thing about your vote that gets counted is whether you prefer Labour or Conservative.

The problem is that although AV raises the number of votes a minority party will get, it also raises the number of votes they need to get by an even greater margin.

Under FPTP, a minority party can and often will get a seat in a divided area because parties frequently win with a relatively small fraction of the vote, and because voters can “punish” the majority party they favour by defecting to a smaller party. Under AV, this doesn’t happen – they still vote for a major party, and the result is that rather than benefiting smaller parties as was claimed, AV reinforces the two party system.

Does this make AV the “worse” system? No. But I think it’s fair to say that it makes implementing AV at the constituency level a really bad idea. Fundamentally this is a problem with constituencies more than it is a problem with AV, but that doesn’t make implementing AV at the constituency level a good idea just because it’s “not it’s fault” that it would make things strictly worse.

I still think it’s clear that some form of electoral reform is required, ideally some flavour of proportional representation, and if the referendum came up again I would probably still vote yes simply to be able to signal that electoral reform was needed, but I think it’s important to realise what a massive scam that entire referendum was, and reflect on the complete lack of honesty and/or competence that that entails in the people proposing it to us.

Notes on pilling a cat

This little monster is called Dexter.

He’s a lovely cat, really. A bit whiny, slightly skittish, but otherwise very nice. He likes skritches and laps and is generally very polite when there isn’t a door he wants to be on the other side of.

Unfortunately he also has a variety of exciting bladder problems. He’s currently on a five pill a day regime in an attempt to sort this out.

Dexter does not like pills.

We are lucky in that one of the things Dexter does not do is attack people. He might injure you in an attempt to get away when sufficiently panicked (ask me about bath time for kitties), but he never actually attempts to harm you no matter how much he disapproves of your actions.

Nevertheless, we’re engaged in an arms race. It used to be the case that I could just cradle him in my arms, stick a pill in his mouth, and he would swallow.

That is, uh, not the case any more.

Here is our current cat pilling protocol, presented in case it will be useful to others.

You will need:

Two people. Don’t even think about trying this with one. We are considering whether adding a third person to the protocol would be useful.
All of the pills extremely conveniently to hand.
A dropper pipette with water in it. A syringe would also work but you need to be good at not squirting it excessively.

There are two people in this protocol. For sake of avoiding ambiguity, I will refer to them as David and Dave. David is sitting, Dave is standing. The role of David is cat control, the role of Dave is pill application.

David cradles the cat in his arms lovingly. He pretends this is to lure the cat into a false sense of security, but actually the cat knows exactly what is about to happen.
David sits down with the cat flat on its back and head pointing away from him, wisely exposing his belly to the sharp back legs (you may wish to take protective measures if you have a less polite cat than Dexter).
David grabs the cat by the scruff of the neck. If the cat does not look in mortal terror for his life at this point from how much his face is being pulled back, David has not grabbed enough scruff.
David traps the front paws, which the cat has learned to move while scruffed, under an arm and levers the cat’s jaws open.
Dave swiftly moves in, drops a pill into the back of the cat’s throat, and follows with a squirt of water.
David holds on very tightly. The first time we squirted water into Dexter’s mouth he somehow teleported across the room.
David prises open the cat’s mouth to check that the pill has indeed been swallowed.

Something about the squirt of water seems to cause Dexter to instantly swallow. This appears to be a general phenomenon according to the internet.

In order to understand these drastic measures, here is a list of things that don’t work any more or never worked:

Dexter is terrible at eating. He doesn’t really do it. He certainly doesn’t do it if you try to feed him from your hand. As a result none of the standard “wrap the pill in food” measures have ever been effective on him.
Cat burrito is not a useful control procedure for Dexter. We used to try to use this to control him, but it just freaks him out more because he knows exactly what’s about to happen. We end up having to reset the burrito between each pill, which causes him to freak out more. It’s easier to just hold his legs in place with body weight and, if necessary, hands.
Releasing his scruff, holding his mouth shut and stroking his throat results in him very helpfully makes swallowing motions. You then later discover he’s stored the pill in his cheek.
Anything that allows him to move his jaw. He chews up the pill and starts foaming pill residue at the mouth. If we didn’t know the cause of it, the sight of our cat foaming red at the mouth would have been quite unsettling.

I expect the current protocol is only going to work for so long before Dexter figures out how to not swallow when you squirt water into his mouth. Fortunately I expect to have moved out by then.

Edit to add: I had to pill the cat solo tonight. The burrito technique worked… well at least it worked better than trying to handle the cat without the towel. The experience was significantly less pleasant for both human and feline participants, but I didn’t have enough hands to do it any other way.

Tests are a license to delete

I’ve spent the majority of my career working on systems that can loosely be described as “Take any instance of this poorly specified and extremely messy type of data found in the wild and transform it into something structured enough for us to use”.

If you’ve never worked on such a system, yes they’re about as painful as you might imagine. Probably a bit more. If you have worked on such a system you’re probably wincing in sympathy right about now.

One of the defining characteristics of such systems is that they’re full of kludges. You end up with lots of code with comments like “If the audio track of this video is broken in this particular way, strip it out, pass it to this external program to fix it, and then replace it in the video with the fixed version” or “our NLP code doesn’t correctly handle wikipedia titles of this particular form, so first apply this regex which will normalize it down to something we can cope with” (Both of these are “inspired by” real examples rather than being direct instances of this sort of thing).

This isn’t surprising. Data found in the wild is messy, and your code tends to become correspondingly messy to deal with all its edge cases. However kludges tend to accumulate over time, making the code base harder and harder to work with, even if you’re familiar with it.

It has however historically made me very unhappy. I used to think this was because I hate messy code.

Fast-forward to Hypothesis however. The internals are full of kludges. They’re generally hidden behind relatively clean APIs and abstraction layers, but there’s a whole bunch of weird heuristics with arbitrary magic numbers in them and horrendous workarounds for obscure bugs in other peoples’ software (Edit: This one is now deleted! Thanks to Marius Gedminas for telling me about a better way of doing it).

I’m totally fine with this.

Some of this is doubtless because I wrote all these kludges, but it’s not like I didn’t write a lot of the kludges in the previous system! I have many failings and many virtues as a developer, but an inability to write terrible code is emphatically not either of them.

The real reason why I’m totally fine with these kludges is that I know how to delete them: Every single one of these kludges was introduced to make a test pass. Obviously the weird workarounds for bugs all have tests (what do you take me for?), but all the kludges for simplification or generation have tests too. There are tests for quality of minimized examples and tests for the probability of various events occurring. Tuning these are the two major sources of kludges.

And I’m pretty sure that this is what makes the difference: The problem with the previous kludges is that they could never go away. A lot of these systems were fairly severely under-tested – sometimes for good reasons (we didn’t have any files which were less that 5TB that could reproduce a problem), some for code quality reasons (our pipeline was impossible to detangle), sometimes just as a general reflection of the culture of the company towards testing (are you saying we write bugs??).

This meant that the only arbiter for whether you could remove a lot of those kludges was “does it make things break/worse on the production system?”, and this meant that it was always vastly easier to leave the kludges in than it was to remove them.

With Hypothesis, and with other well tested systems, the answer to “Can I replace this terrible code with this better code?” is always “Sure, if the tests pass”, and that’s immensely liberating. A kludge is no longer a thing you’re stuck with, it’s code that you can make go away if it causes you problems and you come up with a better solution.

I’m sure there will always be kludges in Hypothesis, and I’m sure that many of the existing kludges will stay in it for years (I basically don’t see myself stopping supporting versions of Python with that importlib bug any time in the near future), but the knowledge that every individual kludge can be removed if I need to is very liberating, and it takes away a lot of the things about them that previously made me unhappy.

Important things I have learned about Twitter on Android today

You know that thing where the Android twitter account wants to “use your installed apps to customize your experience”? The latest in privacy intrusion features from the app that also wants to upload all your contacts to Twitter’s servers and will require you to remember to uncheck the box literally every time you set up a new account on any device ever if you don’t want your entire address book to be shared with them.

Well, as with all of the other obviously global settings on Twitter’s Android app (such as “No you may not beep at me. Oh god why are you beeping. Stop, please.”) it’s a per account setting. If you have multiple twitter accounts set up on the app, you need to turn off the “It’s OK to spy on the rest of my phone” on each one.

This was finally enough to prompt my growing desire to uninstall this app, so I did.

At which point I was reminded that Twitter’s mobile website is of the “Hey? Did you know we have an app? Have you forgotten since the last page that we have an app? Maybe you’d like to install our app? Have you maybe changed your mind about maybe installing our app? How about now?” school of design.

Which is why the second important thing I learned is that if you want to not be constantly annoyed by the only way to use Twitter on your phone without it trying to take all the rest of your data along with it, the solution is to click the “Request Desktop Site” checkbox in the settings menu. You’re on the mobile subdomain, so you’ll still get the mobile site, but the UA won’t say you’re on Android so you won’t get the constant needy requests to install the app.

Things that are not documentation

I’m about as bad at documenting my code as the average software developer, but I’m trying to be better. With Hypothesis I think I’m making good progress on that front, though I feel like it’s still very visibly someone who isn’t actually good at documentation but is trying to do better.

One of the first steps of doing better is not to fool yourself with substitutes. The following is a short, incomplete, list of things that developers falsely believe to be documentation:

The code itself
The tests for the code
The comments in the code
The type signatures
The person who wrote the code being available for questions
Discussions in your issue tracker

Here is a short, complete, list of things that are actually documentation:

A document, written in a natural language, which describes how to use your software.

If it is not on this second list, it’s not documentation.

All of the first list are useful tools which aid program comprehension. The presence of good documentation does not remove the need for them. However the presence of them also does not remove the need for good documentation.

The confusion comes, I think, from people confusing “substitutable to a point” with “substitutable”. Type signatures, or tests, can fill some of the use cases of documentation, and can reduce its need for a time, so it’s tempting to think of them as a sort of documentation, but they cannot actually fill the niche of comprehension that documentation enables.

Let me try an analogy: Consider coffee and sleep, subjects dear to my heart. Can you substitute coffee for sleep? Certainly, up to a point – if you’ve had a bad night, coffee will help. Can you substitute sleep for coffee. Certainly. I’ve heard rumours from people who are familiar with the concept that if you have a good night’s sleep then you need less coffee the next day. Can coffee improve alertness even in the presence of enough sleep? Yep.

Is coffee a type of sleep? Uh, no.

The fact that two tools solve overlapping problems is no excuse for confusing them.

Why am I taking such a hard line about this?

It’s because developers hate writing documentation but know that it’s a thing they’re supposed to do.

So if you let people believe that something that is not documentation is documentation, they’ll just do that instead and tick the box that says “Yep! I documented it”, and feel good about themselves for having writing code that does not, in fact, have documentation.