So my slightly crazy article about random ballot was mentioned in the new scientist.

This is, as you might expect, quite exciting. Unfortunately the article gets one really crucial thing about me wrong, and my comment clarifying this on the site has yet to be approved, so I just want to make this clear:

I am not actually advocating random ballot

Observe the following quote from the original article:

I think it has desirable and undesirable features (in particular there’s one really major problem with it), but even with the above list I’m not sure I actually endorse it. Consider this a thought experiment more than a proposal.

“perfect” was in scare quotes for a reason. It has some very nice properties, but it also has a lot of problems (both theoretical and practical). I have no interest in promoting it as the system we should actually be using, it’s just an interesting system to think about it. The reality is that (while it’s not necessarily the system I would have most preferred) I am extremely strongly in favour of the move to AV and will be voting yes on May 5th.

Update: Jacob has now fixed this, mentioning I am “highlighting” it rather than “pushing” it.

As you might have noticed by now, I’m quite strongly pro voting yes to AV in this referendum. Here are some links about that:

Dan Snow’s Alternative is a very simple and compelling explanation of how AV works and why that’s a good thing.

Politics in the Animal Kindom goes into more depth, including some of the problems with the current first past the post system and why AV avoids them.

For a more light-hearted video about AV, watch Is your Cat confused about the referendum on the voting system on the 5th May?. As well as being a hilarious cat video it’s also a good explanation.

In Is AV better than FPTP?, Timothy Gowers writes about the issues around the two systems. It’s very very long, but an extremely good breakdown. It starts out trying to be unbiased, but finishes with “I have largely failed in my aim to adopt a neutral tone. However, that is because most of the arguments put forward by opponents to AV have been clearly wrong”.

Well, yes. Indeed they are

I’ll update this post as I find good links. Feel free to suggest any in the comments / on twitter / etc.

Disclaimer up front: This is not about the referendum. It is a description of an AV variant with mathematically interesting properties. If your opinion on this subject can be described by a hashtag, please move along.

I posted a comment on hacker news yesterday in a thread about the randomized voting system I described. It describes another randomized voting system with preferential voting which is much more reliably guaranteed to elect “sensible” winners (of course, as a result it loses the statistically-PR property of my previous one, so it’s not a replacement in any way, but it’s still a good system). It goes as follows:

1. Every voter ranks some subset of the candidates in order of preference. They are considered to prefer every candidate they list to every candidate they don’t.
2. For each pair of candidates A, B decide whether the majority prefers A to B. A voter prefers A to B if they voted for both and ranked A higher or if they voted for A and not B.
3. For each candidate A prepare a “drop-out list”: The list of candidates B for whom the majority prefers A.
4. If you have one candidate, stop. Elect that candidate.
5. Pick one of the remaining candidates at random. Call this candidate Pivot. If there are any candidates left on Pivot’s drop-out list, those candidates drop out. Else, Pivot drops out.
6. Go to step 2.

This is based on a specialisation of Kwik-Sort (which computes the whole aggregate preference) from “Aggregating Inconsistent Information: Ranking and Clustering”, by Nir Ailon, Moses Charikar and Alantha Newman. The original algorithm is a probabilistically 11/7 tight estimate of the Kemeny optimal aggregation.

Despite the random nature, this approach has many desirable features. In particular it is a generalised Condorcet method – if there are two blocs ProKitten and AntiKitten such that for every A in ProKitten and b in AntiKitten the majority prefers a to b, then the winner will be ProKitten. In particular this system always elects the condorcet winner and never elects the condorcet loser (if either exist), as you can consider them to be a bloc with one entry.

But as the comments yesterday suggested, people really dislike the idea of randomness in elections. I don’t personally agree with this, though I do tend to think that if you can achieve results which are nearly as good without randomness then that is desirable.

So this got me thinking: The fact that the above algorithm is generalised Condorcet isn’t actually in any way dependent on the randomness (it kindof can’t be – otherwise it would only be probabilistically generalised Condorcet, as you could always randomly select the worst option). Thus you can use any voting method you like and select the least popular and the algorithm will still work.

In particular you can use AV. Here’s how it works:

1. Every voter ranks some subset of the candidates in order of preference. They are considered to prefer every candidate they list to every candidate they don’t.
2. For each pair of candidates A, B decide whether the majority prefers A to B. A voter prefers A to B if they voted for both and ranked A higher or if they voted for A and not B.
3. For each candidate A prepare a “drop-out list”: The list of candidates B for whom the majority prefers A.
4. If any candidate has more than 50% of the first votes, that candidate wins.
5. Pick the remaining candidate with the fewest votes. Call this candidate Pivot. If there are any candidates left on Pivot’s drop-out list, those candidates drop out. Else, Pivot drops out.
6. All of the people who voted for a dropped out candidate transfer their vote to their favourite candidate amongst the remaining ones (or abstain if they have no candidates they can stand amongst the remainder)
7. Go to step 4.

This remains a generalised condorcet method: Consider ProKitten and AntiKitten. If the pivot we pick is ProKitten and there are any AntiKitten candidates left, those candidates drop out (if there are not then the winner is obviously ProKitten). If the candidate we pick is AntiKitten then the only candidates who can drop out are AntiKitten because the ProKitten candidates are preferred to this one so are not on its drop-out list.

That only leaves the final step, but it’s obvious that if a candidate has > 50% of the first vote then they must be ProKitten, because otherwise they would be AntiKitten and preferred to ProKitten candidates.

So, we have a mild variation of AV which is generalised Condorcet. Do I think this is the best system? No idea. Haven’t done a detailed analysis of it. But it seems significantly easier to explain to people than many of the Condorcet methods (because despite what a certain hashtag would have you believe, AV is really easy to explain, and this is not substantially more complicated) and retains the intuitive appeal of AV whilst fixing its worst mathematical issues.

So my previous post seems to have struck a chord. I only wrote it this morning and it’s my second most popular article (and the most popular one has been steadily gathering traffic since 2007 and has been posted to reddit twice. Wow.

Anyway, it’s a cool idea. I like it. It has a lot of aesthetic appeal. But it’s not the system I’m going to be voting for.

On May 5th, we (the UK) will be asked “‘At present, the UK uses the ‘first past the post’ system to elect MPs to the House of Commons. Should the ‘alternative vote’ system be used instead?”. I’m going to be voting yes. I hope you are too.

I could at this point launch into a long article as to why I think you should, but instead I’d rather ask: If you’re reading this, and you’re thinking of voting no, can we talk about it? Post a comment, send me an email, whatever you’d like really. I’d like to know, and I’d like the chance to change your mind.

And if you’re reading this and thinking of not voting, please do vote. Regardless of which side you choose, this is really important.

AN UPDATE FROM THE FUTURE: I come to you from the year 2013 to tell you that there is a new version of this post. This is one of my popular posts ever, but I have since updated my beliefs around it somewhat and I’m a little embarrassed by some of the writing in this one. It will remain around for posterity, but as a persuasive and descriptive piece you should really be reading and referencing my later piece “Towards a more perfect democracy”

No, seriously, go read that one instead of this. Shoo.

ARCHIVE VERSION

There are a lot of theorems around voting. Many of them get trotted out in highly inappropriate ways. But even interpreted correctly their results sound quite disturbing: There are no fair voting systems, tactical voting is always possible, etc. Sorry. That was wrong too. There are no fair voting systems within the class of voting systems the theorems describe. The voting system I am describing is not a member of those classes, so the theorems do not apply.

I am going to propose to you a voting system you may think is impossible. It has the following properties:

• It is as easy to explain as FPTP. Possibly easier. In particular your vote is identical to that under FPTP – you cast a vote for a single candidate
• It is constituency based, with one representative per constituency, and the constituencies may be arranged however you like – geographically in particular works fine. It thus upholds our fine British traditions, unlike the PR systems which people seem to object to because they can’t have their letter ignored by their own personal MP.
• As long as the constituencies have equal populations, no voter is disenfranchised by the constituency in which they live. (Larger constituencies are inherently disenfranchising in single-representative-per-constituency models – by definition you have a larger number of people having the same number of votes as a smaller number of people)
• The results will usually be close to proportional representation. Not always, but usually (this qualification will make more sense later).
• There is no incentive to vote tactically. None. You vote for the candidate you most want to see in office. Anything else would be madness

Given the above shopping list, you probably think I’m misrepresenting the facts. It sounds too good to be true.

Well, it kindof is. There is a catch, and it’s a big one. I think it has desirable and undesirable features (in particular there’s one really major problem with it), but even with the above list I’m not sure I actually endorse it. Consider this a thought experiment more than a proposal.

So, what is this magic voting system that flagrantly disregards all the theorems that should say it’s impossible? Here it is:

1. MPs run as they currently do – a smallish number of candidates stand for a single constituency
2. People cast a vote as they currently do – each selects a single candidate to cast their vote for
3. Once all of the votes have been cast, you pick one voter at random and use their choice
4. Wait, what?

At this point you probably think I’m crazy. Well, maybe so, but I’m crazy like a – actually, no. Let’s not go there.

In order to address some of the obvious objections to this, I will conduct an interview with someone who knows I’m crazy. Me. Or rather my online handle.

DRMacIver: So, are you crazy?
David: Probably. But this is actually a remarkably sane idea. It goes against our intuition, but our intuition is rubbish for voting.
DRMacIver: It is? Why?
David: Beyond the scope of this. Google for “Voting Paradox”. Or read these Socratic Dialogues on voting.
DRMacIver: How did you just include a link in a spoken interview?
David: Magic.
DRMacIver: Ok, so, some real questions. Why do the various things like Arrow’s Theorem or the Gibbard-Satterthwaite theorem not apply?
David: Because they only apply to deterministic systems. This one has a random element. It’s essentially a very specialised case of randomly selecting a dictator for each decision (which is itself a form of sortition).
DRMacIver: Is there any precedent for this?
David: For this particular one? No, not really. At least, not that I know of. Election by lottery has some precedent though. The Doges of Venice were elected by a complex system involving the drawing of lots and election by lot was a major and essential feature of Athenian Democracy
DRMacIver: Huh
David: Quite. I genuinely don’t know why it’s not used more today.
DRMacIver: Why should it be? What’s good about it?
David: Well the advantage of the Athenian system is that decision making is not put in the hands solely of those who most want to be in politics and can convince people to elect them – which has an unfortunate tendency to result in wealthy people with too much time on their hands being the ones who actually get into power – but instead spreads the decision making process throughout the population.
DRMacIver: So why aren’t you proposing that instead?
David: Well, I’m not proposing anything. Given the choice between Athenian Democracy and what I’ve described, I genuinely don’t know which one I’d choose. What I like about my current proposal is how remarkably close it is to the current system whilst fairly resolving all sort of injustices with the current system.
DRMacIver: Like what?
David: Well, there are 650 members of the house of commons. Suppose 10% of the population supported the kitten party. How many kitten MPs do you think there should be?
DRMacIver: Well, clearly this is a proportional representation argument, but I’ll play along. Let’s see, carry the one… There should be 65 seats.
David: Right. But suppose those 10% were instead spread uniformly across the country: Each constituency only has around 10% kitten party supporters.
DRMacIver: Right. And thus no constituency elects a kitten party MP because they’re nowhere in the majority. This is classic PR stuff. But people like having their own personal MP tied to their area – it’s one of the reasons PR doesn’t get any traction.
David: Indeed. But the convenient feature here is that you get to keep that while still getting something close to PR.
DRMacIver: Huh? How is what you’re proposing close to PR? It sounds like it has exactly the same problem.
David: Ah, but this is where the magic of probability comes in! Each of those constituencies will have about a 10% chance of electing a kitten party MP. So roughly 10% of the constituencies will get a kitten, and you’ll get about 65 of them in parliament.
DRMacIver: There were a lot of weasel words in that sentence.
David: Yes, there were. And this is where the least desirable of the proposal comes in: There’s a lot of purely random variation in a party’s support. The kitten party (with their 10% of the vote) will typically have a variation of about 15 seats purely by random chance: That’s a lot of their membership. For larger parties the variation isn’t much larger – if you have 50% of the vote then you expect your seats to vary by about 25 either way. In a close race this can be a big deal.
DRMacIver: Let me see if I got that. If you have two parties with roughly the same amount of support, you’re essentially flipping a coin to see which one wins?
David: Yes. Which is of course massively different from the current system.
DRMacIver: I’m going to let that one slide… so what happens if, for example, you just randomly select all 70 people in the country who voted for the “Church of Christ the Bus” party?
David: What happens if all 70 of those people win the lottery and put their money into funding that party?
DRMacIver: What?
David: The chances are really really low. If you’re only expecting to get a tiny fraction of a single seat, the chances of you getting that one seat are slim and the chances of you getting more than one seat are basically negligible. Also, because minority parties tend to be geographically clustered rather than spread out everywhere the maximum number of seats they can get is basically bounded.
DRMacIver: So this can’t elect crazy minority parties?
David: Oh no, it totally can if the minority is large enough. For example the BNP have a shot at getting in under this system.
DRMacIver: Isn’t that a massive flaw?
David: No. I don’t regard it as one. The BNP have a percentage of the voters, they should get a percentage of the say. The fact that I consider someone to be utterly despicable doesn’t give me the right to disenfranchise them. Also, as Timothy Gowers so eloquently put it, WE SHOULD NOT LET THE BNP DICTATE HOW WE RUN OUR POLITICS.
DRMacIver: Ok, fine, but let me make sure I understand: It is possible for this system to elect an MP which is supported only by a tiny minority of the constituency?
David: Yes, absolutely. It’s a trade off. I don’t necessarily think that’s a bad thing though.
DRMacIver: So a staunchly Labour area could get a Conservative MP?
David: Yes, that’s right.
DRMacIver: Are you totally fucked in the head?
David: I don’t think so.
DRMacIver: How is that not completely broken?
David: Well, for starters, it’s a nice thing in that it stirs things up: If you try something different every now and then, maybe it won’t be as awful as you thought it would be. You might learn something. But more importantly it’s a trade off – in the kitten party example, 10% of the population are currently disatisfied. In the new system 10% of the constituencies will get a kitten MP, which will disastify 90% of them. that’s 9% of people disastisfied by the kitten MPs, which is 1% of the population better than before. That’s some six hundred thousand people we’ve just made happier with kittens.
DRMacIver: Hrm. It feels wrong.
David: But how can it feel wrong when the numbers are so right?
Maf: Assuming only two choices (kitten vs. non-kitten) in your proposed system, isn’t the proportion of dissatisfied voters actually 18%, i.e. 90% of the voters in the 10% of the constituencies that get a kitten MP plus 10% of the voters in the 90% of the constituencies that do not?
David: Bugger. You’re absolutely right. This scuppers a key part of my argument. I have to go away and think about this (and read more). Perhaps “number of happy people” is the wrong metric to be considering, as it should have been obvious that that was always going to be maximised by picking the candidates with the majority first-preference.
DRMacIver: So your idea is shit?
David: No, all the desirable properties I listed still hold. It’s just not a happiness maximiser.