With the recent death of Margaret Thatcher, one of the things that got brought up was the percentage of the popular vote her party got during the elections for which she lead it: 43.9% in 1979 and 42.4%. In particular I saw these figures quoted as mentioning that while they look low they are in fact really high.

Depressingly, this turns out to be true. I looked up the data on the Wikipedia entry on UK general elections, and it turns out that the last time a UK government was elected with more than 50% of the popular vote was 1931 (it was the Conservative party).

In the course of doing this I scraped the tables on this page into a slightly more usable form. Here’s a gist with the scraper and the parsed data as a tab delimited table.

I shouldn’t really be surprised by this result. It’s well known that this sort of thing can happen under a regional system, especially under FPTP, but I confess I had somehow missed that it not only can and does happen but that it’s actually completely routine.

One of the issues that often gets cited as an objection to random ballot is that it’s too random. What happens if a lunatic fringe gets voted in to power? People might get a majority simply by random chance.

This is just a brief note on how unlikely that is.

Suppose there are \(N\) seats, and we have a party with a fraction \(q\) of the vote. Let \(W_i\) be a random variable which is 1 if the party wins seat \(i\) and 0 if it loses. Let \(W = \sum W_i\) be the number of votes won and \(q_i = P(W_i = 1)\).

\(W\) is a sum of independent random variables. I’m going to assume that \(N\) is large enough that we can apply the law of large numbers and approximate it as a normal random variable.

So from now on we’ll pretend \[W \sim \mathrm{Normal}(qN, \sigma^2)\]

Then the probability of getting a majority is
\[
\begin{align*}
P(W > \frac{1}{2}N) & = P(Z > \frac{(\frac{1}{2} – q)N}{\sigma})\\
&= 1 – \mathrm{erf}\left((\frac{1}{2} – q)\sqrt{\frac{N}{q(1-q)}} \right) \\
\end{align*}\]

erf is monotonic increasing, so this value is maximized by making the argument as small as possible. Everything in there other than \(\sigma\) is fixed, so this is done by maximizing \(\sigma\), or equivalently \(\sigma^2\)

When does this happen?

Well \(W\) is a sum of independent random variables, so its variance is the sum of their variances. So \[\sigma^2 = \sum q_i(1 – q_i)\]

We want to maximize this subject to the constraint that \(\sum q_i = qN\). We can turn this constrained optimisation problem into an unconstrained one by adding a lagrange multiplier t. So we’re optimising the function \[f(q_i, t) = \sum q_i(1 – q_i) + t (\sum q_i – qN)\]

Taking partial derivatives we have

\[\frac{\partial f}{\partial q_i} = 1 – 2 q_i + t\]

Setting this to 0 we get

\[ q_i = \frac{1 – t}{2} \]

The significant part of this being that it’s a constant that doesn’t depend on i. So applying the constraint we have \(q = q_i\), and \(\sigma^2 = N q (1 – q)\).

(I think it’s probably possible to reach this conclusion without the normality assumption, but I haven’t thought through the details).

Thus
\[
\begin{align*}
P(\mathrm{majority}) & = 1 – \mathrm{erf}\left(\frac{(\frac{1}{2} – q)N}{\sigma}\right)\\
& = 1 – \mathrm{erf}\left((\frac{1}{2} – q)\sqrt{\frac{N}{q(1-q)}}\right)\\
\end{align*}
\]

At this point we’ll just plug some numbers in. Here’s some python code:

import math
from scipy.special import erf
 
def probability_of_majority(n,p):
    sd = math.sqrt(n * p * (1 - p))
    deviations_to_majority = n * (0.5 - p) / sd
    return 1 - erf(deviations_to_majority)

At this point let’s just plug some numbers in.

Consider the UK house of commons with its 650 seats. Suppose we have 40% of the vote. What are the chances of us getting a majority?

Plugging them into our python code we get that the answer is 1.84e-13. i.e. if we ran a general election 5 times a year between now and the point when homo sapiens first started to appear, it would still be extremely surprising if any parties with 40% of the vote had ever achieved a majority.

At 45% of the vote we have a probability of 0.00029. If we ran an annual election every year since the birth of Christ, it’d still be more likely than not this would never have happened but it wouldn’t be very surprising.

At 48% of the vote we’re up to 0.15. At this point we expect this to happen every few general elections.

So the lunatic fringe? They’re not going anywhere. The lunatic majority? Well, that’s what you get in a Democracy. But unless someone actually has a majority they’re not getting in.

Compare this to the percentage of the popular vote with which parties typically reach a majority in the UK under a deterministic system.

From this I can only conclude that deterministic voting (in a constituency based system) is simply too random to be practical.

As you might have gathered, I’m a little bit in favour of electoral reform. I hide it well, but you might have got an inkling of it.

I was very keen on the yes to AV vote, not because I think AV is great – I’m not even 100% sure it would have been an improvement – but because I think the current system is deeply broken and that a vote no can be, would be and indeed was spun into a vote saying that the status quo was A-OK and we shouldn’t change anything.

I don’t think there’s a hope in hell of my preferred choice of voting system ever getting enough popular support. It’s so unlikely I would never even try to be honest – no one is going to be willing to burn political capital over the idea.

But there’s an idea that’s been knocking around in the back of my head for a while which I think could actually work. It’s in some ways a little drastic, but I think with the right spin it could be actually appealing to people.

Here it is, at its simplest:

Each constituency gets to choose its own electoral system.

That’s kinda it. The regions who want to use AV (there weren’t very many of them) get to use AV. The regions who want to use random ballot get to use random ballot (you might get one of them, but I doubt it). If you want to sell range voting to a region, go for it.

Why is this a good idea?

It’s partly because will to change is often very localized. It’s comparatively much easier to get momentum behind a local campaign. As we demonstrated in 2011 it’s very hard to persuade a nation of 63 million people to change its ways. A region of about 70,000 people in comparison is much more localized and experiences a much more coherent set of shared problems. A campaign can target specific issues (This MP gets voted in every year and has for the last 20. Isn’t it time for a change?) and appeal to a more common shared set of experiences.

I also think it just might be a good idea independently of the will to change. It gives you more capability to experiment. Whenever you change the voting system wholesale there’s the possibility of a whole general election going catastrophically wrong. When you change it for a couple regions the worst case scenario is that you have a couple rubbish MPs. Insert your own political commentary here. If the changed electoral system appears to work well then its popularity can spread. If it doesn’t appear to work well then it probably die out.

Here’s how I imagine it might work in practice. This is just a sketch and obviously might have some serious flaws in it.

There is a central registry of acceptable voting systems. This starts out with a single entry: The existing FPTP system.

There is a small (initially) committee whose job it is to vet voting systems. They’re probably appointed. They should consist of a mix of people from different backgrounds – I’d want at least one academic who studies the subject of voting systems and one person with practical experience of being involved in elections. Ideally more. Their job is to assess proposals for electoral systems according to three criteria: Anonymity, unbiasedness and practicality of implementation. Unbiasedness here being both “No single voter has more power than any other voter” (in the sense of “if you swapped who cast these two votes would the election result change”, not in the sense where FPTP gives more power to majority parters) and “No single party has an advantage over any other” (Again in the sense of vote swapping). Practicality of implementation includes things like “Is this completely incomprehensible to people?” (this should be solved by consulting actual people and seeing if it’s comprehensible to them rather than stupid rhetoric). Whether the voting system would be a good idea or not is explicitly outside of their remit.

Voting systems are privately proposed rather than via the committee. An initial proposal for a voting system consists of:

• A description of the implementation of the system, including how people would vote and how the votes would be counted to produce a result. This is not required to completely specify all implementation details but should be unambiguous
• A list of not more than 5 people who would be responsible for seeing the proposal through to completion
• A list of at least 10,000 signatures of people who are prepared to endorse the proposal

The committee then makes a decision on whether it is worth progressing with this initial proposal or whether it is fatally flawed (fatal flaws include “You have proposed a dictatorship” or “Your proposal requires the use of a super computer to implement if there are more than 5 candidates for election”. It does not include “You’re kidding, right?”). If they think it is worthwhile, they will provide a small grant for turning the proposal into a final result. This grant should be enough to pay one or two people a modest salary, fund any research that is needed, etc. I have no idea what is practical here.

A finished proposal for the system should include:

• A guide on how the voting system works for normal citizens who have to cast a vote. This should be purely practical in nature and without spin
• A detailed account of how the ballots are to be counted

This should be produced through a process of iteration: A draft form is submitted to the committee and they will either accept it or reject it with comments. This process may continue indefinitely until the committee decided to put their foot down and say they will not accept any form of this proposal, however the grant will dry up at some point. Hopefully rather than either of these things happening the committee will at some point vote to accept the proposal. At that point the proposal enters the list of accepted voting systems.

(There needs to be some process for amendments, etc. I don’t really care what it is. Do something sensible here)

At this point we now have a list of voting systems that are approved for use. Individual constituencies may now choose to switch voting systems. They do this as follows:

At any point a proposal may be put forward by a constituency to switch voting systems. Again, this is privately arranged. The proposal consists of one of the voting systems from the approved list, some suitable deposit (about £10k probably) and signatures from at least 5% of the constituency saying they would like this voting system to be implemented. Upon the successful submission of a proposal, a vote is triggered within that constituency that is a simple yes/no vote for switching to that system. There should be about a six month window between the proposal being accepted and the vote to allow people to properly respond to it. If the yes votes meet some threshold (probably somewhere in the 10-20% region) then the deposit is returned. If more than 50% of the voters vote yes then the constituency has now adopted that system and the next election for an MP there will use it (regardless of whether it’s due to general election, death or disqualification of an MP, etc). If the system changes, it may not be changed again until the new system has been used at least once. If the system is voted out there is now a cool-off period of a year where no one who put their name to this proposal may sign for a new one, and where the rejected system may not be proposed again. Other people are free to propose different systems. This is to stop people basically proposing systems to lock out other people.

Would this work in practice? I don’t know. I think it might. Initially at least you will get very few new systems – It seems likely that it would be a year or more before a new system even made it through the committee, and that you’d be unlikely to see more than a dozen non-FPTP constituencies for a decade or more to come – but I think it would open the door to change, and would avoid some of the visceral reactions to specific voting systems that many people have.

…and I have no idea what its properties are and whether it’s a good idea.

I was wondering how you might extend random ballot to multi-winner elections. Specifically the case where you have C candidates and you want to elect N of them. The following is the system I hit upon:

Voters provide a list of candidates they would like elected, in order of preference for how strongly they would like them to be elected. They do not have to rank all the candidates, though strategically I think it’s advantageous to rank the full candidate list.

We now run the following algorithm:

While we have elected fewer than N candidates, iterate the following procedure:

1. Remove all votes including only candidates that have already been elected.
2. If we have no votes left, something has gone terribly wrong and we can’t actually elect enough candidates.
3. Now pick a random voter.
4. Add the voter’s highest ranked preference who has not yet been elected to the list of elected candidates.

Note that the sampling is with replacement: We do not remove a voter when we pick their vote.

I currently have no idea what the properties of this procedure are, but I think it might be interesting. My intuition is that it’s probably strategy proof in the sense that you have no incentive to vote other than your true preference (if so, the sampling with replacement is crucial for this – otherwise you might rank a less preferred but also less widely popular candidate first), but I haven’t worked through the details so there’s a high likelihood of this being wrong.

I mentioned in the previous post that I now thought that random ballot was a genuinely good idea for electing your house of representatives. It occurred to me that I have never actually said why in a concise form. So this post is a list of awesome things about using random ballot in this way.

1. It provides a very strong form of proportional representation. If x% of the votes go to a candidate with a given property, the expected percentage of elected candidates with that property is x%. This property can be “party”, in which case we get something like traditional PR, but it can also be things like “opinion on this specific issue”, age, race or gender
2. It is empowering: Your vote always counts, no matter where you live. It is literally impossible for you to be disenfranchised by your constituency (this isn’t quite true in that voter turnout in your constituency affects exactly how much your vote is likely to affect the result, but it’s pretty near)
3. There is no spoiler effect: If candidate A steals votes from candidate B then while the probability of B being elected goes down, the probability of one of A or B being elected remains constant. An interesting consequence of this is that parties are free to run multiple candidates in a single seat
4. It is continuous: Small changes in opinion produce small changes in results. This prevents parties gaining a lot of power with very little change of mandate
5. It is sensitive: Small changes in opinion do produce small changes in results (well, result probabilities) rather than having no effect. This allows gradual changes over time much more easily and stops you getting stuck in a rut

That is all I can think of at the moment. I’m sure I’m missing some. I’ll add more as I think of them.