I’ve been thinking about the queuing model of business I posted the other day and I’ve realised it has a bit of a flaw that I’d missed because it’s not really relevant in the original medical context I came up with this in. Fortunately, some thinking around this flaw has caused me to come up with a model that I think is actually better, even for the original medical application.
The problem comes in two parts, but really they’re the same core issue: Your highest paying customer can always hog resources (this was much less of a problem for the medical case because you’re naturally limited in terms of how many resources you can actually consume as an individual).
Consider the following two problem cases:
- Your top customer wants to transcode 1000 videos
- Your top customer is asking for the meeting room every day
In the first case your customer is spamming the queue with a large number of jobs. Each job takes up a position in the head of the queue and they all have to be cleared for any jobs from other customers to be cleared.
In the second case you are not filling the queue at any point, but the fact that you are repeatedly queue jumping means you’re getting a highly disproportionate share of the resources.
So how do we fix this?
The basic idea remains the same: You have a FIFO queue and a second queue which allows people who pay more to get processed sooner. However we make two modifications:
- The second queue is a queue of customers, not jobs. Whenever a customer’s turn comes up, their earliest submitted job gets processed.
- The second queue isn’t really a queue at all. It’s a probability distribution.
The second point might require some elaboration…
The idea is simple: If we have three customers with scores 2, 1 and 1 respectively, we want to give the first customer 50% of the resources, but no more than 50% of the resources. The second two should split the remainder equally.
So what we do? We run a lottery! (yaaaay).
When we want to process a paying customer, we pick a customer at random with probability proportional to their score. So someone with twice as high a score gets picked twice as often.
And, yeah, that’s pretty much it.
As well as being fairer as less prone to pissing your customers off, this also has a really nice business benefit: People always have an incentive to pay you more. In the old model there was very little benefit to increasing the amount you pay unless you increase the amount you pay by a lot. In the new model every penny you add increases the percentage of the priority resources you get and thus is potentially of benefit to you.