Hi. This post is for information – I’m not actually hiring at the moment, but when it next happens I would really like to know how to do it right. Also this is entirely a post off my own bat, rather than an official Aframe position on anything (though I’d expect and hope the rest of the company would agree with me).
So I work with a really great development team at Aframe. We’re a bunch of competent interesting people, and everyone is well motivated and good at their job. It’s nice.
There’s just one problem…
I say “people”, but that’s because I tend to default to gender neutral nouns and pronouns. I really mean “men”.
Generally speaking, straight white men.
I’m not 100% sure how this happened.
As far as I can tell, it’s not the result of explicit prejudice on our part. I don’t think it’s even the result of implicit prejudice, and I’d like to tell you a little bit about our current hiring process in order to back that up.
Here is the process we used for our recent hiring of back end developers:
- Applicants come in to our development manager. He does a basic screen for “not an utter waste of time” (e.g. “I am definitely suitable for your senior backend developer role because I know photoshop”) then passes the candidate on to the rest of us.
- We review the answers to a set of screening questions. We do not even know the candidate’s name, let alone CV, at this point. They are given a candidate number.
- If we like their answers, we send them a coding test. Still just a number, not a free person.
- If we like their answers, we’ll take them through a phone interview. We will vaguely look at a CV at this point for context for the interview. If their answers are borderline and we’re inclined to reject we might use the CV to get more evidence that they’re worth talking to, but we never rejected anyone on the basis of their CV at this stage.
- In the phone interview we ask the same list of questions of everyone. There is of course some general chatter as well, but in theory the decision is made on the basis of the answer to those questions.
- If they pass the phone interview, we take them through to an in person interview. This is a series of specific, structured, tasks.
- We make a hire/no hire/wait and see decision based entirely on the outcome of those tasks.
I largely think this is an example of Doing It Right. It’s not perfect, and we’ve had some complaints about specific details which we’re trying to improve, but it does a pretty good job at eliminating biases.
Except the only people who made it through to the phone interview stage in the recent hires I was involved in (for which we hired three people in the end) were white men.
I honestly have no idea if this was representative of the candidates we didn’t let through to phone interview. I assume it is, but because that stage of the process is blind I don’t really have any way of checking (I’ve asked to see if we can find a list of who the candidates were, but I don’t think we collated that information). But it would be surprising if these tasks introduced a bias. The only plausible biasing factor to them is that they’re quite time consuming (we’re working to reduce that as a factor – currently considering swapping the order of the phone interview and coding test).
As partial evidence that this works in eliminating bias, the Aframe back end dev team is significantly older than you might expect than a start up tech team. At 29 I think I’m the youngest in the back end team by some margin, and I’m the second youngest overall. I think the dev team’s median age is over 40, and the mean certainly is. Given that age bias is a huge problem in technology, and that a lot of it is probably subconscious, I think this is at least a point in favour of the process if not wholly convincing. It might also be that those of us who were hiring just don’t care that much about age, so hard to say for sure.
So, my assumption is that something is happening that is filtering people out before they get to us. I have three possible candidates for this:
- Our job postings may be off putting in some way. Historically this was almost certainly the case, but I think we fixed that. I may be wrong. This is what we posted
- Recruiters. We did use job boards (I think stack overflow was our most successful one there), but I think everyone we actually hired came through a recruiter. Recruiters are reportedly often assholes to deal with if you’re a woman
- The initial “not an utter waste of time” review is not blind, and that’s a bit of a red flag. It’s supposed to be sufficiently black and white, with any shades of grey passed on to the rest of us, that I’d hope that it wasn’t introducing any significant biases here, but it was invisible to me so I can’t guarantee that
- A lot of it is probably just the background population. My impression is that “straight, white and male” is by far the dominant demographic in the London startup scene. I don’t have statistics to back that up at the moment, but either way I’m sure it’s not as total as we were seeing. So this is probably a large factor, but I can’t imagine it’s the only one.
So, that’s enough about us. Now we get to the real point of this post: What can we actually do to improve things?
Useful advice I’ve received so far:
- Get people from a range of demographics to review your job posting, even if they’re not interested in applying
- Be really explicit with recruiters that you want a diverse range of candidates. If they claim they don’t have a diverse range, use a different recruiter
- Target professional networks that have a more diverse selection of members rather than relying so heavily on recruiters
- Keep doing what we’re already doing – keeping as much of the screening process as possible blind. Where not possible, keep it structured. Where not possible, at least try to decide based on concrete results rather than “I liked them”
Like I said, we’re not hiring at the moment, but when we do next it would really be nice if we knew what to do to avoid this happening. Any advice you can offer would be greatly appreciated.
Feel free to leave comments, contact me on twitter as DRMacIver or by email if you want to keep it private.
The job ad is slightly nerdy, nothing more to it if I can be honest.
“Not an utter waste of time” is all about the very important thing – how well can we work with the candidate and vice versa? This means that if there was a bias introduced there, the cost of removing it would be hiring people you can’t work well with. I don’t think it’s worth it.
Recruiters are a different story. More than likely, quite a lot could possibly be improved in that area, since the amount of power they hold in contemporary IT job market is certainly corrupting ]:-> On the other hand, if you press for diversity, you might end up with the recruiter presenting you with a choice between a great programmer and a female/coloured/non-straight person. I’d very much expect it, since explicitly asking for diversity – when it’s such a common part of corporate policies worldwide – is questioning the recruiters competence.
You know, to be completely honest, I would be entirely ok with recruiters throwing every woman and minority who came through their doors at us as candidates, and then having them continue to pick and choose amongst the SWM candidates.
Why? Because the volume isn’t high enough that it would negatively affect our hiring process, and I trust our processes to screen out the bad ones a lot more than I do the recruiters.
The “Not an utter waste of time” screening is not about personality fit. It’s really about “there exists a non-zero probability that they can do the job better than some guy we’ve grabbed off the street at random”. Personality fit comes in to the phone interview and the face to face. Honestly I’d like to ditch the initial screening prior to the questions altogether. It’s sufficiently easy to filter people out with good screening questions that the potential for bias introduced earlier is more worrying than the potential for wasting our time.
Hey… that’s actually an interesting idea. Specifically exempt under-represented minorities (women, non-whites, or whatever your company seems to have none or far too few of) from the earliest of the filters. It would certainly help… I have been interviewing (on and off) at my current company for about 10 years, and I have only interviewed a female candidate for programmer twice in all that time. (Unsurprisingly, we have a large IT workforce and extremely few female programmers.) I know it can be better than that (I’ve been married to a female programmer: I know they exist).
The one problem is that this arguably falls afoul of (US) law because it applies a different criterion at some point in the process based on the race or gender of the applicant.
Yeah, it’s very hard to figure out how to do positive selection without actually doing that. Really the only way to do it is to target your recruitment rather than tune your process.
On the other hand, affirmative action exists as a thing, right? Is there not some loophole along those lines you can use for this sort of thing?
I’ve no experience with the subject, but what you describe sounds sensible to me.
I don’t know if this is a good idea, but maybe mention your double blind evaluation process on the application to indicate that competent people are encouraged to apply and not worry that this job may be “not for them”?
When you can easily find top notch staff with litlte effort, you do not need a recruiter. But when you need to spend considerable time and effort and often still find marginal staff, you absolutely need a recruiter.There are three keys I think about when working with a recruiter. The first, naturally, is working with the right recruiter. The second is making the time to really work with your recruiter. They need to know what you are looking for, and where they are hitting the mark and where they are missing the mark with candidates they send you. And the third is moving quickly when you have the right person. If you find a great candidate, that person is in demand even in a bad economy. The old saying is very true: Time kills all deals!