II. We Will Not Test Devs with Computer Science Riddles

If you’re thinking of using Cracking the Coding Interview to come up with interview questions when hiring candidates, don’t. Obviously it works for Google, Amazon, and Facebook—or at the very least it doesn’t prevent them from maintaining gigantic profits. So I suppose, if you have no shortage of incredibly talented candidates that you can incentivize with valuable RSUs, go nuts. On the other hand, if you’re a startup or midsize company trying to build features for real customers, it’s a dumb approach.

In 2015, Max Howell tweeted “Google: 90% of our engineers use the software you wrote (Homebrew), but you can’t invert a binary tree on a whiteboard so fuck off.”

Would Google have been better off hiring the author of Homebrew? Maybe, maybe not. But what I do know is that if you have a process that rejects someone who has a record of building wildly valuable and successful software because they can’t correctly answer computer-science riddles, you’d better be in the business of answering computer-science riddles and not building valuable and successful software.

Think hard about what your company needs to do to be successful. Chances are it will boil down to making customers’ lives better and easier to the point that they’re thrilled to give you money in exchange. It’s tempting to think that people who have mastered algorithms and data structures will be able to build software that makes customers happy—and to build it faster and better than people who lack such knowledge. It might also be tempting to think that if these hiring processes work for Google and Facebook, then surely they’ll work well for you too. In reality, this is an odd bank shot to take.

Here’s the deal: The farther you get from testing what you actually care about, the worse off you are.

Interviewing candidates is an instance of a larger class of problems. Classification, screening, and detection all take the same shape; we interact with these sorts of tasks all the time in daily life.

For a dramatic example, let’s look at airport security. It’s difficult to efficiently predict if any one person passing through a security checkpoint will go on to do something horrible. In the wake of 9/11, some people argued that it would be more efficient to use racial profiling to give more or less scrutiny to particular ethnic groups. The thinking went something like this: Members of group A (Muslims) are more likely to have behavior B (terrorism), and therefore we should focus attention on group A to have a better chance of detecting behavior B.

Even leaving aside cultural values and the harmfulness of stereotypes, this doesn’t even make basic security sense. Because this strategy introduces additional complexity in detecting members of group A (since someone’s Muslim beliefs are invisible), it increases the cost and reduces the security of the entire system. Not only that, but by establishing a clear profile that will get less scrutiny, it makes it easier for bad actors to evade detection.

Proponents of the racial-profiling system support it because it’s “common sense.” Someone who looks like Betty White obviously isn’t going to blow up a plane, so it’s a complete waste of time to spend time screening her if she makes the X-ray machine beep. Unfortunately, trusting our gut and following “common sense” are bad ideas when it comes to security and detection. They lead us astray.

It might be “common sense” that someone who can invert a binary tree would be more likely to be a member of group A (developers who know CS fundamentals), and group A is very likely to have behavior B ( the ability to create value for customers). However, building an interview process around this is asinine.

If you want to predict accurately whether or not a candidate will be able to create value for customers or your company, then that’s what you need to test for. This requires creativity. In that sense, it may not be as easy as having someone whiteboard bubble sort or answer a hard logic puzzle—because you can’t plagiarize other companies’ superficially clever hiring tests. But unless your company bubble sorts and solves logic puzzles for customers, a precise screening mechanism will give you a much better signal than an off-the-shelf set of tests.

Hopefully, we can agree that if you need work done on your house, it’s not a good idea to hire a contractor the Google way. Yes, even though your house is unique and will present special challenges, you don’t need to ask hypothetical questions to gauge how well they can think on their feet and adapt. You also don’t need to quiz them on fundamentals. You just need to see examples of previous work that is similar to what you need done.

Software is different, but it’s not that different. You might think that you want someone who knows CS fundamentals and who can adapt to any situation, but you don’t. You want someone who can knock real features off your real roadmap. Test for that. And if your roadmap is too much in flux for that, you have bigger problems to solve before you start hiring devs.