There are three types of technical interviews: coding, system design, behavior. The coding interview is the most essential to software engineers.
For entry-level software engineers, they may face 3–5 rounds of interviews. Each round of interviews ranges from 45 minutes to 1 hour, and they are most likely all coding interviews. Sure we also mix a few behavior questions in between. But they won’t hold as much weight since the candidates wouldn’t have a whole lot of interesting stories to share anyway.
For more senior roles, i.e., tech leads, and managers, we put more weight on system design and behavior questions. Nevertheless, coding will still be in the picture. Since we have to make sure that the candidates have at least the computer science fundamentals and know what software engineer is all about.
Today, let’s focus on just coding interviews — how it comes to be and why it becomes so hard nowadays.
The Rocky Road To Become a Coding Interviewer
Compared to the other two types of interviews, coding interviews are certainly easier to learn.
If you want to become an interviewer for coding interviews, the training process is more or less the following:
- Come up with a new problem
- Test the new problem on your fellow colleagues
- Test the new problem on candidates
- Graduate as a certified interviewer
Coming up with a new interview problem is the trickiest and most time-consuming.
-
The problem’s complexity needs to be just right. If it is too hard, then no one can pass. If it is too easy, then everybody can pass. In either case, it is hard to tell the good candidates apart from the bad.
-
The problem needs to be creative. Ideally, it should be a problem that cannot be found on the internet directly. Otherwise, we’ll be hiring people that have seen the problems beforehand instead of those that are smart.
-
The problem should be interesting. It is a huge commitment for candidates to participate in an interview, especially for those who already have a job and need to use their vacation time. We want to be respectful and grateful. And we want the candidates to have fun and have a good interview experience.
-
The problem is relevant to the nature of the work. Ideally, the problem should originate from the day-to-day work and can test the skills that a candidate needs to do well in the job. That’s why most companies have banded brain teasers, dynamic programming problems, or NP-hard problems.
I personally spent months and went through multiple iterations before I settled down on a set of problems that I was happy with.
More often than not, we don’t ask new interviewers to come up with new coding problems. We sometimes try to reuse existing problems that are developed by other interviewers in the company, and sometimes use problems that we can find on the internet (either with or without modifications).
Training to become an interview is like chopping down trees in a forest. It is not a trivial process. Finding a good interview problem is like finding an ax. Without that, one may never become a coding interviewer. With that, the remaining steps will still require work and practice, but it is more predictable.
A Known Set of Good Problems
Unsurprisingly, it is very costly to train an interviewer. And because of that, more often than not, we skip the part where we require the new interviewers to come up with original coding problems.
It is complicated to reuse existing problems developed by other employees in the company.
- There can be conflicts from different interviewers. The problem set will be small since it is hard to come up with new ones. If a problem can now be used by multiple interviewers, it will be challenging for the interview coordinators — for the same candidates, we have to make sure that the same problem won’t get asked multiple times by different interviewers.
- It is hard to prevent leaks. As the company grows, and more and more interviewers are given, coding problems will eventually get leaked. We can require all candidates to sign the non-disclosure agreement (NDA). But inevitably, problems would still get leaked. Imagine a big tech company with 100k engineers. Disregarding turnover, assuming the hiring rate is 10%, then it takes 1 million interviews to hire 100k engineers. Even if there were just 0.1% of candidates disrespecting the NDA, there would still be 1,000 leaks.
To solve the problem of not having enough coding problems, interviewers eventually turn to the internet for inspiration.
Given the constraint of 45 minutes, there are only so many ways to ask coding questions. Even for the problems that are claimed to be newly invented, we can often find similar problems on the internet.
Coding Problems Become Harder and Harder
If candidates are sheep, then interviewers are wolves. The sheep learn to run faster and faster because they want to survive, and so are the wolves.
Years ago, there weren’t any interview practice materials. New grads would review their Data Structure and Algorithm textbooks to prepare for coding interviews. And we would turn to senior students who have been through some interview process to pick up some wisdom.
In 2008, “Cracking the Coding Interview: 150 Programming Interview Questions and Solutions” was published. It quickly became the bible for candidates, and likely interviewers too. Soon after, interviewers learned to avoid problems from this book since they were all so well-known.
Around 2016, LeetCode became a popular coding interview preparation platform. The problem set is constantly growing too, from a bit over a hundred, to a few hundred, now over two thousand!
Problems are tagged on these practicing platforms too. Not only would we know which companies ask which problems, but also know when they are asked and how often. Candidates that know to use this information end up having an unfair advantage over those that do not.
At this point, it is hard to imagine anyone able to land a job offer at a good tech company without practicing for interviewers rigorously.
Some companies have attempted to fight this too. There is an internal document at Google called “The LeetCode Problem”, discussing how to prevent people from passing the Google interview through practicing on LeetCode.
Not only are the problems are getting harder and harder, but the requirement for completeness and correctness also becomes higher.
In the past, it might be possible to land a job offer if a candidate can explain the approach well, and leave behind some reasonable pseudocode on the whiteboard. Now since everyone else is able to produce perfectly clean code in one go, those candidates that are making little mistakes here and there may look worse in comparison.
And a candidate might have thought that they did fairly very well coming up with the optimal solution during the interview; but a few days later, only to be upset and confused getting a rejection email from the recruiter. Little the candidate knows, that interviewer still had 2–3 follow-up problems.
It is much like a never-ending war; the bar for coding interviews just gets higher and higher.
Is It Fair?
One day, I was interviewing a seasoned engineer from a solid startup. He did poorly, and he knew that.
At the end of the interview, we always give candidates a few minutes to ask questions. So that it is an opportunity for them to learn a little bit more about the company, the culture, the product, or the team.
For this candidate, he clearly gave up. He complained:
“Do you think it is fair? I have decades of industry experience, but I now have to compete with new grads on data structure and algorithm problems?”
He looked sad. And I felt sad.
And there is also this tweet from Max Howell after failing a Google interview:
Google: 90% of our engineers use the software you wrote (Homebrew), but you can’t invert a binary tree on a whiteboard, so fuck off.
So, is it fair?
There is no debate that it is not perfect. We sometimes may have passed on great engineers if they didn’t put in the hours to prepare for the interviews. But on the other hand, we also are also fairly confident that candidates hired this way, at least, know some computer science fundamentals.
Are there any other alternatives?
How about we have sample project(s) that candidates can work on in a full development environment? They can use their favorite IDE, with compilers setup if they want. This can simulate a real work environment as close as possible.
However, it is even more costly to create such project(s). And it would still be hard to be fair for candidates using different programming languages. If we do end up creating multiple projects, it is still very hard to make sure they share the same level of complexity.
The way that coding interviewers are being conducted nowadays, might not be perfect. But we don’t yet have other better solutions.
What’s Next
Where would we end up a few years from now?
Would we be able to come up with a better approach to evaluate candidates? Would it be fairer, less time-consuming, and less intense for candidates?
Or would it get even crazier? Would it eventually require the skill for competitive programming to land a job in the industry?
It is hard to predict, but it would be interesting to see how the industry evolves.
I always want to consider the interviewing and job-hunting process a game. We are all in the game. The rules of the game might involve overtime. But if we want to play, we have to follow the rules.
IMHO, these are at least what we can do:
If you are an interviewer, try to avoid problems that are easily available on the internet or at least tweak them before using them. Try to avoid problems that clearly require practicing, i.e., dynamic programming. Try to focus less on whether a problem is solved perfectly but instead pay more attention to how candidates think and approach the problem.
If you are a candidate, prepare for the interviews as hard as you can! Frankly speaking, that may not be the best way to use your time. But you need to do what you need to do. And after the interview, don’t share the problems.
The world is big and pretty diverse. The discussions above are based on my very limited experience. And they might be wrong in a different context.
And what’s your experience? What do you enjoy and what do you find frustrating? What do think can be improved? And what do you think the whole interview process should be? I would very much love to hear from you too! Thanks for reading.
This article is also available at https://betterprogramming.pub/5a8231326299.
