Engineering Interviews are Strange

In the last few months I have reviewed quiet a few applicants and given a reasonable amount of interviews to engineers. Having also been on the receiving end of, let's say some of those and with me being an opinionated guy the thought struck me that I should write about this.

I decided to start with some failure modes I have experienced. I then try giving you an idea of what I am trying to do to address these. But first I want to address a simple question. What is the purpose of an engineering interview?

Well to me that is at the same time a very easy and a very hard question to answer. I basically want to know 3 things:

• What is this person really good at?
• Where are the limits of this persons knowledge?
• Is this person someone I want to work with for the next X years?

Failure Modes

Work for Nothing, Please?

This is a very basic one. You get a piece of work that you are supposed to complete, and it will cost you some hours of your time. It will afterwards be used to determine your skill/knowledge/structure or what ever thing the interviewer hopes to get out of it. And you do all of this for free.

Now to be fair I think that this is Ok most of the time. It is a toy problem after all and they need some way to assess what parts of your resume actually check out. I generally view this as an investment into a potential new job and it is fine to put in some upfront effort with uncertain outcome for something you want. Especially if the rest of the interview process matches your investment, i.e. if I feel that the effort is there from both parties I think this is fine. This approach communicates a genuine interest to me.

So why then did I put this into the dysfunctions category?

Fairly simple. I have been part of some interviews where it felt that what I was doing was going to be part of some workflow or product that the company was cobbling together not some toy that would be scraped afterwards. Basically I had the feeling that they wanted me to provide them with some free work from which they stood to benefit regardless of whether they accepted me into the position or not. This type of behavior is something I think is very ill advised. It is nice of you to show me your real world problems. It is not nice to expect me to work on them without compensation.

The Whiteboard Coding Challenge

I will not say much here. Coding on a whiteboard, even if it is some kind of pseudo-code is not great at best. Either you want me to explain some basic concept, where a picture would probably be better suited to the task or you want me to show that I know libraries or functions by heart in which case I am asking: Why do you think that this is a good test of my skills?

There is simply nothing in the world of modern coding anymore that comes close to this. There is always a combination of IDE support, documentation, examples, the debugger and a million other helpers at your finger tips and they support you in doing your job. Going without those things is possible yes, but do you really want your developers to do so?

We had this in the past. What is your take?

Now this is very similar to the upfront thing in that it can be quite Ok if there is a genuine interest in seeing your way to approach a problem.

However sometimes you get a question in a live interview where you feel that if you walk into the room next door you will find a bunch of engineers hammering away frantically at their keyboards trying to solve the exact same thing in the exact same moment. This is basically the "work for free" issue on steroids. The second way this can go wrong is if you are asked a question along the lines of "We had problem X in production a few weeks back. How would you solve this?" and you go on telling them your solution. At some point you are stopped and the interviewer tells you "Yes well that would have solved the issue, but we did not think of that. We did Y. Given Y how would you continue.". At this point you are basically asked to reproduce their solution, given their constrains, which you might not know, using their view on how things should be done, which you might not share.

Assuming that you are being brought on board for reasons of being a smart cookie and for broadening of horizons that might not be an ideal approach.

My Current Approach

So the way I currently do interviews is fairly boring and straight forward. The applicant has gone through some vetting process before their CV arrives at my desk. Usually together with a solution to a task from a predefined pool of tasks that I have prepared.

I look at the CV and the solution, talk to my colleagues and we make a joint decision on whether or not to proceed with the applicant.

If we proceed we have an in person chat around various topics, some other company specific stuff happens and if every box is ticked the candidate will get a positive response.

The Up-Front Task

Now this is specific to the job the applicant is applying for. They will always get a toy problem from me and it will be formulated as list of requirements. The requirements are very open, which means that they can pick and choose their approach to solving the task at hand. It is fine to shoot back some questions, and depending on the quality of those it might earn the applicant some bonus points.

I will usually ask the applicant to perform the same task in 2 different ways just to make sure that they understand the problem not just the technology and add some additional constraints in a third section. I also include a part where the applicant is asked to provide suggestions and improvements to the parts of the codebase they received as part of the task.

First of I have one knack: I look at the documentation. Yes having documentation is a requirement. I check if there is an answer to every question. Does not need to be correct, does not even need to be an answer in the classical sense a simple "Did not attempt this" is perfectly fine. Totally missing a point however is a pretty big minus for me. It raises doubt, did they miss this point, did they fail at it and are embarrassed, did they just not attempt it, wait am I missing something. If one thing is missing or undocumented that will not make the applicant fail. If more than one quarter is missing however that will be a hard pass.

Now after that providing a technically sound solution is the ground floor of what I look at. What makes a candidate stand out to me is structure and technical documentation. I do not need to like the structure used, I just want to see that there is structure. As for documentation did the applicant provide me with enough detail that I can deploy and run their solution without breaking a sweat or being overly familiar with their toolchain?

The Interview Itself

Now once they made it this far they are almost in. I will basically go over the applicants CV and the job description and ask about how they would address the challenges coming up in the offered position given their context. I will also go through some points in the CV that where not covered in the take home task and ask some theoretical questions about those. What I am looking for is an intelligent discussion about the topic that ends in one of us (preferably me) saying something along the lines of "I don't know" at which point brainstorming solutions can commence.

Here I am looking for, yes knowledge but mostly for approaches to problem solving and the ability to admit that the applicant has to consult sources and does not know every nook and cranny of the problem space.

Now if all of this sounds strongly biased towards No that is because it is. When working on small teams I think it is very important to hire with great care as you will need people that are able to pivot and do more than one thing.

Something I Might Try

Now there are things with my current approach that I do not like. Mostly that it

• focuses to much on the resume, which let's face it is written to get you past HR
• does not appropriately tie in the initial task, which makes for a less continuous experience
• feels not very efficient to me, which just bugs me
• does not provide sufficient room for the applicant to shine and show versatility

What I will try in the future is quite simple. I will provide a short scenario. And ask the applicant to outline their solution. I will then add constraints and see how they cope.

For example I might ask them to build TinyUrl. I imagine something along the lines of

"Look I am a guy with no idea of technology but a ton of money, i.e. a venture capitalist. Do you know TinyUrl? Good. Build that for me and explain to me how that works."

If the applicant has no idea what TinyUrl is I can explain that in let's say 5 sentences. After they sketched out how they would build this and have explained the technical details I will start throwing stuff at them, no not literally.

• We are 6 months in. The service is slow. Why is it slow?
• I want this to be able to withstand regional outages. How does that work?
• We will now work on a freemium model with adds injected prior to extending the URL. Where would that fit into the system?
• We want short URLs to be available only for 5 hours after generation. How do we include that?

And about a million other things that come up in the moment based on the applicants chosen solution.

That should solve some of my pain points, at least those I have a hard time with and gives me the possibility to work on the others down the road.

I'll give it a whirl and come back to you after a few tries.