The Coding Exercise

When I first got involved in interviewing people way back in 2006-07 I started out like most others, asking textbook questions and hoping for text book answers. e.g.

• Can you describe the ASP.Net page/control lifecycle for me?
• What objects and methods would you use to retrieve data from a SQL Server DB?

I am also guilty of having asked some very specific questions related to some corner of the API that I felt a developer should have memorised. Don't get me started on the modern day "have you memorised the security API and remember which parameter goes where" that some recruitment processes force you to go through today :-(.

It became apparent quite early that this wasn't doing a great job of filtering the poorly skilled out and keeping the excellent people in.  

Someone smarter than me suggested that we include a coding exercise within the interview, we'd limit it to 30 minutes, and we'd sit with them so that they could talk through what they were doing as they were doing it. This was an awesome way to filter applicants, and it significantly increased the correlation between how well we thought someone would do day to day, and how well they actually performed on the job.

But there were still problems. Some people were so stressed with having people sit with them, and having a short deadline, that they had the shakes/sweats/etc. Some of these people actually seemed like they knew their stuff, but we were placing too much pressure on them. In addition, we felt that 30 minutes wasn't enough time for the interviewee to properly demonstrate their skills and professionalism.

Our next evolution was to let them take their time at home, we opened up the time a little by providing a guide of 1-4 hours depending on the particular exercise we sent them. This has been a life changer. It gives the candidate the time and lower stress environment in which to demonstrate their skills and style, and it has enabled us to filter people much more efficiently.

We have a few different exercises that we send out, and we try to find one that matches the persons skills and experience somewhat. 

All that said, it isn't perfect yet, and every time we go recruiting we tweak it a little and try something new. Hopefully we are making the experience better for everyone as we do so.