Technical testing has a role to play in any screening process for technology staff. Hiring someone under-qualified will be as damaging to that person’s career and confidence as it will be for the business appointing, so for most hiring managers, technical tests are an instinctive, non-negotiable element of any screening process.
Would it surprise you to know, though, that hiring the right person is rarely the key driver in the adoption of technical tests? That honour belongs to an overwhelming desire to reduce the number of people a hiring manager must meet in order to appoint and apparently knowing of no other way to achieve this reduction in numbers.
The least considered factors in selecting and administering technical tests, though – and actually the two that are most critical for SME technology firms – are the impact such tests have on both the attraction of applicants and the reputation of the business using them.
Take, for example, the most common technical testing method used in the London employment market. This takes place either on receipt of a CV or following an initial telephone call, and involves sending every applicant a task-based assignment requiring an average of four hours of their time to complete. What is the impact of such a test? The first, is a simulated picture of how those completing it approach a technical task and some evidence of their coding ability. Second, it acts as a measure of their interest – if they have taken the time to complete it they must be motivated to work for the company in question. Third, as a benchmark and point of discussion in an interview it will be doubtlessly useful.
But there are other impacts. Undoubtedly, such a test will eliminate applicants from the process but the majority of them won’t be those who fail it, they will be those who have no interest in completing it. To understand this, it’s critical to picture the job search of the average highly skilled technology professional. “Job search”, in fact, being a somewhat outmoded expression, with Linkedin exposing most engineers to something like ten opportunities daily whether they are looking for work or not. Those who become active job seekers will have their pick of some 30+ roles instantly, and that’s without undertaking any passive research at all.
First, they reply to the recruiters at the top of their Linkedin messages. Now, after receipt of job specifications and probably a brief discussion with the first 20 who contact them, they authorise the submission of a CV and await interview requests. It’s at this stage their first real decision making kicks in. Which interviews should I attend?
If five of fifteen interview requests require completion of a four hour technical test prior to even meeting, will our busy applicants meticulously research each business and role then make an informed decision as to whether they wish to invest an entire day in arduous testing? Or will they simply pursue equivalent roles for businesses interested enough just to invite them for interview? The answer is both self-evident and the current reality in London.
Those who do complete tests will be those who don’t have the wealth of options offered to the best engineers, which means technical testing prior to interview, or even post-telephone interview, has the unwelcome impact of driving away almost anyone who has a high number of opportunities open to them. It will surprise no one to know that those engineers with multiple opportunities are also the best engineers.
This small example illustrates why technical testing cannot be considered in isolation. It must form one element of a complete selection process that screens out under-qualified applicants, protects hiring manager time, builds a positive picture in the minds of high-value applicants and makes each applicant not selected feel good about the entire interview process.
This, of course, probably seems like unhelpful advice to hiring managers receiving 30-40 CVs of applicants per-role where, at such an early stage, interest and technical competence are essentially impossible to gauge. How, then, can one reduce this pile to a manageable number while ensuring that those who do make it through to the next stage are those worth investing time in?
The first step is to ask oneself who it is we are actually aiming to attract. Most job specifications are so vague and hard to interpret that the breadth of applicants being sourced by recruitment agencies is such that some sort of mass-screening exercise on the potential employer’s part has to occur. Is this really needed, though? How about we simply reposition the criteria that will lead to a hire using essential softer skills to drive down applicant numbers and increase relevance at an early stage?
Few engineering roles are just about creating code. How an applicant thinks about engineering is perhaps as – or in some cases more – important. What is their role in user story creation? Can they describe the last one they worked on and frame its importance to the business? Can they describe the construction – in simple terms – of the software they are currently building? How do they deploy software and what are their views on this? How about QA and where it should sit in a Scrum team? And come to that, just how Agile was their last team and what would they have changed about it if they could?
This type of business-based interviewing against clear competencies can be passed to recruiters either internal or external, thus freeing the time of hiring managers to focus on those applicants who get through it. What’s important to note is that while hiring managers won’t know if these applicants are the most technically competent of the 100 or so being screened, what they will know is that culture fit exists.
Going back to our previous question “who is it we are aiming to attract”, blanket answers of “the best engineers” are completely unhelpful. Unless the role is 100% backroom coding and requires an absolute technical wizard who lives and breathes nothing but code, we are certainly not aiming to attract the top 1% of coders. This means that rather than show us who is “the best” (because in all forms of testing this is subjective anyway), a test must be used to show us who is competent.
For this reason, long, complex multiple choice or scenario-based testing becomes overkill when we have 3-5 applicants we know will want to perform the role and fit with the team. We just need to know if they are technically competent and we ideally want to use this opportunity to increase their interest. A paired programming exercise at interview is often the most natural way for an engineer to show their skills and for an existing team member to build a relationship with them. This exercise need take no more than an hour and can be focused around the actual user story your team member is currently engaged in building.
Alternatively, if you don’t want to tie up an engineer for 60 minutes, paired programming can be replaced with a short code review. Provide interviewees with some recently written code from the team and ask them to review it, making comments or even refactoring if they wish to. This need take no more than 30 minutes and, as well as gauging their technical competence, also provides an excellent talking point for the rest of the interview. In fact, a 30 minute kick off around motivations and current situation to show real interest, the code review followed up with a technical Q&A session around it then a final “meet the team” with a chance to ask questions, is an ideal structure for one-stage interviews.
Paired or review exercises are only practical, though, where the 3-5 shortlisted applicants are definitely people you want to hire if their tests reveal technical competence. Trying to take a time-heavy approach across 20 interviews is generous to applicants but also highly impractical for the team members drawn into it. To pull the approach off you must have a clear target applicant in mind that you know you will hire and a rock solid job specification that speaks specifically to that target applicant. Without this – with 20 vague bullet points being pushed out to the market by recruiters who don’t understand them – you are better off with mass, early stage technical screening where applicants – not your team – invest their time.
This, of course, brings us back to the danger of appearing to be an impersonal employer more inclined to take from employees than give to them. By investing time in a personal technical screening process where pre-qualified applicants receive as much as they give, your firm will stand out against competing interviewers and secure higher quality talent as a result.
This willingness to empathise with future employees extends to understanding the pressures faced by applicants engaged in an active job search. Taking time off repeatedly is extremely challenging where an employer may not know that a staff member is looking to leave, and engaging in off-site technical tests is just as hard. Even holiday can be suspect, and trying to get this at short notice is often hugely disrespectful to an existing employer, especially where there is an emotional connection with long standing colleagues. The lack of appreciation for this among hiring managers who themselves have faced identical issues in their own job searches is quite staggering. Not responding to applications, cancelling interviews last minute or simply not attending them, not reviewing tests fully, or demanding that applicants complete and submit tests at specific times to suit their future employer all render the idea of implementing impersonal technical tests to save time at the start of a hiring process laughable. Technical tests are frankly irrelevant when hiring is configured around the idea that applicants are “just another number” and should be happy to have an interview at all. The best engineering teams are built on empathy and mutual respect, something an interview process is often an applicant’s only gauge for in deciding where to work. An arm’s length, impersonal approach will alienate the best engineers far more than any sort of test will identify them.
So, to summarise, for optimum technical interviewing that both identifies and attracts talent:
Clearly identify a target hire you know exists and you will appoint if proven competent.
Distil such competence into clear “soft” screening criteria and pass to pre-qualified recruiters.
Invest your time in personal exercises with this small, pre-screened group at interview, either paired programming or code review.
Minimise stages wherever possible. If you can do everything in one session, do.
Show respect to every single applicant. Those you treat like valued employees will become them.