People want easy solutions, especially to important and complex problems. So we look for the one exercise that will significantly change our bodies or the miracle diet that will take off that excess weight.  Easy solutions, like “planking” or “South Beach diet”, are fine as long as the problem isn’t too complex.

Very often, executive pre-hire assessment is approached with this understandable desire for simplicity. People are looking for the one test that will identify star employees whose talent will propel the organization to greatness.  This singular focus on finding the one test leads to mistakes that can be avoided.  Here are some of the more common mistakes:

ASSESSMENT MISTAKE #1: Looking for THE one best test – I have heard many clients talk about being a fan of one test or another, “We are a big believer in assessment, everyone has to take the (fill in name of preferred test) before being hired.”  I suppose that might work if everyone was doing the same job.  That is the equivalent of someone saying, “I am a big vitamin C fan, I use it to treat all health issues.” Psychological tests measure specific traits or abilities.  No one test measures all traits and abilities.  Each job requires a unique mix of traits and abilities.  Therefore, no one test is appropriate for every job.

ASSESSMENT MISTAKE #2: Lack of job clarity – If you have three people in a car and they haven’t discussed and agreed upon their intended destination, then you are likely to have at least two disappointed passengers when the car stops.  I once was hired by a client to assess candidates for a Head of Risk role.  The COO wanted someone who could revamp the risk function’s archaic systems and processes.  The CEO wanted someone who could effectively manage relationships with the board and external regulators.  The Chief Investment Officer wanted someone who could consult with portfolio managers and help them manage risk through scenario testing and better reporting.  Three different views requiring three different sets of skills.  This highlights the often overlooked point that assessment starts by defining the job and what knowledge, skills and abilities are critical for the role.

ASSESSMENT MISTAKE #3: Over-weighting some abilities at the expense of others – I have done several projects looking at the position of “quants” or “strats.”  These are the uber-brainy researchers who work for investment firms and look for patterns in data that can lead to returns.  Often these individuals are Ph.D. level scientists with degrees in math, physics or computer science.  The top organizations screen these candidates heavily on quality of past research and intelligence. Strats often fail because of deficiencies in their communication or organizational skills, rather than technical deficiencies, but little or no time is focused on assessing these areas.  Why?  One potential contributing factor is the “similar to me” bias.  Hiring managers look for people with the skills they value in themselves.  This is the reason why so many former division I & II lacrosse players are employed on securities trading floors.  Another potential issue is that individuals are poor judges of skills and abilities they are weaker on.  So a very poor communicator is unlikely to be able to properly gauge the communication demands of roles reporting to him/her and also would unlikely to be able to properly recognize gradations of the skill.

ASSESSMENT MISTAKE #4: More is always better: While a trait or an ability may be important for a given job, it is not always true that more of it is better than less of it.  For example, height is undeniably correlated with basketball performance.  Yet if you always picked the tallest available players, you may overlook Michael Jordan at 6”6 (which is not tall by NBA standards).   That is because any job is an interplay of several different skills and abilities.  On some dimensions, there is a threshold that once you hit, it may cease to be a differentiator.  For example, many general executive jobs require high intelligence, but once you hit a certain level, other abilities become more important to job success.  On other dimensions you have a “cologne” effect.  A certain amount is good but a lot can be really bad.  Aggressiveness and goal orientation are good examples of that effect.  So higher test scores on a single dimension are not always better.

OK let us stop here for now. I am going to write a separate piece on operational mistakes that decrease the effectiveness of the assessment process.

The solution to all four mistakes listed above is to spend time defining the job deliverables, and requisite skills and abilities, prior to choosing your assessment methods. While there is a bit of art to job analysis that comes with experience, here is the general approach:

  • Have all stakeholders agree on the top 1-3 job deliverables – Best case is for this to be a discussion where deliverables are expressed in 1-3 bullets that all agree to. Easier said than done in some cases, but there is no use in starting a selection process before you get agreement.
  • Based on the deliverables, identify critical skills and knowledge – Note which ones are needed day 1 and which can potentially be developed on the job. For those skills and abilities you are selecting for, gauge the desired proficiency level needed. Using the prior example of the Head of Risk, the candidate needed to have expert level knowledge of financial products and markets, but only needed strong enough communication skills to be credible for the board.

Job analysis has a long history and there are many tools out there that can help.  At the heart of it, job analysis involves taking enough time to agree upon and understand what the individual will need in order to be successful.  The time devoted to this step is the foundation for the assessment process.

Feel free to reach out to me with your thoughts and ideas.

Avi Shatzkes Ph.D.
adstalentconsulting.com
[email protected]