Test-Taking Comes to the Office
Tomas Chamorro-Premuzic, author of the HBR article “Ace the Assessment,” explores the rising practice of using tests in hiring and promotion decisions.
SARAH GREEN: Welcome to the HBR IdeaCast from Harvard Business Review. I’m Sarah Green. I’m talking today with Tomas Chamorro-Premuzic, CEO of Hogan Assessments, Professor of Business Psychology at University College London and on the faculty at Columbia University. He’s a frequent contributor to HBR.org, and he’s the author of the new magazine article “Ace the Assessment.” Tomas, thank you so much for talking with us today.
TOMAS CHAMORRO-PREMUZIC: My pleasure.
SARAH GREEN: So the article is all about how to score well on the assessments and tests that companies are increasingly using in hiring and promotion practices. But I’m just wondering as we begin here, how common are these tests, and who’s taking them? Are these entry level jobs, senior jobs?
TOMAS CHAMORRO-PREMUZIC: Those are great questions. And I would say that on the one hand, they’re quite common in large organizations. So we estimate that 60 plus percent of organizations with more than 100 employees use them, and predominantly for making decisions about incumbents, so current employees. Typically, it will be things such as should I promote this person from an entry level job or an individual contributor position to a managerial position. Those are the most common uses. The second most frequent would be for recruitment or selection.
But when we look at the overall market and we also consider smaller companies– and most companies in the world have fewer than 100 employees– they’re really not that common. They have been increasingly been used in the past five years, but my estimate would be that fewer than 40 million of these assessments or tests are used for recruitment on a yearly basis, recruitment or promotion. If we consider that there’s about 4 billion workers in the world, that means that only about 1% of the potential workforce of the world is assessed with these instruments.
SARAH GREEN: Well, so it is interesting to me, though, that you note that there has been an uptake in the last five years. And there is something that is futuristic sounding to me about these tests. So I was really surprised in the article to note that these actually– you write in the article, go back to the Han Dynasty in the third century? That’s a long time ago.
TOMAS CHAMORRO-PREMUZIC: Yes. Absolutely. So they were the first organization, if you like, to design a purpose test for assessing candidates that weer applying for a job. And those tests look very much like general knowledge tests. Nowadays, employers are more interested in evaluating your styles, your competencies, your values. And the main difference between the early days of selection at work and the current time is that because we’re not that able to predict what sort of tasks employees would be doing at work, it’s more accurate, perhaps, to come up with a generic profile of the individual you want to hire.
So instead of saying, I want somebody who will work as an accountant and these will be the tasks that they complete on a daily basis, we can say, I want somebody who is entrepreneurial and creative. Because I don’t really know what problems they’re going to have to solve in the next four or five years, but I know that they will have to think independently, exploit the resources available, and produce something [INAUDIBLE].
SARAH GREEN: And as you’re saying that, I’m wondering, are these all computer tests, or paper and pencil tests, or are there are other ways of assessing people’s behavior that maybe aren’t computer tests as well?
TOMAS CHAMORRO-PREMUZIC: There are multiple ways. And new ways are being pioneered and developed as we speak. I would say that between the 1930s and the 1980s, most of these tests or assessments were paper and pencil tests. Since the early ’90s till– I don’t know, five or six years ago, perhaps, most of the tests or the assessments that candidates completed were online adaptations of these earlier paper and pencil assessments.
And nowadays, of course, because people are spending so much time online, we’re also seeing a variety of new methodologies from algorithms that scrape the web to gamified versions of the assessments, so things that look more like computer games or interactive engagement games but are still profiling you. And of course, internal uses replications of big data where it’s part of a talent analytics or people analytics movement where companies are trying to scrape and track all the behaviors of their employees while at work and translate that into some sort of profile or a prediction of that employee’s future performance.
SARAH GREEN: So Tomas, I’m just wondering as we’re talking about some of these future looking ways of assessing people, are these new methods really better than the traditional methods? Is it really better than getting a reference, or looking at someone’s previous work, or interviewing them?
TOMAS CHAMORRO-PREMUZIC: That’s a great question. At this stage, the new assessment methodologies are still about at an experimental stage. And what I mean by that is that we still don’t have much independent scientific evidence published in peer-reviewed journal articles that test the incremental validity– so how much better, or whether, in fact, they add anything over traditional personality assessments or ability tests. Now, it’s possible that these new methodologies provide something new, something that we can’t infer or evaluate with traditional methods.
At the same time, we need to understand that most of these new methods are just new manifestations or more high tech versions of old methodologies. For example, using endorsements from LinkedIn or other social media sites is just a modern version of a reference or letter of recommendation. Looking at your profile on LinkedIn and what skills you report is just a new version of the CV. And even looking at scraping your Facebook or Twitter feed is really just a new version of evaluating your self-presentational style– something that in the past or traditionally, we have done via interviews.
So I think these new methodologies are just new adaptations of old assessment methods via technology. And the big advantage, of course, is that they enable people, employers to scale the selection process and evaluate millions of candidates for a lower cost.
SARAH GREEN: Well, and it’s funny, because I was thinking at first, well, maybe this is unfair to people who are just bad at taking tests. And then, of course, I thought about all the people who are just bad at doing interviews. So in some ways, it seems like maybe some methods are unfair to everybody and others are more fair to other people. I don’t know.
TOMAS CHAMORRO-PREMUZIC: Those questions are really interesting. Traditionally, consumers– or in this case, potential employees or candidates– have seen these tests as potentially unfair or something that lies between them and their preferred career or a potential job. In reality, we need to understand or help people understand that there are two benefits of using these assessments for employment or HR decisions.
One is, of course, that when the client is the organization, that we are helping potential employers to find the right employee, the right leader, the right manager, the right person for a job. But then there’s also the part for the employee or the candidate. Just like you don’t want to end up in a relationship that is not good for you, you really shouldn’t end up in a job that is not well suited for you. You might be happy when you get the offer, but then if it doesn’t work out, it can have all sorts of negative implications.
And finally, the test cannot just be used to match the right person to the right job. But in some cases, there’s a very, very important piece of information that candidates can get from their feedback– not just their scores, but their narrative feedback, where they can learn about their strengths, their opportunities for development. And really, these assessment tools are a great way to improve your self-awareness and understand who you are and how you compare to other people.
SARAH GREEN: So that’s interesting that you mention self-awareness. Because I do have some questions about if you are a test taker who either is not very self-aware, and so you’re answering these questions, but they don’t actually– they line up with your idea of yourself, but not the reality of yourself, or if you’re a dishonest test taker and you’re intentionally giving the wrong answers, does the way that these tests rely on self-reported data mean that sometimes they may not actually give you a very accurate portrayal of who the people are who are taking them?
TOMAS CHAMORRO-PREMUZIC: Yeah. And it’s difficult for people to understand how these tests work, primarily because different tests might look superficially the same, but actually, they can have different mechanisms or algorithms that are used for the scoring. Tests that look like a self-report test– for example, I’m good at everything I do– that will be a question– don’t necessarily rely on people’s introspection or self-knowledge. The good tests give you different statements or items that invite you to present yourself in whichever way you want.
But really, they don’t take your answers at face value. What they do is they compare your answers with the answers of other respondents. And typically, they can rely on hundreds of thousands or millions of participants in a database. And the crucial scoring engine works by associating or linking your choices, how you present yourself to objective, work-related outcomes. In other words, if people who tend to say yes or endorse questions such as I’m good at everything I do end up performing poorly or seen by their colleagues and subordinates as arrogant and not very good at everything they do, then that item or that question might be used to evaluate your narcissism rather than your competence.
So sometimes, it’s tempting to imagine that these tests are easy to trick or to fake, because we can choose how we answer or how we respond to each question. But in reality, because we are linking your answers to actual outcomes, these tests are very difficult to fake.
SARAH GREEN: OK. So I do have one more devil’s advocate question about that kind of thing, though, which is that in the article, one of the tips you give people to improve on their test taking skills is to take more tests and practice. And if the test is really designed to assess your competence, your work ethic, and you get 20% better because you’ve taken the test a bunch of times, you’re not working 20% harder. You’re not 20% more competent. You’ve just gotten better at taking the test.
TOMAS CHAMORRO-PREMUZIC: Yeah. So the main improvements that have been recorded or observed in the research are not so much on personality tests, tests that evaluate things like your work ethic or your integrity or your likability or your extroversion, but tests that evaluate your ability. And there’s two reasons people tend to improve if they practice on ability tests.
The first one is that they get used to the type of questions that appear. So even if they’re not taking an identical form or an identical test to the one they’re administered in high stakes settings when they apply for a job, they get a feel or a sense for the sort of questions that appear, and it really saves them time. And since most of these tests are timed, when they complete the real test, they can take shortcuts and focus on the problems as opposed to the instructions.
The second reason is, of course, that some of those tests are evaluating things related to the domain of general knowledge or expertise in different areas. So much like studying for an exam at school or university will improve your performance, studying for a test that will evaluate your knowledge of certain subject matters will improve your performance.
What’s interesting is historically, people have tended to see this as a negative thing– that if you practice, you get better, and then you’re not really getting smarter. But in reality, that improvement is driven by conscientiousness, ambition, and determination, which are all job relevant skills. So I don’t care if people practice, and because of their practice, they improve their scores. Because I want to hire people who practice and who take the assessment seriously.
SARAH GREEN: OK. So I guess it sounds like these assessments are maybe not as easy to game as maybe some of us assumed, and also maybe not actually as scary or intrusive as some of us may have feared either. What is your ultimate hope for if more organizations adopted using assessments– what is your hope that would actually happen as a result?
TOMAS CHAMORRO-PREMUZIC: So I think for organizations, the key reason to adopt these assessments is that they predict relevant organizational outcomes. There is a clear ROI, return on investment whereby if you hire people who are better suited or a better much for a role, they will perform more highly. And that should translate into a more engaged workforce, into more productivity, higher revenues, profits, and so forth. So they shouldn’t do it necessarily because they care about the individual, but that’s the key way to increase their competitiveness and win the war for talent by hiring people who will be better at their job.
And then there’s the other side of the equation. What will be the benefits of these assessment methods being used more widely? What will be the benefit for the candidate? And the reality is– I have a lot of friends and colleagues who often ask me, I have to complete this assessment or go to a job interview or complete this test for this job. What advice do you give me? And my advice is just take them seriously, answer honestly. Be yourself, but on a good day– much like when you go on a first date or something.
And then understand that if the assessments are scientifically validated and the employer knows what he or she is looking for, if you don’t get the job and you don’t pass the test, that’s also good for you. You were not made for that position. And I know that can be hard to understand initially. But at the end of the day, much like in the realm of love and relationships, there is a job for every person. There are more suitable positions for some people than for others. And you have to keep searching.
So all these assessment methods and tests do is help us increase the match or fit between an employee and a job. And that’s a good thing given that so many people are unhappy with the jobs they have.
SARAH GREEN: Well, and I also find myself wondering– in a world where so much hiring and promotion is really done with networking, and who you know, and not always what you know, are these a way to actually make that less biased and maybe give a leg up to some people who maybe aren’t super great networkers or something like that?
TOMAS CHAMORRO-PREMUZIC: Absolutely. That’s what people often forget. What’s the alternative? Is the alternative to rely on our intuition? Well, we know that our intuition plays us tricks, and that people make biased decisions, and that things such as attractiveness or sense of humor or being similar to the person who makes the choice influence the processes, and then those things don’t predict high performance. At the same time, as you mentioned, if we only rely on networking and who you know, we will go back to the 19th century or aristocratic times when nepotism is the only method that is used to select individuals.
And if you rely on that, you will suffer, because organizations that are more competitive will rely on solid or well-established assessment methods. So I think everybody wins. Organizations and even economists will become more competitive and more effective if they rely on proper assessment methods for selecting their employees.
SARAH GREEN: Well, Tomas, thank you again. It is such an interesting area that is still developing in companies today.
TOMAS CHAMORRO-PREMUZIC: OK. My pleasure. Thanks.
SARAH GREEN: That was Tomas Chamorro-Premuzic. For this article, “Ace the Assessment,” and more from Tomas, go to HBR.org.