Pop quiz: Two senior managers at different hair gel companies apply for a top management position at Aveda. Manager A’s division increased sales by 15% in a growing company, while Manager B's group increased sales by 10% in a company with no growth. Who gets to attend the Aveda Christmas party next year (and what time does Train C arrive in Chicago)?
Manager A, according to a study published Wednesday in PLOS One. Through a series of experiments, Samuel Swift and his colleagues determined that what matters most for getting into school or getting that promotion is your final performance record, regardless of how difficult it was to succeed.
The tendency to attribute an outcome solely to ability, without considering context, is called correspondence bias, and psychologists have been studying it for 45 years, Swift said. But despite its prevalence in academic circles, it has rarely been tested using real-world data. Swift wanted to know how common it was in school admissions and in the workforce.
His curiosity stemmed from personal experience. He went to college at Carnegie Mellon University in Pittsburgh, where he worried that the school’s well-known aversion to grade inflation would hurt him when it came time to apply to grad school.
“ ‘Smart people have high GPAs’ is a simple rule that gets you the right answer most of the time,” he said. “But we’re in a more complicated world where good score may or may not mean smart person.”
He wound up getting into grad school – he studied organizational behavior and theory at CMU – and that allowed him to carry out a series of experiments based on real and hypothetical data to test for correspondence bias.
First, Swift and his colleagues asked undergraduate admissions officers from a selective American university to consider a scenario in which they had to evaluate nine fictitious applicants for an MBA program. The nine applicants were from nine colleges that were rated equally in terms of quality and selectivity, though the average GPAs for the schools were different. The GPAs of the individual applicants also varied.
The admissions officers were allowed to pick four of the nine students. Swift’s team found that the students with the highest GPAs were most likely to be admitted, which was no surprise. But they also found that applicants from schools with higher average GPAs were more likely to get in, regardless of where their GPA fit relative to the average.
In a second hypothetical scenario that was set in the workplace, Swift found the correspondence bias thriving again. Fictional airline executives who worked at airports with a good track record of on-time performance were favored for a promotion over those who worked at airports where flights were more likely to be delayed. In the test, it didn’t matter how well the executives at the more-delayed airports had done their jobs.
To really underscore the power of correspondence bias, they did a third experiment in which contestants tried to guess people’s weight in a series of photos. In one group of contestants, grading was strict; a correct guess was considered anything within 4 pounds of the actual weight. In the other group, grading was lenient, with a guess within 30 pounds considered acceptable.
Another set of people then evaluated the contestants, selecting those who they thought would do best in another round of weight guessing. Swift found that the evaluators preferred contestants from the lenient group, despite knowing that their grading criteria were much more lax.
Finally, Swift’s team examined 10 years’ worth of real admissions data from four selective MBA programs in the U.S. In addition to extensive demographic and academic data on each applicant, they got the average GPAs for many of the colleges they attended (several schools refused to divulge this).
This allowed the researchers to compare each applicant’s GPA to the average from his or her college, to get a measure of their relative skill or ability. It turned out that it was helpful to have a good GPA relative to one’s classmates, but it was just as helpful to go to a school where the average grades were higher.
In other words: Grade inflation works.
Although these studies established the pervasiveness of correspondence bias, Swift was most concerned with its presence in school admissions.
“The problem is that the schools that have higher grades on average tend to be the more expensive private schools that also tend to be better schools,” he said. “You get into a situation where the people who are fortunate enough to afford … those higher-priced private schools will have a higher GPA environment, look better coming out, and get an additional boost after college.”
The situation might improve if schools reported students’ performance relative to their classmates, using a measure similar to class rank. But in general, there’s no requirement for schools to do so.
“It’s a difficult policy question,” Swift said. “It’s the decision-makers who have the power.”
Follow on Twitter: @BradBalukjian
Return to Science Now.