Need to Make a Good Decision? Join the Crowd

When Smarty Jones -- who on Saturday will try to win horse racing’s Triple Crown by taking the Belmont Stakes -- headed into the starting gate at the Kentucky Derby, most horse-racing experts were deeply skeptical of his chances. The Daily Racing Form, for instance, asked 20 professional handicappers and pundits to predict the finish. Remarkably, only one picked Smarty Jones to win. But although the pundits were skeptical, the masses believed. The crowd of Derby bettors made Smarty Jones the favorite and saw its faith rewarded when he galloped home to victory.

The wisdom of the crowd that day was not a fluke. Instead, it was an excellent example of an intriguing fact: Under the right circumstances, groups (like the motley collection of bettors at a racetrack) are remarkably intelligent and are often smarter than even the smartest people in them. As a result, when you’re faced with problems that have a right answer -- like, which horse is most likely to win a race -- a diverse group of people, even if many of them are not especially well informed, will consistently offer better solutions than even the smartest expert.

The simplest demonstration of this is the jellybean experiment. Ask a group of 50 people how many jellybeans are in a jar, and the group’s average answer will be uncannily accurate -- within 2% of the right number -- and it will be better than the answers of nearly everyone in the group. And though the jellybean experiment is artificial, the truth is that groups demonstrate the exact same intelligence in the face of far more complicated problems.

At the racetrack, for instance, the crowd of bettors routinely offers accurate forecasts: The odds on a horse reflect, almost perfectly, its chances of winning. (Horses that go off at 3-1 odds, for instance, win just about a quarter of the time.) The Internet search engine Google relies on collective wisdom to rank the pages that have the information you’re looking for. It treats a link from one Web page to another as a vote on the worth of that page, effectively getting the Web to decide collectively which pages are most valuable. Google’s popularity is based on the routine excellence of those decisions. And the return on the stock market, even with investors’ irrationality and occasional bouts of hysteria and panic, is next to impossible for mutual-fund managers to beat.

The idea that large groups can be collectively intelligent, let alone more intelligent than the best expert, seems deeply counterintuitive. Popular images of group behavior have their roots in the 19th century, when writers inveighed against the madness of crowds and the destructive force of democracy. Our sense of caution about groups is justified -- there are myriad examples of collective craziness, ranging from lynch mobs to the stock-market bubble of the late 1990s. But in our haste to recognize that groups sometimes go wrong, it’s easy to overlook how often, and how powerfully, they go right.

To be intelligent, crowds have to be made up of people relying on diverse sources of information who make their choices relatively independently of each other. Diversity is so important, in fact, that without it, we have a hard time being intelligent: Scott Page, a political scientist at the University of Michigan, has shown that if you set two groups to work on a complex puzzle, one packed exclusively with excellent problem solvers and the other assembled randomly from people with diverse levels of ability, the diverse group consistently does better. Similarly, competent groups work best when they don’t ask their members to compromise or conform, but when they turn different individual judgments into a single collective verdict, as does Google’s technology or a stock market price.

If the crowd’s so smart, why don’t we see it? Or, in other words, why aren’t we collectively intelligent about our own intelligence? On the one hand, we are. For all of the dismissal of crowds, we do rely on collective decision-making to guide our economy (the market, not a politburo, dictates how much orange juice will be produced in the next month), and we entrust our government to voters.

On the other hand, our faith in experts and individual leaders keeps us looking for geniuses to solve our problems. Nowhere is that more true than in businesses and the U.S. government. Most corporations, for instance, are rigidly top down, putting enormous power in the hands of their chief executives. Even after any number of supposed visionaries flamed out at the end of the 1990s, the corporate faith in a leader has not waned. But we’d do well to put that faith away. Just as the Derby crowd saw something in Smarty Jones that the experts did not, so too organizations can collectively see more clearly and make smarter decisions than any one man can.

When it comes to the Belmont, bet on Smarty. And when it comes to solving problems, bet on the crowd.

James Surowiecki is a New Yorker columnist and the author of “The Wisdom of Crowds” (Doubleday), published last week.