Telling You What You Like

Geoff Lester’s two iPods can hold 20,000 songs, so he turned to the iTunes Music Store, which stocks 1.5 million online selections. But how to choose?

After the 28-year-old Los Angeles resident picked a few tracks, the software algorithms powering iTunes took over, popping out song after song they calculated that he might like.

He did. He estimates that he bought 500 tracks he otherwise wouldn’t have over the last two years. He snapped up songs from the Dining Rooms, Zero 7 and Headset -- bands that aren’t exactly mainstream.

“Before, you’d only find out about new music from whatever’s playing on the radio,” said Lester, an account manager for Nascar. “Now, you can find out about all kinds of bands. It’s definitely opened up some genres for me that I would never have found any other way.”

Time was, a trendy friend would offer advice on what to watch or listen to, where to stay or eat, what to wear or drive. But in the Digital Age -- when customers are a mouse click away from virtually everything -- not even the most plugged-in human can keep up with all the choices.

But computers can, and the growing ranks of so-called preference engines try to capture the intricacies of taste to help buyers navigate the plenty.

Like Coldplay? Check out Moby. Fan of Ian McEwan? Try Philip Roth. Those who bought “Harry Potter” also liked “The Chronicles of Narnia.”

“We’re just being flooded with content,” said Erik Brynjolfsson, professor of management at the Massachusetts Institute of Technology Sloan School of Management. “And people are increasingly relying on recommenders to help them sort through it all.”

Preference engines emerged in the earliest days of e-commerce to boost sales -- the Internet equivalent of “Would you like a belt to go with that?” -- but they have improved with technology and incorporated human feedback to more precisely predict what someone might like.

Their spread worries some who fear that preference engines can extract a social price. As consumers are exposed only to the types of things they’re interested in, there’s a danger that their tastes can narrow and that society may balkanize into groups with obscure interests.

“As these things get better and better, nobody has to encounter ideas they don’t already agree with,” said Barry Schwartz, professor of sociology at Swarthmore College and author of “The Paradox of Choice: Why More Is Less.” “We lose that sense of community we had when there were shared cultural experiences, even though we may not have liked them. Now we can create our own cocoon and keep all that unpleasant stuff out.”

The most common recommendation tools involve collaborative filtering, a technique that suggests products based on what other people with similar tastes have bought. These tools keep tabs on what people purchase, what items they browse or whether they put items into their shopping carts. Some take a further step by asking people how well they liked their purchases.

The idea is to divine clusters of taste, based on the actions of thousands of people, so that when a new person arrives, the website can start matching their taste against others in the database and begin making recommendations.

These systems boost sales. They also solve a problem created by the Internet -- the problem of too many choices. Consumers confronted with dozens or hundreds of choices generally have little difficulty making their selections. But in an online marketplace with thousands or millions of options, many freeze, unable to decide.

“It paralyzes people into indecision,” Schwartz said.

Having such a wide selection also makes navigation unwieldy, particularly when the choices are obscure. The original MP3.com site, which offered songs from unsigned artists and bands, exemplifies the challenge.

“It had just so much music on it, it became completely unnavigable,” said Tim Westergren, chief technology officer of Pandora, a company working on a recommendation system for digital music. “After a while, it became, ‘You’re on MP3.com. So what? Who’s going to find you?’ ”

Preference engines are useful because they bubble to the surface a selection of items that otherwise would be overlooked. At Rhapsody, a digital music service offered by RealNetworks Inc., 90% of its database of more than 1 million songs are played at least once every month, thanks in part to the service’s preference system.

Researchers then wanted to see whether something as complex as taste -- for music, movies, books -- could be boiled down to a mathematical formula.

“Human taste is complex, and I’m not sure it can be accounted for,” said Ken Goldberg, a UC Berkeley computer science professor who studied whether a computer program could recommend jokes. “This is one of the biggest challenges we face because preference engines are based on the idea that if we agree on three movies, we will agree on the fourth. But it often doesn’t work that way.”

Preference engines have encountered other problems, such as when people buy gifts.

“If I shop for a present for someone else, the system can get confused,” said Thomas Hofmann, chief scientist at Recommind Inc. and a computer science professor at Brown University. A bachelor buying a one-time gift for a baby could, for example, trigger the program to recommend more baby products in the future, when the suggestions are no longer relevant.

Preference engines also are not so good at telling when people’s needs have shifted, said Andreas Weigend, who was chief scientist at Amazon.com Inc. until 2004.

“Say you’re going on a vacation to China and you buy a few guides before you go,” he said. “Once you’re back, you’re no longer interested in guides to China. But the system keeps recommending them to you because it’s too naive to figure that out.”

Recognizing the limitations of pure formulas, some companies built their recommendation systems around human judgment. Pandora, for instance, relies on a large team of musicians to characterize the tone, feel and lyrics of songs.

“Music discovery is very social,” said Jon Herlocker, computer science professor at Oregon State University and co-founder of MusicStrands, which makes music recommendations by tracking what its subscribers do. “It should never be all machine.”

By letting people put in their two cents, preference engines have a better chance of making recommendations that are surprising, Weigend said.

“Old-style engines tend to narrow you down,” he said. “But the world is about people. When you let them in, you begin to create the serendipity of discovering truly new stuff.”

Online movie rental company Netflix Inc., for example, lets its customers write movie reviews. Netflix subscribers also can invite one another to become “friends” and make movie recommendations, peek at one another’s rental lists and see how other subscribers have rated other movies.

Combining all these tools has helped make Netflix recommendations more in tune with subscribers’ tastes, said Neil Hunt, Netflix’s chief engineer.

To gauge how well Netflix does, the company compared how well customers liked the recommended movies with how well they liked movies that were picked from a list of recent releases, many of which were heavily marketed upon their release on DVD. On a five-star scale, movies recommended by Netflix scored half a star higher.

“If you spend $50 to $100 million promoting a movie, you can persuade a lot of people to watch it,” Hunt said. “But if a movie stands on its own merit and matches an individual’s tastes, they may enjoy it a lot more.”

Although conventional advertising still has the power to drive large numbers of people to watch movies, preference engines help narrow the gap between blockbusters and independent films.

One example is “Control Room,” a documentary about Arab television outlet Al Jazeera. Because Netflix has a 12% share of the movie rental market, one would expect its share of the rentals for “Control Room” to be in the same range. But Netflix accounted for 34% of the title’s rental activity in the U.S. the week it was released on DVD. The difference, according to Hunt, is primarily because of Netflix’s recommendation tools.

Still, independent movies such as “Control Room,” which was rented 137,986 times by Netflix members that week, were far short of blockbusters backed by big marketing campaigns such as “I, Robot,” which was rented 908,641 times the week it debuted on DVD.

“This is really powerful,” said Netflix subscriber Mike Kaltschnee, a 40-year-old vice president for a stock photo agency in Danbury, Conn. “There are 45,000 movies in the Netflix collection. This helps me find the good ones. Before, my taste was limited to what I happened to see at the local rental store. Now I rent foreign films, like ‘Whale Rider.’ I never would have rented that before. Not in a million years.”

Like Rhapsody, the vast majority of Netflix’s titles, up to 95%, are rented at least monthly.

That suggests that recommender systems are exposing people to a wider variety of movies, said MIT’s Brynjolfsson.

“Previous Internet researchers, including myself, thought that the promise of the Internet was that it would drive down prices and produce more efficient commerce,” he said. “That missed the forest for the trees. The biggest role of the Internet was to help you find products you wouldn’t have otherwise come across.”

But the benefit may carry a corresponding cost.

“There’s a dark side,” Brynjolfsson said. “In the physical world, I bump into all kinds of people by chance. But online, if recommenders were perfect, I can have the option of talking to only people who are just like me. There’s a danger that if we don’t have some level of shared interaction, it can be destructive to our social cohesion.”

Cyber-balkanization, as Brynjolfsson coined the scenario, is not an inevitable effect of recommendation tools but rather one possible result.

“That is to say, the cheerful scenario of a global village is also not an automatic outcome of the Internet,” he said. Because people are inundated with content in the digital world, the tendency is to rely on recommender systems to filter in only the things we’re interested in. “It does take a conscious effort for people not to give in to tribalism.”

Not everyone is as worried.

“There are two arguments against the theory that technology is going to balkanize people,” said Joseph Konstan, computer science professor at the University of Minnesota. “The first is that the technology is not that good. The second is that for these systems to work, you need the commonality of popular items in the world. Without them, these systems would fail.”

It’s also possible to build a recommendation system that lets people decide how much mainstream material to let in.

Researchers at the Georgia Institute of Technology built an online newspaper in 1995 called the Krakatoa Chronicle that let readers devote a portion of their personalized newspaper to popular articles read by others.

“Say you hate sports, but you want to know who won the Super Bowl so you can go to work knowing what people are talking about,” Konstan said. “People ... want some level of commonality.”

Paul Resnick, a professor at the University of Michigan School of Information, came to a similar conclusion after he and a graduate student, Kelly Garrett, examined whether Americans were using the Internet to screen out ideas they didn’t agree with.

In their 2004 survey of 1,510 adults, Resnick and Garrett wrote that “wired Americans are more aware than non-Internet users of all kinds of arguments,” even those that challenged their own point of view.

“In fact, some people have a slight preference for information that challenges them,” Resnick said in an interview.

But recommenders are unlikely to get someone to try something they have no interest in. Take Lester, for example.

“I have pretty broad tastes, and iTunes has definitely opened up some genres for me,” he said. “But it didn’t get me into country or classical music. I’m just not interested in that type of music.”