During his two-hour morning bike ride, Eric Hartman doesn’t pay much attention to his iPhone.
But the iPhone is paying attention to him.
As he traverses the 30-mile circuit around Seal Beach, Hartman’s iPhone knows precisely where he is at every moment, and keeps a record of his whereabouts. That data is beamed to Apple Inc. multiple times each day, whether Hartman is using his phone to take pictures, search for gas stations or check the weather.
And it’s not just the iPhone that’s keeping track.
Buying milk at Ralphs? Playing World of Warcraft? Texting dinner plans to friends? Watching an episode of “Glee”? It’s all recorded.
Over the course of a day, hundreds of digital traces pile up, each offering more insight into the way Hartman and his family live.
For this kind of surveillance, no fancy spy gadgets are needed. The technological instruments that capture details of the Hartmans’ lives are the ones they use most often: their computers, smartphones and TV systems.
“Essentially, each of us is being tailed,” said Kevin Bankston, an attorney at the Electronic Frontier Foundation. “Whether you went to the family planning clinic or a psychiatrist, or to be treated at the cancer specialists’ office,” data gleaned from cellphones alone reveal “an enormous amount about us.”
As a day with the Hartmans shows, few parts of our private lives remain shielded from digital observation. The modern home, stocked with networked devices, has become a digital transmission station, endlessly relaying data to a wide array of for-profit companies that are largely invisible to the average parent and child.
This explosion in the amount of data being collected has raised alarms in state capitols and in Washington, where lawmakers of both parties have proposed more than a dozen pieces of privacy legislation this year.
But regulatory efforts are drawing resistance from companies such as Google Inc. and Facebook that rely on personal information to sell advertising, and so far, none of the bills has passed. Privacy observers say it may be years before legal protections catch up to industry practices.
In the meantime, as more people become aware of the extent to which their actions are being recorded, some privacy advocates worry that people will begin to censor themselves when using technology and avoid going places or seeking information that others might find objectionable. That could include such areas as religion, sexual orientation and suicidal feelings.
“People might not bother to look into what they were going to look into,” said Ryan Calo, the director for privacy at the Stanford Law School Center for Internet and Society. “It’s too much of a feeling of being constantly watched and judged.”
The Hartmans’ digital devices, like those of millions of other U.S. families, feed into a massive river of personal data that flows back to the servers of technology companies, where it is often kept indefinitely. The data are sifted for behavior patterns that can be of great value to marketers eager to zoom in on the consumers who are most likely to buy their products.
Eric Hartman’s iPhone, perhaps the best-known mobile device of all, has been a lightning rod for privacy concerns.
Like other smartphone providers, Apple keeps databases of locations sent to it from tens of millions of iPhones. The company uses those databases, it says, to improve its product offerings, which include a mobile advertising system called iAd that allows advertisers to target consumers based on their current location.
Increasingly, marketers are trying to reach consumers where they are, so that coupon services like Groupon and Living Social can offer deals to restaurants and yoga classes that might be right around the corner. Location-based services will balloon to an $8.3-billion business by 2014, nearly triple what it is today, according to research firm Gartner Inc.
In addition to sending its location back to Apple, Hartman’s phone also checks in frequently with AT&T, which tracks the phone’s location so it can quickly send it calls and text messages when necessary. AT&T does not say how detailed its location logs are, but earlier this year, German politician Malte Spitz sued his provider, Deutsche Telekom, to see what data they’d collected about his whereabouts.
The resulting set of nearly 36,000 pieces of data was plotted by the German news site Zeit Online, and showed six months’ worth of Spitz’s movements around Germany — often at the street level, on trains, on planes — as well as when he made phone calls and sent text messages.
Earlier this year, after researchers discovered that the iPhone kept a detailed log of its precise whereabouts, Apple said bugs in the iPhone’s software had caused it to store up to a year’s worth of user location data. The software was later changed to store only a week’s worth of the data.
Data collection can also pop up in surprising places.
When Evan Hartman, 11, logs into World of Warcraft, a popular online video game played by millions, Blizzard, the game’s maker, records his location, what kind of computer he’s using and information about his playing behavior.
When Eric Hartman and his wife, Nia, go grocery shopping, he uses an iPhone application called CardStar that stores digital versions of loyalty cards for a dozen retail stores. Instead of carrying around plastic cards for Ralphs, PetCo, BestBuy or Footlocker, he can use his iPhone to show a barcode for the loyalty program to the checkout clerk.
Loyalty cards allow those chains to capture years of data about what each customer is buying — data they farm out to companies that specialize in scrutinizing the information for buying trends. The stores can then better target certain customers for promotions, or cluster products that are more likely to be bought together.
“We’ve found grocery retail to be a rich and fertile vein,” said Matt Keylock, an executive at Dunnhumby, which processes data for dozens of retail chains worldwide, including Home Depot, Best Buy and Ralphs owner Kroger Co. Whether the data tell them a customer is an adventurous, frugal, healthy or family-focused consumer, he said, “you can bring to life who a customer is based on the kinds of things they buy.”
Building a behavioral profile of a customer becomes even easier in the world of social networks, where the first thing consumers do is create a detailed self-portrait.
When Spencer Hartman, 16, reaches for his iPod Touch to check Facebook, he is mostly interested in seeing what his friends are talking about.
“I usually check it to see if anyone is saying anything funny,” he said. “Usually they’re not.”
When Spencer clicks on friends’ profiles or photographs, or leaves messages on their walls, he may forget what and who he clicked on that day, but Facebook, one of the largest data harvesters in the world, does not.
On a Web page describing Facebook’s ability to provide “precise targeting,” the social network says that each of its 750 million users “fills out a profile where he or she shares information such as: what they’re doing at the moment, their birthday, occupation, all-time-favorite band, movies, TV shows and other interests.”
The tendency of social network users to declare their interests to friends has become a boon to online marketers. On Facebook, advertisers can target their pitches to thousands of sub-categories that users have identified with, whether that’s “gay marriage,” “World War II history” or “insects.” (There are 6,600 U.S. Facebook users who have declared an interest in “insects,” according to an advertising tool on the site.)
Facebook has frequently faced criticism over the way it handles users’ private data. In one of the most recent instances, the security firm Symantec said a flaw in the social network for years left the personal information of hundreds of millions of users exposed to advertisers.
Indeed, consumers who spend hours each day using free Web search and social networking services from companies such as Yahoo, Google and Facebook may not always remember that the firms closely monitor users’ online habits in order to generate detailed profiles about their behaviors, preferences and buying patterns.
“By watching transactions and clicks we have a massive telescope into human behavior at a scale we’ve never had before,” said Prabhakar Raghavan, the head of Yahoo Labs, a division of the Web giant that invents many of its most powerful computing algorithms. Yahoo, which was the nation’s second-most-trafficked website in August with more than 177 million unique visitors, makes nearly all of its $6 billion in annual revenue from online advertising.
That same level of data gathering is now ramping up in the living room. When the Hartmans sit down in the evening to watch TV together, their TV providers are watching back. When they flip through the channels, their cable box records their viewing choices, while their Apple TV and Nintendo Wii devices relay their movie and TV rentals back to Apple and Netflix, respectively.
TV ratings have traditionally been estimated using large groups of volunteers who actively log the shows they watch, often by clicking a remote control to indicate they haven’t left the room. But set-top boxes are now thought to be a far better way to capture viewing data, without the need to involve the viewer. Boxes like those from TiVo, Time Warner Cable and Verizon can monitor what consumers are watching at any given second. Set-top-box data can include whether viewers have changed the channel, fast forwarded through commercials or muted the volume.
TiVo and Time Warner Cable declined to specify the precise types of data their boxes gather. They and other companies insist the data are “anonymized” — stripped of names and personal information that could identify the user.
But critics warn that even without identifying information, many kinds of data can still be traced back to individuals, often by comparing it with data from other sources.
“There is no longer any doubt that almost any type of data can be deanonymized if combined with external information,” Arvind Narayanan, a computer scientist at Stanford who studies data privacy, said in an email.
For now, the Hartmans are trying to take a realistic approach to their data. They understand that information about their habits may be collected by companies they’ve heard of and some they haven’t, and that it’s up to them to be the watchdogs of their own privacy.
Emily Hartman, 18, prefers to keep her digital dealings to a minimum when possible.
“I communicate with my friends when I’m at school,” she said wryly. “It’s a whole new experience.”
The Hartman family was located with the assistance of American Public Media’s Public Insight Network, a resource for members of the public to share their knowledge and experience with news organizations.