New apps are helping smartphones to recognize images
“OK, this one is pretty cool,” said 6-foot-tall Omar Tayeb as he held his left hand in front of himself and snapped it with his iPhone.
He didn’t move the phone away. He left it there as an app detected each of his fingernails and proceeded to paint each one. On the phone’s screen, Tayeb’s nails turned a dazzling shade of Maybelline purple.
“So the app was able to detect where my nails were, paint them in one of Maybelline’s nail polish colors, then adjust to the room’s lighting so it looks natural,” Tayeb said. “It’s quite an impressive usage.”
And it is impressive, given that as little as five years ago the smartest smartphones would have struggled to recognize the most basic of images, let alone identify individual fingernails and accurately color them.
Early fashion and beauty apps used image-recognition technology to crudely superimpose hairstyles and clothes onto people’s bodies, often to hilarious effect: shirts and dresses sitting at awkward angles, hairstyles that looked more like clown wigs.
But technology moves fast, and of all the research in artificial intelligence technology, image recognition is moving the most quickly through popular culture.
Neural networks — computer programs roughly inspired by the workings of the human brain — are especially good at recognizing objects, faces and digital images. Google uses such technology to try to tell a trout from a salmon in a Google Image search.
Now image recognition is going mobile. Small but sophisticated smartphone cameras serve as mobile machine eyes, scouring the world for images with the help of their human owners, and processing them right there on the phone.
More than 1 billion smartphones were sold in 2013, according to research firm IDC. And where there are people, there are marketers trying to sell them stuff.
Like ketchup. Blippar users can snap pictures of a Heinz ketchup bottle and get recipes and ketchup videos. Or cereal. Kids can point their phones at a box of Lucky Charms and interact with leprechauns through the iPhone screen in a kind of reality-meets-computer-screen mash-up known as augmented reality.
But Tayeb, a 28-year-old developer turned founder, has loftier goals. Blippar is building a mobile image search engine, never mind the direct marketing. Blippar is teaching its platform to recognize everyday objects. Point your phone at a dog and it will tell you what breed it is. Point it at a book and you can read the reviews. Point it at a movie poster and it will pull up the relevant IMDB and Rotten Tomatoes Web pages.
Blippar wants to making “blipping” — the act of scanning an object through the app — as habitual as opening Google Maps in a search for directions or opening Yelp for restaurant reviews.
Blippar is one of many commercial operations riding the image-recognition wave. Semiconductor company Qualcomm Inc. in San Diego already makes many of the chips that run mobile devices. It has also developed Vuforia, an image-recognition software platform for developers that underlies more than 15,000 apps with an augmented or virtual-reality component.
Neural networks, points out Tim McDonough, Qualcomm’s vice president of marketing, are able to accept feedback and learn new things.
“We call it cognitive computing, which is kind of a dorky term that makes you think of a person playing chess, but the basic idea is machines can learn, and they can make decisions based on what they learn,” McDonough said.
“You can train a computer to recognize images by giving it really large numbers of images to learn.”
Blippar’s image-recognition technology works in a similar way: A person feeds the computer large numbers of annotated images and, over time, the computer becomes more accurate at identifying what it sees. A computer may only need to see one image of a movie poster to be able to recognize that movie poster every time, but for other, more generic objects, it may need to see tens of thousands of images before it can identify a specific type of tree or breed of animal.
Advertising is an obvious use, with brands such as Lego, Pepsi and McDonald’s already finding ways to incorporate image recognition and augmented reality into their marketing materials.
Companies like Facebook are using facial-recognition technology to identify and tag people’s faces in photos.
The technology helps make driverless cars possible: Is that a person at the crosswalk or a lamppost?
Some of the possible uses are semi-comical. People wearing virtual-reality headsets, deeply immersed in pretend worlds, might not realize they’re about to trip over a real-life electrical cord. Machine eyes could pass the warning — or tell them that a stranger has just entered the room.
Jay Wright, vice president at Vuforia, has his own vision: He’d like to summon email to show up in his field of view, type by simply holding his fingers out in front of him and to never need to use a device with a screen.
“I think it’s on the five- to 10-year horizon,” he said.
Twitter: @traceylien
