Mapping the automation of automobiles
Luke Pulaski circled a Berkeley block in a white Volkswagen hatchback looking for a place to park. After several loops, he found a spot — right in front of a research outpost of Here, the mapping software division of Nokia.
He proceeded to make his way to the entrance before stopping, patting his pockets, and turning around. He’d forgotten to feed the parking meter.
“Well, that would have been embarrassing,” he said, imagining the car getting ticketed or towed.
His VW is no ordinary hatchback. It’s a specialized data collection vehicle, plastered with Nokia stickers and chunky antennas, laser scanners, cameras of various sorts — tens of thousands of dollars’ worth of data capture devices.
He walked inside. Engineers were monitoring hundreds of similar vehicles around the world, gathering data to create the next generation of online maps.
This group of engineers isn’t particularly interested in traditional mapping, also known as “turn-by-turn” — the kind people use to find their way with Google Maps.
The maps they’re developing aren’t even intended for people to use, not directly. Ultimately, they are intended to guide self-driving cars. Robot to robot.
Berlin-based Nokia Here calls them HD maps, for high definition. HD maps are considered so important to the auto industry that a group of three carmakers — Daimler, BMW and Audi — are in talks with Nokia to buy Here for themselves.
The automation of automobiles is well underway. Audi and Volvo have already demonstrated cars that can park themselves. As technology moves ahead, human drivers will increasingly become less relevant and at some point — no one’s quite sure when — fully automated self-driving cars will be commonplace on the roads and highways.
In such a world, the automakers don’t want to depend on Google — which is also heavily invested in HD mapping — or on anyone else with so much power over map development that the automakers lose control of the data and, ultimately, the way they design and manufacture their cars. It’s not just automakers, either: Uber, the on-demand transportation app giant, bought map start-up deCarta in March.
HD mapping, however, is no trivial pursuit.
“The maps you look at online on Here.com or in Google Maps, or even in your car’s dashboard, they’re very simple representations of the world,” said John Ristevski, Nokia Here’s vice president of reality capture and processing. “But for self-driving cars, we need at least 99% accuracy.”
And when he says accuracy, he doesn’t mean just getting the street names right.
For a car to drive itself, it needs to know its environment down to how many lanes are on a road and the exact location of road markings. Sure, self-driving cars have sensors, but according to Ristevski, they still need a reference map.
An HD map also helps the car make decisions, such as when it should begin changing lanes as it approaches a freeway offramp. For that level of detail, the company invests in expensive rigs that capture more information than a typical driver would ever need.
Each rig, like the one mounted on the Berkeley Volkswagen, looks like a mechanical lollipop sticking out of a car’s roof. At the top is the GPS antenna, which collects basic information such as where in the world the car is located.
Below that is a Lidar sensor, a laser scanner that captures 700,000 points per second. The sensor lets engineers construct dense 3-D models of every surface the laser hits so they can identify road markings, signs and even road curvature. Below that is a box containing four panoramic cameras that take a photo every five meters traveled. The cameras have an 182-degree field of view, capture images at 24 megapixels and allow Nokia Here to construct a full 360-degree image.
The whole rig connects to a computer in the car’s passenger seat via an ethernet cable, and the data are saved to a hard drive that is sent to one of Nokia’s data centers in North Dakota for processing once it’s full. Ristevski estimates a driver will record approximately 100 GB of data a day.
“We get amazing detail,” Ristevski said. “Every detail of the road surface, the building surfaces, you can even see 3-D people standing by the side of the road.”
According to Pulaski, who oversees the deployment of the vehicles and rigs, the engineers back in Berkeley can see how much ground a car has covered each day, and if they’ve run into any problems.
“So if there’s an obstruction on a road, like maybe there’s too much snow or there’s a truck in the way, they can drop an icon on the map and we’ll come back to it,” he said.
With a new standard of maps comes a new set of problems, though. Where turn-by-turn maps can be created once and updated every year, or sometimes every five years, HD maps place new demands on mapmakers. The maps need to be as up-to-date as possible, which means the work of Nokia’s little Volkswagens is never quite done.
And because the rigs capture so much data, mapmakers now also have to develop and maintain algorithms to automatically extract information from images, teach computers to construct maps from those images and figure out a way to map every street in the world in high detail without error.
“That’s why making maps is so hard,” Ristevski said. “It involves tech, people, operation at a global scale, data extraction. That’s why it’s so hard to replicate what we do.”
HD maps are already being used today in advanced driver assistance products, where automakers use data like road slope and curvature to help cars with fuel efficiency. And according to Ristevski, in the next five years, we’ll get closer and closer to self-driving cars.
“Everything from lane-change assistance to entry to a freeway, the HD map will evolve to support that,” he said. “And eventually do fully autonomous driving.”
