3-D Technology Blends Fact and Fantasy

Sometimes less is more.

Take “augmented reality,” a technology that is starting to make its way out of university research labs and into aerospace manufacturing facilities, hospitals and other earthbound locales.

Augmented reality uses less of the splashy 3-D graphics that have made the fame of its cousin, virtual reality, but it promises to deliver far more practical applications.

Instead of being immersed in a 100% computer-generated (and often fantasy-inspired) virtual environment, an augmented-reality user wears goggles that superimpose helpful graphical data on real-world images viewed through one’s own eyes.

A computer constantly feeds data to the goggles through a wireless connection. The result, to take one example, is that an otherwise nondescript metal slat in an aircraft factory appears studded with markings that show an assembler where to drill the rivets that attach it to the frame of a wing or fuselage.

The uses are multiplying. Doctors in North Carolina are already using augmented reality, or AR, systems to help pinpoint the center of a tumor so they can perform better biopsies. At Boeing, AR speeds up mechanics’ inspections of 757 airliners. Eventually, AR could change the way museum-goers view exhibits and help amateur mechanics repair their own cars by supplying computerized information that enhances the real-world experience.

“It’s the most promising interface between humans and computers,” said Henry Fuchs, a professor studying AR at the University of North Carolina in Chapel Hill.

Ivan Sutherland, one of the fathers of computer graphics, is generally credited with creating augmented reality in the 1960s, when he conducted experiments in his lab at the University of Utah.

One of his disciples from that era is Ulrich Neumann, a computer scientist at USC’s Integrated Media Systems Center. Neumann is working with McDonnell Douglas engineers to create an AR system that will display text and graphics so aircraft assemblers at its Douglas Aircraft Co. facility in Long Beach can build planes more quickly and accurately.

Such a system would help workers by relieving them of the need to refer back and forth to blueprints or instruction manuals, said Anthony Majoros, a senior engineer scientist in McDonnell Douglas Aerospace’s advanced transport aircraft development group in Long Beach.

“We’ve identified about 20 classes of applications for aircraft manufacturing and maintenance that could benefit from augmented reality,” Majoros said. “It makes things easier to do and more likely to be done correctly compared to reading instructions on paper.”

At Boeing, David Mizell is using a grant from the Defense Advanced Research Projects Agency to try to use AR to simplify the process of bundling hundreds of wires.

Traditionally, workers use foam boards with complicated pre-printed diagrams to lace the wires into a bundle. AR might allow a worker to use a blank board and rely on graphics in a head-mounted display to show where each wire should go, said Mizell, manager of virtual systems at Boeing’s information and support services group in Bellevue, Wash.

Researchers at the University of North Carolina are focusing their efforts on medical applications. Fuchs has studied ways of using AR to display computer-generated images from CT scans so doctors can see the precise location of a tumor as they prepare to operate on a patient. So far, doctors have used the technology to perform biopsies on three women with breast tumors.

Eventually, augmented reality could be used in all kinds of surgery, Fuchs said.

*

AR can also help people view information that would not normally be displayed visually. For example, an inspector touring a chemical plant could view a readout of the temperature and other conditions inside a holding tank displayed on the tank’s surface--data that could be transmitted through a wireless connection from a sensor in the tank via computer to the inspector’s head-mounted display, said David Breen, an engineer and assistant director of the Computer Graphics Laboratory at Caltech in Pasadena.

For all that, the technical hurdles keeping AR from wider application are daunting. For starters, an AR system must be able to recognize a critical number of reference points--often shifting with the motion of the user’s head--to accurately line up graphical images with objects in the real world.

At USC, Neumann is using statistical-recognition techniques to help a computer and camera find those reference points. The computer analyzes an image and decides if its shape corresponds to a feature--such as a circular target or the square edge of a table--it has been programmed to recognize. Then it maps the graphic elements on the user’s view screen to match up with the reference points.

That’s a tricky process because the algorithms used to identify key features sometimes break down. When they do work, they often take a long time to chug through their statistical calculations.

*

But the calculations must be processed rapidly in order to be repeated constantly as the camera’s eye shifts angles. Neumann and his colleagues are trying to write better programs that require fewer calculations.

Ideally, once the reference points are identified, the computer can use triangulation to figure out the location and orientation of the camera--a stand-in for an AR user.

“It’s like using the stars to navigate,” Neumann said. “They are fixed in relation to each other, and you can tell where you are based on how they look in reference to each other.”

There are also hardware obstacles. No one has yet developed goggles affordable and light enough to be worn for a long period.

For some delicate applications such as surgery, the goggles would have to be sophisticated enough to measure eyeball movements to figure out exactly where the user is looking.

But those obstacles don’t stop hard-core augmented-reality fans from envisioning a wide range of applications.

Ronald Azuma, research staff member at the Hughes Research Laboratory in Malibu, would like to use the technology to make history lessons more exciting. He imagines outfitting visitors to a Civil War battlefield with AR goggles so that they would see the grassy slopes as they appeared to soldiers in the 1860s, instead of as they look today.

Said Azuma: “Certain types of educational experiences might be more interesting and effective if they were taught like that.”