Say What? It Doesn’t Compute
Within the next decade, we’ll see machines that truly understand language, so we can communicate with them as easily as the guy behind the cash register: “Gimme a burger and a medium Coke. Make that large.”
In the future, a computer hearing such a command will not respond with baffled syntax errors, puzzling about whether “that” refers to the entire order, just the burger or just the Coke. But computers that can really comprehend language as humans use it are a long way off. And nothing illustrates that better than a computerized Army training simulation developed at the University of Southern California.
The simulation--which runs like an enormous video game, playing on a screen about 4 yards high and 20 yards wide--puts the operator, who plays the part of an Army lieutenant, on the ground in a generic city that appears to be somewhere in the Balkans. The lieutenant comes upon a U.S. convoy that has been involved in a traffic accident with a car driven by civilians. A young boy lies on the ground; a woman, apparently his mother, grows increasingly agitated.
The simulation is designed to respond to the operator’s actual voice. Snap out a question, such as, “What happened here, sergeant?” and the platoon leader will fill you in. Later, the medic bending over the injured child intones, “Sir, the kid. . . .” His voice trails off, and he glances at the mother. “We’ve got to get a Medivac in here.” The lieutenant’s job is to secure the area, call in a chopper for the child and keep everyone safe. Give the right orders at the right time and everything works out. Make a mistake and things get ugly.
As virtual experiences go, the Mission Rehearsal Exercise System, or MRES, is completely mind boggling. The sound is excellent, and the graphics, though cartoonish, are crisp and well-developed.
While members of elite tactical units run through this kind of scenario with real people, the military doesn’t have the resources to walk every grunt through a personal development program using dozens of people playing roles. Using computers, however, could let everybody get a taste of what it’s like to be forced to take command in a volatile situation.
The MRES promises a kind of completely intuitive system, designed to run based solely on cues issued by the operator using standard, spoken sentences. That kind of power is the future of entertainment--and, indeed, the future of computing. Part of the reason today’s gizmos are so hard to use is that computers are completely literal, and our interactions with them require us to communicate in unnatural ways. Typing search parameters into a search engine doesn’t require a knowledge of grammar but of algebra: “Nixon AND Sirica AND Liddy OR Magruder.”
Even a child understands what’s meant by the phrase “Time flies when you’re having fun.” But a computer could very easily interpret that as an order to use a stopwatch to see how fast insects move. Getting a computer to understand and respond to the ways humans use language is staggeringly difficult.
That’s the real charm of the MRES: It is designed to respond to people the way they talk. Except, unfortunately, the simulator does not actually work.
The young man playing the role of the lieutenant in the MRES demonstration I sat in on was clearly following a script. That’s a reflection of the limits of the technology.
“If he says what we expect him to say, it’s fine. If he doesn’t, then we’re in trouble,” acknowledged Bill Swartout, director of technology at USC’s Institute for Creative Technology, where the MRES was built as part of a $45-million grant from the Army.
If, instead of asking something like “What happened here, sergeant?” the operator simply snaps, “Who’s in command?” the system probably wouldn’t have had any idea of what was going on. Swartout explained that the demonstration was designed as a kind of look into the future. “We want to show what we think will be possible.”
The problem is not really hardware at this point but developing incredibly complex software that can grasp the fundamentals of human communication. As the MRES demonstrates, such systems are a long way off.
*
Dave Wilson is The Times’ personal technology columnist.
