Just Beyond Our Windows

Point. Click. Drag.

Point. Click. Drag.

Tired of operating personal computers like this? A lot of researchers are, and they’re trying to change the way people interact with the ubiquitous beige boxes by creating virtual spaces that take into account the way people behave in the real world.

When Apple introduced the Macintosh and its graphical user interface, or GUI, in 1984, the folders, windows, icons, menus and mouse pointer that made up the desktop took the computing world by storm. It was quite a change from the black-and-green screens and obscure commands that had dominated personal computing until then.

Pioneered in the 1970s at Xerox’s Palo Alto Research Center, popularized by Apple in the 1980s and embedded into daily life by Microsoft in the 1990s, the GUI ushered in a new era of computing.

But that was 18 years ago. Things have changed.

The first Mac had no hard drive. Today, hard drives are packed with gigabytes’ worth of MP3s, digital photos and word-processing documents. Back then, there was no World Wide Web, and e-mail was virtually nonexistent. Today, the Web is everywhere and in-boxes overflow with messages.

All of which leads to a sort of information glut.

“The Windows environment is not the environment that is going to take us into next century,” said David Gelernter, a professor of computer science at Yale University and chief science officer at Mirror World Technologies. “It’s out of control. It’s out of hand. It was designed for a vastly different and vastly simpler information world.”

So in the same way that computer scientists in the 1970s sought a more user-friendly interface, researchers of today are investigating whether there is something better than Windows and Mac OS.

“When you give a digital camera to someone and they can take thousands of photos, the task of managing those photos, of organizing them, of sending them to other people or just browsing them, becomes terribly hard with the current interface,” said Daniel C. Robbins, 3-D Interface Designer at Microsoft Research. “We’re trying to make that easier.”

Robbins believes that one way to deal with the massive amounts of data on computers is to transform the flat digital desktop into a three-dimensional landscape that works with the natural human ability to organize information spatially.

The idea, Robbins explained, is that a 3-D environment would provide a sense of “place,” which the countless layers of similar-looking folders of the current desktop fail to do. A 3-D environment would have “landmarks” that people could use to group information as they see fit.

One test project, called Data Mountain, presents users with a 3-D view of a mountain range that is used as a file storage environment. The easily recognizable cues that the mountains provide, such as hills, valleys, paths and rivers, allow users to orient themselves in the 3-D space while giving them the freedom to put files wherever they choose.

“When you start putting things in a hierarchy, in folders, it almost makes that structure sacred and you become very afraid to mess with it,” Robbins said. “If you give people a more free-form space with rich cues, it lets them move things around and make ephemeral relationships that change over time, because really the way we want to look at our information completely changes depending on the task and the context of what we’re doing.”

In future versions of Data Mountain, Robbins anticipates that the environment will react to a user’s habits. So if a user put files that they often delete in a certain place, the environment might start to form a chasm into which files can be dumped and deleted.

Likewise, if a user stores files for a long time in a certain area, those files might start to look like ice, indicating they have not been used but are safely stored. The idea is to make the environment react to a person’s habits while keeping the environment stable so that actions can be easily remembered and repeated.

But different people have different spatial abilities, so a set 3-D environment wouldn’t work for everyone. Robbins envisions a personalized system that would use artificial intelligence techniques to adapt and refine a basic environment into a customized system.

“We would imagine that the system would, when you first encounter the computer of the future, somehow in an unobtrusive and hopefully fun way, assess your spatial abilities and the kinds of view you would want to live in,” Robbins said.

“One person might have a metaphor that looks like a forest,” he said. “Someone else might have a more abstract metaphor that would approximate living inside a Kandinsky painting with just abstract geometry that you personally have assigned meaning to. The space should really be tuned to the user.”

Gelernter also sees 3-D landscapes as a way to escape the messiness of the desktop metaphor, but said few mainstream computer programmers have demonstrated the ability to create something consumers can use.

“We’re going to see a lot of 3-D systems that insofar as they are three- and not two-dimensional, are even more preposterously complicated than current Windows-type environments,” he said. “This is a new degree of freedom, and I think most of the industry has not shown that it makes very good use of that freedom.”

His alternative is called Scopeware, a system he developed at Yale and then spun off into a commercial product with his company, Mirror Worlds Technologies. Scopeware departs from the hierarchy of folders in the desktop by placing every piece of information, from e-mail to bookmarks to photos to office documents, into a “stream” organized chronologically.

The stream is a straight line of documents organized onto what looks like cascading cards that stretch back into the past and forward into the future. Nothing has a name. When you scroll over a card, a thumbnail and description of the data pop up to the side.

The research system is based on peer-to-peer technology, which allows computers to share data directly with other computers. The idea is that when a user creates a document on a desktop computer, it also shows up at the front of the stream on a laptop or hand-held without extra effort.

Peer-to-peer also would allow users to tune into other streams, perhaps those of a business partner or newspaper, and seamlessly integrate their public content into a private stream.

Other user-interface researchers are less critical of the desktop, but see an issue with devising interfaces that work equally well on the desktop and the burgeoning number of mobile devices.

The GUI “is an interface that works pretty darn well on a display sitting in your office, doing office kind of work,” said Jock Mackinlay, a member of the research staff at Xerox PARC. “But for mobile devices there is an obvious problem, which is that the standard GUI doesn’t work. The display on my Palm or my mobile phone is just too small to support the windows and the menus.”

Mackinlay said he sees mobile devices making use of speech technology and incorporating “big but small displays,” which would either fold into something pocket-size or utilize a tiny projector that could display information on any flat surface.

Still others envision moving away from the computer as a separate device. Hiroshi Ishii, associate professor of media arts and sciences at MIT’s Media Laboratory, seeks to build upon the concept of “ubiquitous computing,” the seamless melding of technology with everyday objects, pioneered by late scientist Mark Weiser.

“The GUI was so successful that people ... tried to remove the physical models and physical practices, and tried to imitate everything in the digital world,” Ishii said. “To me, that seems to be wrong.”

Ishii wants to bring a “tangible” interface to applications. Instead of staring at a screen and manipulating a mouse to do work, Ishii thinks direct physical manipulation of objects that take advantage of computers is a more natural and useful way to deal with technology.

For example, as an alternative to logging on to a Web site to check the weather, Ishii helped design a bottle that reveals sounds corresponding to the day’s weather when opened. A sunny, warm day might sound like chirping birds, while a rainy day might emit sounds of thunder.

Urban planning is one of many areas in which Ishii believes tangible interfaces are superior to computer simulations. He was involved with a project that enables urban planners to manipulate models of buildings on a table; a computer detects their orientations and positions.

“Urban planners have used physical models of buildings for 5,000 years,” Ishii said. “Now computers provide advanced simulations, but they are inside the computer and separate from a physical model.... People can solve this problem by moving buildings, like kids move Lego blocks. It’s much more natural and direct.”

*

Eric B. Hanson can be reached at eric.hanson@latimes.com.