At Last, Computers Are Getting Milk
Animators working on the summer sleeper “Shrek” spent two months crafting a technically complex 3 1/2-second shot. The star wasn’t the rotund green ogre in the movie’s title role, but a frothy glass of milk.
Considering that Hollywood special-effects wizards have managed to create dinosaurs that stalk real actors, computer-generated hair that appears to blow naturally in the wind and countless fiery explosions, an ordinary glass of pure white milk wouldn’t seem too difficult.
In reality, the task is among the most daunting in the field of computer graphics. Even with two months of effort, the milk in “Shrek” is still a long way from the quality animators would like to see.
Stanford University computer scientist Henrik Wann Jensen has been searching for a way to create realistic digital milk, and after a year of work he has come up with what computer graphics experts say is the first truly lifelike solution.
“It seems like something that’s so simple, but no one has rendered a convincing picture of a glass of milk,” said Jensen, who will present his findings this week at the annual Siggraph computer graphics conference at the Los Angeles Convention Center.
In his quest to breathe light--and therefore life--into purely digital objects, Jensen relied on a simple pocket laser pointer and a few bags of groceries from the local supermarket. He also used the same advanced mathematics and physics that optics experts use to study the behavior of individual photons of light.
“Who’d have thought so much science would be needed just to make milk look good?” said Eugene Fiume, chairman of the computer science department at the University of Toronto and an expert on creating realistic computer-generated images.
Hollywood stands to be the biggest beneficiary of all this science. Jensen’s technique is expected to go a long way toward making skin and faces look far more realistic than they did in “Final Fantasy,” the most ambitious attempt yet to digitally create a photo-realistic world. It could help usher in the long-awaited arrival of lifelike virtual actors who perform dangerous stunts or even fill starring roles.
The technique also could be critical to physicians in diagnosing certain types of skin diseases and to cosmetics makers in testing the effectiveness of their products. Even art historians could use it to render digital versions of ancient marble sculptures and see them as they looked in their original glory.
Half a dozen digital entertainment and special-effects firms already have invited Jensen to brief them on his technique. Pixar Inc., the studio that produced the groundbreaking “Toy Story” films, has hired him as a consultant for its upcoming film “Nemo” and for other projects.
“When I see that kind of thing happening, I realize they’ve been thirsting for this,” said Mark Levoy, an associate professor at Stanford who worked on the technique with Jensen. “It’s going to make an impact quickly.”
Scientists have tried for years to infuse the artificial worlds created by computer graphics with the translucent qualities that bring a sense of life to organic materials such as milk.
“Pretty much everything about milk is hard,” said Jonathan Gibbs, the film’s lead effects animator at PDI/DreamWorks in Palo Alto. “It’s surprising. It’s such an absolutely mundane thing.”
Accurately rendering translucent materials is one of the last significant hurdles in computer graphics. Biologists define life based on an organism’s ability to convert nutrients into energy so it can grow and reproduce. In the world of computer graphics, life boils down to things such as color, fluidity of movement and, most of all, translucence.
Capturing the way light softens the tip of a nose, glints off a leafy tree or glows inside a glass of milk is key to convincing the human eye that it is witnessing something real. After hundreds of thousands of years of evolution, people are instinctively aware of even the slightest faults in computer models of things like skin.
“All day every day, all you do is look at your skin and other people’s skin,” said Scott Stokdyk, digital-effects supervisor at Sony Imageworks in Culver City. “If anything is a little bit off, people immediately recognize it. How often have you looked at someone and seen that they were pale and wondered if they were sick, and they are sick?”
The problem is the enormous complexity in the way light interacts with translucent objects.
When particles of light, called photons, hit a glass of milk, a small portion of them will bounce right off. But most will enter the glass and keep on going until they bump into fat globules and other molecules suspended in the liquid. Each photon could bounce around hundreds of times before exiting the glass.
In the meantime, the mishmash of photons illuminates the glass of milk, producing its telltale glow. The effect can be observed easily by shining a concentrated beam of red light from a laser pointer into the glass and seeing it turn pink.
The zigzagging paths of these millions of photons can be modeled on a computer. But it would take months for today’s computers to process all the calculations. “It’s not really an appropriate way to approach the problem,” Fiume said.
So animators looked for shortcuts. They created computer models that assumed light bounces off translucent surfaces, or maybe knocks around a few times under the surface before exiting. Those techniques tend to make objects take on an artificial, plastic-looking appearance. Animators compensate with a few tricks to add light to their pictures.
At Sony Imageworks, artists working on the movie “Stuart Little” added a pinkish glow to the ears of the title character--a mouse--whenever he stood in front of a light source, Stokdyk said.
“Whenever we have surfaces that need to have some translucence, we look at what we want the final product to look like and we figure out how to cheat that,” he said. “We don’t normally go into the physics of it. It is too computationally intensive.”
For computer graphics researchers, the goal is not to cheat but to produce the most realistic image possible in a way that requires the fewest computing resources. The crux of Jensen’s solution is to approximate the effect of most of the photon bouncing by replacing it in his model with an easy-to-calculate fuzzy sphere.
“Milk scatters a lot,” said Pat Hanrahan, a professor at Stanford’s computer graphics laboratory who collaborated with Jensen on the research. “The theoretical trick is to recognize that’s it’s like a blur. So instead of following a photon, we just say it will diffuse. That’s an enormous simplification. People were trying to follow all of the photons, and that’s expensive. If it’s expensive, people aren’t going to do it. This way is thousands of times faster.”
Jensen, a 31-year-old from Denmark, was a postdoctoral fellow in MIT’s computer science department in 1999 when the problem worked its way out of his subconscious.
He was shining his red laser pointer on marble slabs and noticed that the beam spread into a fuzzy circle when it hit the marble, an indication that the light was scattering beneath the surface. He quickly shut off the light in his office and saw that the entire slab lit up with a faint pink tint. Then he turned his laser pointer on a glass of milk.
Those simple experiments led him to the realization that a computer model would have to deal with the diffusion of light below the surface if it were to have any hope of producing a realistic image of a translucent material.
Jensen’s quest began in earnest last fall with a comprehensive search on the Internet. That yielded references to a few promising papers in scientific journals, which he tracked down in the musty, fluorescent-lighted stacks on the second floor of the Physics Library at Stanford, where he is a research associate in the Computer Graphics Laboratory. For months he pored over papers, books and doctoral theses.
The trail led him to a 9-year-old medical physics paper in a Journal of Applied Optics. It described what happens when light from a single point is shone onto skin, an important means of diagnosing skin disorders such as psoriasis.
The paper proposed a radically simplified model that used a fuzzy sphere as a stand-in for a raft of calculations describing the trajectory of a single photon.
Jensen figured he could adapt the model to computer graphics. By early December, Jensen’s results were starting to look good. The amount of light scattering predicted by the model for a virtual marble block almost perfectly matched the measurements taken from an actual marble block.
Then he was ready to put his model to a more thorough test. He headed for Andronico’s supermarket in Palo Alto on New Year’s Day in search of milk and other translucent items.
Armed with a red laser pointer, Jensen and fellow researcher Steve Marschner tested produce, dairy products, meats and condiments. Several kinds of soft cheeses exhibited some light scattering, as did oranges and potatoes. The laser pointer turned a green grape into a glowing pink orb. Chicken breasts, turkey fillets, Catalina salad dressing and ketchup all produced significant scattering.
They spent $27 on milk and other groceries that they took back to the lab for further examination in a pitch-black room equipped with a special camera. Jensen’s computer images seemed to disperse light in the same way as the real thing.
After adjusting some of the parameters in the model, Jensen used it to render a marble bust of Diana the huntress, of Roman mythology. A dual 800 MHz Pentium III PC created the image in just five minutes. Traditional computer graphics methods would have taken nearly 21 hours to produce the same degree of clarity.
In Jensen’s version, the corners and edges of the bust take on a softened glow, and parts of the face and hair appear to be lighted from behind. Because the photons scatter throughout, the object almost becomes a light source itself, said Levoy, who teaches a class at Stanford called the Science of Art.
Jensen said he was most eager to compare skim milk with whole milk and cream. He expected to see a pattern emerge among the three. He was right.
“The main difference between skim milk and whole milk and cream is the amount of fat globules, which make milk more opaque,” Jensen said. “Light spreads in skim milk more than in cream, because there is less stuff to smash into.”
Just as he’d predicted, the glass of skim milk spread the most pink light.
“The Holy Grail is to render images that are indistinguishable to the eye from real-life images,” said Fiume of the University of Toronto.
“People won’t say, ‘Hey, look at how good that milk looks!’ They would complain if it doesn’t look right. That’s really when you know you’ve succeeded--when people don’t notice it because it looks so good.”
To compare new and traditional animation images, go to https://latimes.com/animation.