It’s not just that the fallen Hobbit is a figment of J.R.R. Tolkien’s imagination; even on the set of the blockbuster motion pictures, he was nowhere to be found. Gollum is made not of flesh and blood but of ones and zeros and is widely considered to be the most sophisticated character ever to be created through computer animation. He owes his existence to a team of animators and computer scientists—including Cornell assistant professor and Ph.D. alumnus Steve Marschner.

While doing post-doctoral research at Stanford, Marschner was part of the team that discovered a way to make computer-generated skin much more realistic. Their findings have been incorporated into the digital palettes of many special effects houses, appearing onscreen in a host of high-profile films. And on Valentine’s Day, Marschner and colleagues Henrik Wann Jensen and Pat Hanrahan donned tuxedos to accept an Academy Award for technical achievement in recognition of their contribution to movie magic. “This is one of the holy grails of computer graphics,” says Richard Edlund, chair of the Academy of Motion Picture Arts and Sciences’ Scientific and Technical Awards Committee. “One of the difficulties of creating lifelike characters in the computer world is the problem that skin is not opaque. If you render a faithfully scanned or created character and the skin is opaque, it doesn’t look real.”

The problem Marschner and his colleagues addressed, called subsurface scattering, describes the way light penetrates into and reflects off of skin and a wide array of other substances—including marble, cloth, wax, printer ink, bread, milk, plants, fish, and snow. Earlier computer models had treated such substances as purely reflective rather than translucent, a technique that made such renderings seem false to the human eye. A digital depiction of a marble statue, for example, would make it look like plaster. “Without accounting for translucency, it’s very hard to set things up so the skin looks like skin,” says the 32-year-old Marschner. “It looks too hard, or too smooth, or too shiny. But once you account for translucency, then suddenly you can get another level of realism. Subjectively, it looks like a soft material, like skin.”

The work grew out of a Stanford research effort directed by Marc Levoy ’76, M.S. ’78, known as the Digital Michelangelo Project, a marriage of art and science in which extremely high-resolution scans were made of several works including the famed David and the Slaves, unfinished statues that seem to be struggling to emerge from the rock. Although Marschner didn’t travel to Italy—he joined the project after the scans had been done—he worked on the data-crunching in Palo Alto. “The project was pushing the limits of three-dimensional scanning technology, but it was also creating an archive of data that’s useful for art historians and others who study these statues,” Marschner says. “They’re detailed enough so that with a large statue like the David, which is about twenty-three feet tall, over the entire surface you can see individual marks from the chisel that was used to make it.”

One of the project’s questions was whether, in the context of such detailed scans, marble’s translucency could cause problems with the data collection. The answer turned out to be no, but it sparked the team’s interest in the issue of subsurface scattering. Taking principles of physics into account and drawing on methods from medical physics, they created a simple model to describe the behavior of light in relation to translucent materials, then translated it into a method of rendering such materials via computer graphics. The researchers didn’t create a piece of software, but rather a tool for effects masters to use in their own rendering systems.

And use it they have, in such films as Harry Potter and the Chamber of Secrets, which features its own entirely computer-generated character in Dobby the House Elf, Harry’s high-strung servant. And in Terminator 3: Rise of the Machines, in the form of the lovely-but-lethal T-X robot. And The Matrix Reloaded in which the technique was used for “digital replacement heads”—when a computer-generated version of a real actor’s head is used to replace that of a stuntman, or to top a digitally rendered body during tricky action sequences. “When they give these awards, frequently they go to the studios and the people who did the implementing for the films,” says Professor Don Greenberg, director of Cornell’s Program of Computer Graphics (PCG). “But enough studios were using this that they looked deeper and went back to the source, which is the research.”

Although Marschner and the Stanford team were honored for their work on translucency and its improvements in skin rendering, they’ve also contributed to research on another physical attribute: hair. “Hair is something that you need in renderings of animals and people and monsters,” Marschner says. “It’s also something that the cosmetics industry is very interested in the exact appearance of. But the models that have been used for how light scatters from hair are really quite simple.”

In work begun at Stanford (and which Marschner is continuing at Cornell), the researchers made a breakthrough in the study of how light interacts with human hair. Using samples from volunteers—including blonde locks from Marschner’s wife, Heidi—they suspended a single strand from a clip, illuminated it with a focused light beam, and recorded the reflection with a digital camera. Their discovery: that the standard method for rendering hair, known as the Kajiya and Kay model, didn’t take into account the fact that many hair types have a cross-section that’s elliptical rather than circular.

This phenomenon (commonly seen in people of European and African descent, less so in Asians) causes the hair to behave like a lens, focusing light in certain directions. “People who have elliptical hair will often have glints that you can see if it’s reasonably light-colored,” says Marschner, who presented the team’s findings in July at the 2003 SIGGRAPH computer graphics conference in San Diego. “They’re an important part of the appearance, and they lend a distinctive texture to the hair.”

Marschner’s work on modeling hair, skin, and other materials doesn’t just have applications in the worlds of motion pictures and videogames; there are many other fields in which it’s vital to reproduce reality in exacting detail. Architects need realistic renderings of their designs to show clients; doctors need accurate simulations for training surgeons; online retailers need to show consumers the exact drape and luster of that silk blouse or cashmere sweater. “One of the things that I think is great about computer graphics is that it’s an interdisciplinary area that’s at the boundary between computer science and other things—physics, animation, optics, the dynamics of how things move,” Marschner says. “I get to learn about new things, all the way from science to art, as part of my work.”

Like many of his contemporaries, Marschner can date his fascination with computer graphics and special effects to 1977 and the release of a little movie called Star Wars. As a child in suburban Chicago, he read everything he could get his hands on about how the filmmakers used intricately designed models and motion-controlled cameras to create the movie’s spaceship battles, set “a long time ago in a galaxy far, far away.” Years later, he would make a professional visit to Industrial Light and Magic, the effects house that director George Lucas founded to produce Star Wars; he chuckles when he ponders how he would’ve reacted if, as a young boy enraptured by the inner workings of the Death Star, he’d known that someday he’d be standing in the temple to Luke Skywalker and Darth Vader. “I was completely fascinated by Star Wars, and really excited about it,” recalls Marschner. “I had a video camera and played with trying to make little models myself, although that never turns out quite as well as you expect it to.”

He was also fascinated by computers and programming at an early age; he talked his parents into getting him an Apple Two Plus when he was still in elementary school. He went on to double-major in math and computer science at Brown, where he worked on computer-animated short films. Spending the summer in Ithaca while his then-fiancée was finishing her undergraduate degree, he took a summer job in the Program of Computer Graphics. He eventually joined the graduate program, earning a Ph.D. in 1998 under Greenberg. His thesis was on inverse rendering—using real-world images as the basis for computer graphics algorithms. “Ever since then, my research has been defined by keeping things grounded in science, doing things the right way and backing them up with measurements,” he says. “It’s very important to make sure that what you’re doing is grounded in physics and reality, especially if you’re trying to make physically realistic images—ones that you can intercut with real footage or use in a simulation and be sure they’re really predicting what’s going to happen.”

After research stints at Hewlett Packard, Microsoft, and Stanford, Marschner joined the Cornell computer science faculty in the fall of 2002. He and his wife, a copy editor and graphic designer for scholarly publishers, live in Ithaca’s Belle Sherman neighborhood with their two-and-a-half-year-old daughter, Zoë; their hobbies include hiking and cooking, with a current bent toward Italian, Indian, and Asian cuisine.

“He certainly has great creativity and imagination,” says research project leader Stephen Westin, who runs the light measurement lab where Marschner carries out many of his experiments. “But he’s very thoughtful and careful; he’s meticulous.” Greenberg lauds Marschner for his crackerjack measurement skills, which he calls among the best in the world. “Steve’s both a scientist and a computer scientist, so he understands the need for experimentation and verification and physics and optics,” Greenberg says. “He also has the smarts to be able to translate it into computer algorithms, and that’s a very rare commodity.”

Marschner is presently continuing his research, refining the models for depicting hair and skin realistically and expanding them to include other materials such as cloth. In addition to his research activity, he teaches undergrad and graduate courses as one of the department’s dedicated computer graphics faculty (with Greenberg and assistant professor Kavita Bala). “The recent hiring of Steve and Kavita reflects our belief that graphics has an increasingly important role to play within Computing and Information Science and the larger circle of engineering and science,” says Charles Van Loan, chair of the Department of Computer Science. “Together with Don Greenberg, they have put together a world-class graphics curriculum that permits our wonderfully talented undergraduates to go deep into the subject.”

“He’s really a joy to work with one on one,” says computer science major Andrew Butts ’05, who took Marschner’s undergrad course last year. He received a research grant sponsored by United Technologies through the Learning Initiatives for Future Engineers program in the College of Engineering to work with Marschner over the summer. “It was a really enriching experience,” says Butts. “He always had lots and lots of ideas. It seemed like every time I had a meeting with him, he would leave me with five or six new paths that I had to choose from.”

Butts’s summer project involved writing software to simulate how light interacts with rough surfaces. He ran the simulation on the PCG’s computing cluster and collected and plotted “a really large amount of data,” but didn’t get as far as implementing a production-ready rendering; he’s continuing the research this semester as an independent study. “Especially in computer graphics, his math skills really shine,” Butts says of Marschner. “He pretty much knows all the mathematical theory behind any topic.”

Butts himself came to Cornell to study computer graphics after running across the program’s website while doing research during high school; he hopes to have a career in the field after graduation. How does he feel to be working under one of the people responsible for the striking realism of characters like Gollum? “It boggles my mind,” he says with a laugh. “What you see in The Lord of the Rings is definitely state of the art. You won’t find anything better than that right now. But there were people saying exactly that about the liquid metal guy in Terminator 2 when it first came out. ‘That’s amazing, we’ll have computer-generated humans in no time.’ But now we look back at that and say, ‘We could have the undergrads whip that out in a couple of hours.’”

The motion picture academy’s Edlund—whose own credits as a special effects master include all three Star Wars movies as well as Raiders of the Lost Ark, Poltergeist, Ghostbusters, and Die Hard—notes that as techniques get more sophisticated, so do audiences. “I always hearken back to my severest critic—that is, the ten-year-old who knows what reality looks like when it’s photographed and has seen many movies,” Edlund says. “If something doesn’t look right he says, ‘Hey, that looks funny,’ and you’ve lost him. These technologies that are developed by the supernerds—and I say that lovingly—are part of the creative force behind movie making. So the academy tries to seek out and applaud those who make it possible to make more convincing movies.”

At the Valentine’s Day ceremony at the Ritz Carlton in Pasadena, Marschner and his colleagues received certificates, not Oscar statuettes; clips from the event were shown during the main Academy Awards broadcast on February 29. Still, the honor puts him in the fraternity of Cornell-educated computer graphics gurus—including George Joblove ’76, M.S. ’79; Doug Kay ’76, M.S. ’79; Roy Hall, M.S. ’83; and Rob Cook, M.S. ’82—whom the Academy has honored for contributions to the field. As Van Loan points out, “The combined research strength of the PCG and the CS department makes Cornell one of the top places to study graphics at the graduate level.”

“It’s very gratifying to see the stuff being used, especially in films that I really enjoy,” Marschner says. “With Lord of the Rings, even as a computer graphics person, my attention was essentially focused on the performance of Gollum, rather than on the details of the rendering. Usually I’m always looking at all the little flaws. But that one was done so well that it’s the performance you’re seeing rather than the technology.” Edlund notes that in the case of the tortured Gollum—whose good and bad sides are at war with each other as he leads the Hobbits on their quest to destroy the evil Ring of Power—a computer-generated character was seen to convincingly argue with himself. “To me,” he says, “this was a historic moment in moviemaking.”

Beth Saulnier is a journalist and mystery author who lives in Manhattan. Her next novel, See Isabelle Run, comes out in March 2005 under her married name, Elizabeth Bloom.