NEO REALISM:  Virtual cinematography helped create this scene in which Neo lashes out at mulitple Agent Smiths.  Photo courtesy of Warner Brothers USA.

How quickly the future becomes passé.

Take bullet time, the effect used to depict Neo’s mind-over-Matrix capabilities in the 1999 original. That oft-imitated, never-duplicated technique for whooshing an impossible camera around an impossible stunt just wasn’t slick enough for Reloaded. This time, the Wachowski brothers didn’t want Neo to fight a single adversary—they expected him to take on a hundred.

Bullet time’s waterloo was dubbed the Burly Brawl. It begins as a rematch between Keanu Reeves’s Neo and Hugo Weaving’s Agent Smith, but when Smith falters, he quickly multiplies to 8, then 24, and finally 100, while the camera spins about the combatants at supersonic speed. Visual-effects supervisor John Gaeta soon realized that using the original bullet-time process—which employed 120 Nikon still cameras firing in sequence around the action, lots of digital interpolation to simulate the missing frames, plus a digital re-creation of the environment—was too “labor intensive” and “anal retentive” for Reloaded. What the Burly Brawl demanded was the first real-world application of virtual reality. “We had all these characters doing layered, complex, and absolutely impossible choreography, which immediately pointed to virtual humans,” Gaeta says.

To create them, Gaeta and Kim Libreri, the visual-effects supervisor at ESC, built a gigantic motion capture stage, then let stunt choreographer Yuen Wo-ping go wild, pitting Reeves against Weaving plus 8 to 11 stuntmen. The massive amounts of motion capture data recorded were then edited and multiplied. “We created this ultimate clip and paste library,” Gaeta says, “then assembled the bits into layers of action with Neo as the epicenter.”

The computer-generated bodies were fleshed out with digital skeletons and musculature, then cloaked in photographic elements extrapolated from the actors’ wardrobe and CG cloth simulations. Next, Libreri’s team arrayed five Sony 900 HD cameras in a semicircle around each actor’s face to capture every grimace from every angle, then mapped those real expressions onto the characters’ CG doppelgängers. “We didn’t want to pretend we could animate a performance that would be as good as the actual actors’,” Gaeta says.

“People speak about it like it’s the end of the actor,” Reeves says. “But they’re not scanning someone else to be Neo, they’re scanning me. So you’re still acting.”

Once these CG characters were placed into a digital re-creation of the environment, ESC’s artists could circle a virtual camera around the virtual action at supersonic speed and with incredible precision, in ways that never could have been achieved with bullet time. A fist impacts a face and the virtual camera is there, then the point of view whips impossibly to capture a body whizzing headlong from the fray. “Once we’ve created these master fight templates,” Gaeta says, “we can make complementary camera moves so we’re beautifully in sync with the action.”

But does the virtual cinematography portend the rise of a Matrix-like virtual reality? “It’s 100 percent sourced from the real and driven by the real,” Gaeta says enigmatically. “It’s the first time you’re going to see what virtual reality looks like.”

Will the Wachowski brothers be able to top themsleves in The Matrix Reloaded? Talk about in our forums!

You want more Matrix?  No problem.