Motion capture mediation et al.


I was recently directed by a poster on a message board to this fascinating featurette, which touts the innovative marriage of technology and artistic virtuosity by an actor.  Andy Serkis is already getting Oscar buzz for his performance as Caesar in Rise of the Planet of the Apes.  Caesar is an ape, but instead of those wonky body suits you remember from the original Planet of the Apes, he is rendered with the latest advances in motion capture technology.  The question on the board was whether or not Serkis deserves an Oscar nomination, but I think the question has broader implications than that.  Many films in the last several years have featured performances delivered with the aid of mo-cap technology, the real question, for me, is how these technological advances should (or will) affect the way we think about film acting itself.

Robert Zemeckis’s last three films have used mo-cap; Steven Spielberg’s upcoming Tintin movie uses mo-cap.  The most financially successful motion picture ever made, Avatar, depended upon mo-cap for the performances of the actors playing aliens.  Kristin Thompson devoted a blog entry to the synergy between the physical performance of actors and the final product of motion capture effects, and it was a quote from Avatar’s director, James Cameron, that spurred her commentary:

“I’m not interested in being an animator. . . . That’s what Pixar does. What I do is talk to actors. ‘Here’s a scene. Let’s see what you can come up with,’ and when I walk away at the end of the day, it’s done in my mind. In the actor’s mind, it’s done. There may be a whole team of animators to make sure what we’ve done is preserved, but that’s their problem. Their job is to use the actor’s performance as an absolute template without variance for what comes out the other end.”

Either Thompson was being very kind, or she thought that being overly blunt was not material to her piece, which, in great detail, contradicted and dismantled Cameron’s assertions.  The blunt version is this: Cameron completely misrepresents what the technology did in his film, either out of ignorance or chicanery.  The whole notion that there is absolutely no “variance for what comes out the other end” is just plain wrong, as Thompson ably demonstrates:

The nose and the wrinkles on and above it have been considerably changed. Unlike human noses, those of the Na’vi are smaller at the bottom than at the top, somewhat resembling lions’ noses. The wrinkles seem to be derived from canine or feline faces as well, extending from the inner end of the eye and arcing down toward the tip of the nose. The human frown lines at the lower center of Saldana’s forehead are transformed into larger, longer, curved wrinkles at either side; these start between the eyebrows and move up and to the sides. There they get extended by the curved areas of darker blue that radiate across the upper forehead, so that the lines of anger seem to cover more of the face. I suspect that relatively little of what the actors did with their noses has survived the special-effects processing. (In the image below, even the shape of Saldana’s naso-labial folds has been slightly altered.)

In other words, by virtue of the fact that Saldana is not playing a human character, the facial expressions endemic to the human face cannot be directly replicated on her character’s face without considerable variance.  The gist of the performance is there, but it is not the actual performance.  For all his techno-acumen, James Cameron apparently knows jack about the ins and outs of what his own technology literally does.

Thompson goes on to discuss Andy Serkis’s lauded performance as Gollum, making similar points about the physical dissimilarities between Serkis and his character.  Ultimately, Thompson ponders some options for creating new awards categories for “hybrid” performances like these before concluding thus:

Despite fears that motion-capture may someday make directly photographed actors obsolete, there seems little chance of that happening. These techniques are fantastically expensive and are likely to remain that way for some time to come. There seems little point to using such elaborate technology to make an actor look like a real person when traditional cameras can do it so much more easily, so motion-capture performances seem suitable primarily for fantasy beings who cannot be as believably created in any other way.

This is a reasonable prediction, and I do find it somewhat comforting, but it is not wholly satisfactory.  Thompson even mentions a special award granted to The Curious Case of Benjamin Button, which is nominally a fantasy, but the more spectacular VFX work done to make Brad Pitt age in reverse was not used because he needed to look like an alien.  On the contrary, the digital effects were used to make a human being look unmistakably, plausibly human, even though his human physique is not behaving the way human biology usually does.  And most of the characters in the Zemeckis films are human; the characters in the Tintin movie will be human (or so I expect, at any rate).  And what about Tron: Legacy, in which Jeff Bridges played multiple roles.  In one, he was the genuine article, a human actor being photographed in all his middle-age glory; in another, he was the film’s villain, a CG creation modeled on how Jeff Bridges looked and acted back in 1982.  Bridges even played his 1980s self and his permanently-80s self in a few scenes.  All of these films are technically genre films, as is Rise of the Planet of the Apes.  But film acting is film acting, whatever the genre.  Even in a science fiction action film, performances are judged on their verisimilitude, or at least on how successfully they fit the overall aesthetic of the film.

The way that digital technology is used in a lot of these films is more analogous to makeup or costuming than it is to the kind of total fabrication we typically think of when we think of computer generated effects.  CGI was used to enhance the range of expression on Aaron Eckhart’s face in The Dark Knight, just as it was used to replicate the effects of various ages on Pitt’s face in Benjamin Button.  In both cases, special makeup effects were used, but digital technology was used to enhance those effects.  In essence, I think the way mo-cap is being used in Rise of the Planet of the Apes is an extension of that.  Just as the actors in the 1968 original were buried in layers of the most advanced available makeup effects, my impression is that the filmmakers of this new Apes movie are thinking of the mo-cap the same way: it’s being used to enhance the realism and impact of the actor’s performance.  But that’s not what’s actually happening.

Traditionally, if an actor needed radical makeup effects, those would be applied before shooting a scene.  Actors playing aliens or apes would typically show up for work hours ahead of time, sit in a chair, and have highly uncomfortable goop pasted onto their face while everyone else drank coffee or tinkered with lighting setups.  They would then finally get costumed, find their marks, and start acting for the camera.  In this tradition, the radical makeup effects would be used more or less as props.  The actor would work with the artificial enhancements and build them into the performance to the greatest degree possible.  The linear progression is fairly easy to chart.  First, the makeup people would do their job, then the actor would take it from there.  Fairly straightforward, right?

With motion capture, the progression starts in a similar fashion.  Consider the featurette.  The tech crew will suit up an actor (Serkis) and do their best to make the actor aware of what the ultimate effect will be.  The actor takes his props and goes to work.  Very similar.  But when Serkis’s work is done, the mo-cap VFX crew still has loads and loads of work to do.  Different groups within the FX department work on different components for the performance.  Programmers code and render the visuals that ultimately wind up on screen.  Designers research and craft molds and models for the programmers to use.  And after they’ve done all the preliminary work they can do, the director probably sits down with them and tells them precisely how to tweak their work in order to make it Just Right.  What the actor does is not the final word on the “performance” that the audience will see.  Between the time the actor takes off his mo-cap gear and the time we see the fruits of his labor, months and months of subsequent judgments and interpretations of that performance have occurred.  In short, the actor’s acting has been mediated.

Obviously, on a larger, more conceptual level, every aspect of a film is mediated to a certain extent.  Everything from lighting to sound mixing to editing choices ultimately affects how the audience will interpret a performance.  But it’s very rare that the minutiae of the performance itself are subject to the tinkering of teams of people.  Even if the crew responsible for rendering Caesar the ape treats Andy Serkis’s acting as Holy Scripture, what they do is still hermeneutical.  At best, what they accomplish is a large-scale collaboration to achieve a single performance.  This kind of thing is common in animated films, and it has been true of every performance given so far with the aid of motion capture technology.  Yet why do filmmakers like James Cameron insist that what is obviously true is not the truth at all?  Why is it so important for filmmakers to sell (and the audience to buy) the illusion that what we’re seeing is the “real” performance of a single, human actor?  Why would so many of us collude in denying the truth of how a performance like this is achieved?  Thompson provided a link to an essay by Mark Harris as an addendum to her blog entry, and there is an illuminating quote contained in it:

The fact is, a computer can’t pick up every nuance of an actor’s work, because great performances have nuances that are ineffable and unquantifiable, not to mention vulnerable to eradication with one thoughtless flick of a digital paintbrush. 100 percent? Not even close.

You may remember Harris from that GQ article pinpointing Top Gun as the harbinger of the death of creativity in Hollywood.  (I commented on it in one of my first entries.)  My own prejudices would lead me to agree with Harris in his assessment of the limitations of the “digital paintbrush.”  But my common sense tells me that such a conclusion would be just another overly broad brushstroke in itself.  I really don’t know if computers are or are not capable of capturing the nuances of a great performance.  If they aren’t now, perhaps one day they will be.  What struck me about Harris’s EW essay is the fact that he assumes as much about motion capture tech without really quantifying it.  If you read carefully, you’ll notice that his premise (that real, live acting in the traditional sense is best) is also his conclusion.  He calls the actors in Avatar “puppeteers,” and I thought that was a funny analogy, since I’m fairly certain that professional puppeteers would take offense to the notion that they are not “real” performers.  Harris also neglects to consider how awesomely expressive animation can be, irrespective of its fidelity to the nuances of the human range of physical expression.  There’s a kind of prejudiced elitism at work in Harris’s essay that suggests that actors photographed directly by the camera are the only ones capable of giving great performances.

This prejudice is understandable.  I believe that most people respond most deeply to films that reflect what they know of humanity.  Sure, it’s possible to enjoy spectacle and explosions and whatnot for their own sakes, but the films that uplift us, make us cry, or engender weeks of haunted rumination — those are the ones that we commonly describe as “art” in the generally false dichotomy of “arts & entertainment.”  Harris is right, in one sense: there’s no substitute for the human face.  It is humanity with which we most deeply connect, and when a film or filmmaker tries to pass off an obvious fabrication as indistinguishable from humanity, it rankles us.  Or perhaps I should say that it rankles some of us more than it does others.  Personally, I do fear that today’s generation of moviegoers really can’t tell the difference between what is true and what is not; I mean this in spiritual, moral, and thematic terms, as well as literal and aesthetic.  It’s one of the reasons that I’m so fascinated by the docu-horror subgenre, viral marketing stunts, and the potential of immersive video games.  Like many people, I sense the approach of a singularity, in which the divisions between reality and fiction in our increasingly digitized world become blurred on an almost literal level.  On a basic level, I think that most reasonable people can tell the difference between reality and fiction when they come right down to it.  But I don’t like the idea that people can (and would willingly) mislead us about that division.  James Cameron’s comments seem to be in the deceitful vein.  But Harris’s comments seem to be in a rigid, prescriptive vein.  As long as we know that cinema relies so heavily upon technical artifice in order to illuminate larger truths, why should we restrict the range of expression out of stubborn loyalty to a certain way of doing things?  Isn’t it better to have more ways of approaching Truth than fewer?  Harris may be prejudiced against a more heavily mediated approach to film performance precisely because he fears the autocratic manipulations of egotists like Cameron.  I understand and empathize with that, but I don’t think such a reactionary stance is constructive.

Instead of propping up this prejudice, let me suggest that we simply need to expand the way that we think about film acting.  My suspicion is that as motion capture technology evolves (and it has already evolved quite rapidly), it will challenge actors to develop new ways of practicing their craft, just as new film stocks and digital photography challenged cinematographers.  I don’t think the changes will be all-pervasive, but I think they’ll be noticeable, and that actors like Serkis or Doug Jones, who are willing to experiment with the kinds of roles that “real actors” take on, are already paving the way.  The more that professional actors come into contact with mo-cap and its inevitable descendants, the more they will subtly adapt to the ways that digital effects are blended into areas like makeup and costuming.  I’m sure that this will provoke a kind of laziness and crutch-leaning in some, but I’m equally sure that some actors will embrace the collaborative nature of the new technology and accept the ways in which their performances can be more of a tool in the filmmaker’s toolbox than the front-and-center vehicles for perpetuating stardom they currently are.

Filmmakers and critics also need to expand the way they discuss film acting.  The fundamental aspect of mediation that I outlined earlier is critical.  We shouldn’t neglect or marginalize the important work of actors who specialize in motion capture roles or those who are subject to the effects of alterations and manipulations, a la Benjamin Button.  But the role that the VFX technicians play in the final impact of the “performance” should be equally highlighted.  It isn’t really practical to discuss each and every member of an effects team in the context of a piece of criticism, but it is vital to acknowledge that the performance we see is not simply the result of the actor’s “absolute template” — it is the result of a synergistic collaboration requiring the expertise of dozens (if not hundreds) of artists, all of whom are focused on the singular goal of creating an emotionally, spiritually truthful performance.  To that end, instead of praising the performance of Andy Serkis, perhaps we should praise the performance of Serkis et al.  It’s the responsible thing to do, for two reasons.  One, it reinforces our knowledge that the performance of the human actor is mediated, and we should never confuse what is real with what feels true, just because it is technically manipulated (and manipulative).  Two, it acknowledges in yet another profound way the miracle of a single, incredible effect being achieved by the talent and skill of large numbers of people working together.  Movie-making is one of the grandest metaphors for humanity itself.  Whatever goal we set for ourselves, as long as it is within our means, it can be achieved.  Whether that end is disastrous or resplendent may differ, but in the case of a mediated performance by an actor et al, the glorious expression of the human soul, augmented and affirmed by teams of other souls working in concert, is a wonder to behold.  I have no idea why it works, but as long as I know how it works, I can simply marvel at the things already achieved and the possibilities yet to come. ☕

About mjschneider

Reads. Writes. Watches movies. Occasionally stirs from chair. Holds an advanced degree in heuristic indolence. View all posts by mjschneider

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 47 other followers

%d bloggers like this: