[June 17th, 2008]
Why enter the Uncanny Valley?
Posted by Michael
Over at ZS Forums kradproductions went on a rant after some frustration with Moviestorm. This is a slightly more general attempt to address some of the concerns. While general frustration with tools is something that does not really come as a great surprise he posted a good question when he compared character-driven live movie productions with character-driven machinima productions:
So why do you guys do it? Firstly, I’ll concede that a live-action production of “BloodSpell” isn’t exactly practical with our somewhat-similar budgets.
But what about Overman’s “Apology,” Lit Fuse’s “Jill’s Song,” or Dr. Nemesis’ “BEAST?”
They’re all character-driven stories. They’re all excellent. And they’d all take half of the time to create with actual living actors, and would look twice as good for it. The time and effort invested in the production of each of these films is astounding. I can’t help but wonder if that time and effort would have been better invested in live-action, though.
First of all: here are the links to the mentioned movies: Apology, Jill’s Song, Beast. One could add Monad to the list as well as a number of other machinima pieces. Those are all very good machinima pieces that could have been done with live actors. But would it have been the same “live”? I do not necessarily mean in terms of camera work. Most of the camera work in those movies is not necessarily using the freedom of the virtual set, nobody dollys back heads or through whole forests nor does the camera whiz through closed windows or does any other shebang as seen in Beowulf. Nor do I mean in terms of production. Instead, I mean the performance and expression of the actors.
A fair warning upfront, I do not think that I have found a proper conclusion to the question and the following is more or less an assembly of two points I still struggle with. So this is more work in progress then some final academic stand.
Is there some benefit from using the more wooden, game-like, often standardized characters with default skeletons, limited control mechanisms that try to look kind of realistic in some way? Is it worth to enter the uncanny valley when the performances are so “off” and difficult but obviously want to be taken seriously?
Machinima is an intertextual format that can use the new frame it builds from being a video game-based format to re-frame “content” new. Some time ago we debated that in an earlier thread. To avoid the same issues and to clarify the argument let us condense the question to a simpler one: Why use virtual actors when you could use real ones with less effort in a comparable dramatic scene?
And let us exclude all the practical production-driven reasons such as aging, payment, unions, stunts etc. They are surely important but – I would argue – not the main point at hand. Let’s instead concentrate on the question whether there is something that a virtual actor can tell about “real” drama that a “real” actor could not. The two points I would bring up to answer that would be: machinima has elements of virtual puppetry and of virtual mask play.
Brighter minds have written about the role and value of puppetry and its own level of expression probably should go undisputed. What is interesting is that the same values might just translate into machinima. The connection has been often connected to virtual puppetry folks like Hugh Hancock but also in blogs like Machin-X and our own experiments. The parallels are sometimes thrilling. A mere 200 years ago Kleist mentioned for the use of puppets in dance:
Puppets need the ground only to glance against lightly, like elves, and through this momentary check to renew the swing of their limbs. We humans must have it to rest on, to recover from the effort of the dance. This moment of rest is clearly no part of the dance. The best we can do is make it as inconspicuous as possible…”
My reply was that, no matter how cleverly he might present his paradoxes, he would never make me believe a mechanical puppet can be more graceful than a living human body. He countered this by saying that, where grace is concerned, it is impossible for man to come anywhere near a puppet. Only a god can equal inanimate matter in this respect. This is the point where the two ends of the circular world meet.
The same rings true for many good machinima performances. But what about facial expressions and simpler body movements? kradproductions despaired over a relatively simple scene set up and it is only to true: the simplest animations can be mindblowingly difficult to implement in game engines. Machinima makes it much more complicated then it ever would be during a real shoot to get your actor to take a cigarette out of a box, light it and hand it over to his buddy.
You have to be ridiculously good if you want to control it in real-time in Unreal. Even if you follow traditional animation pipelines is a challenge. A Maya queen or a Poser wizard can do it but it is so much more difficult, slower, and more complicated compared to a Robert Rodriguez directing a handful of friends in El Mariacchi and shooting from the hip. And even once completed, it never reaches the same level of detail and realism we have in live action films.
However, it is still worth the extra effort because within this unreal image range lurk possibilities that are not available in the “real.” Enter second point: machinima and mask play.
The virtual faces of machinima actors are projection planes and as such they have their very own qualities precisely because they are “off.” Whether it is the 2D shader in Beast or the good but still imperfect lighting in Monad. Both are important features in the virtual actors’ performances. They are not realistic or perfect and still add to the personalities and the films’ overall values. Even the aforementioned Beowulf is obviously not completely photorealistic and the faces and animations remain unreal – and that’s quite alright. The lack of detail in an avatar’s face can be the void we fill when looking at them and projecting our imagination onto them.
That’s how Eva and Franco Mattes’ 13 Most Beautiful Avatars project works. Behind the ueber-perfect and flawless faces made of Second Life polygons one starts to envision the person who “is behind” this avatar.
I have been wondering for some time whether we put on a mask when we “become Gordon Freeman.” It is a question of identity and representation. Everybody who has ever played theater with masks has probably experienced the strange power of the mask as it changes the behavior of the player. Like the interaction design in Half Life 2, a mask shifts us into a performance and certain behavior. When I am wearing an elephant mask I will walk differently then when wearing a lion mask. And compared to Freeman, masks have even more limited facial animation. Yet, they are immensely powerful tools in theater.
There is a complex connection between a dramatic role and a mask, which is often seen as a “transitional object.” That might be the reason why there are parallels in the transition as it works in video games and machinima production compared to masks in Commedia dell’arte or other mask plays like Japanese Noh theater. There are also parallels in the limitations that masks and games provide, as Mika Tuomola outlined:
CdA [=Commedia dell'arte] was much like a chess game. It created interesting, dramatic situations not because the “game” was written beforehand but because the rules of representation were predefined and clear. Each character, like each chess piece, could only do certain things. They could only use certain masks, mimics, passages and properties.
Murray talks about the pleasure of transformation if one enters a virtual world and it seems that some designer are aware of such a transformation through characters. Many early avatar designs seem to be inspired by masks and standardized interchangeable parts, much like the stock costumes in mask-theater productions. The avatar costumization in Second Life might be just the non-dramatic opposite to that.
But masks do not always work. Henry Lowood pointed me to Halocaust which did not work at all for me. It puts masks (Master Chief’s helmet to be precise) on the faces of the Jews to deliver a machinima film about the Holocaust. It is not a problem of the medium – one can make machinima films about the Shoah – but it is a problem of the way in which the masks work.
Among the countless horrible things that happened during the Holocaust one was to de-humanize the individual Jewish person, make them a number or even less. In using the simple Master Chief mask, Halocaust does that again. If it is applied as a critique this critique is not clear enough because even before the killing is mentioned, the Jews are presented in a simplified sandbox world of Halo. Here, the mask play does not improve the expression but reduces and unifies it without realizing that it does that. That the same robot uniformity can be used effectively to portray individuals in a de-humanized mass was one reason why bot is such a good film.
Overall, virtual puppetry and mask play might be just two good reasons why one should enter the uncanny valley of machinima production. Both offer special features that can enhance or threaten the movie at hand. Both allow the depiction of “real drama” in a new way and thanks to their digital nature they open up new options that indeed are not available to Rodriguez.