Christmas and New Year

ACM Multimedia 2004

Presentation at Columbia University of the paper "Story Reaction Structures to Emotion Detection" in the 1st Workshop "Story Representation, Mechanism, and Context", ACM MULTIMEDIA . You can take a look at the paper and presentation slides at my homepage.

The workshop was very good, not too much people which gave it a familiar touch :). A lot of discussions on story aesthetics and representation, Kevin purposed some funny and thoughtful story exercises. We've seen papers about story focusing on almost every area from philosophy, psychology, linguistics, computer science, film and theater.

Well, here I come again with my doubts and new definitions and new focus and new paths.

Something sort of new today, we'll sort of let down the buzzword virtual reality and substitute it only for videogames.

We were talking yesterday about the fact that the convergence is being approached through film entertainment area and not the art, alternative or independent field. So why do we want to look at Virtual Reality art installations? That makes no sense. In order to follow a scientific systematization we need to develop a multidisciplinary work but we also need to develop convergences between convergent fields, fields with some common inside. And that common inside is also the common goal of the overall work, and that is nothing more that the developing of tools that will be able to deliver an Entertainment Experience, emotionally and significantly richer.

Because of that our area is now more clearly defined and we can say it without shame that we?re really developing academic work in the area of games. We chose for now to call them VR Games instead of video or computer games. Because we'll not define aesthetics in Tetris or Pac-man but we'll work only with games that we'll be able to proportionate a virtual experience in the sense of the world depth being displayed tridimensionally. We are here also talking about a truly interactive story experience in spatial and temporal environment that awaits the user actions to reveal and not a multi-linear bag of cinematic sequences that asks players what story ending they want.

So we're trying to develop systematization or taxonomy of emotions in interactive storytelling through film and vr games, defining aesthetics convergences.

Love Robot
From Schopenhauer 'Metaphysics of Love' to Bjork/Cunningham 'All Is Full of Love'

Machines will never have real emotions because they don't need them.

Love doesn't make any sense in a machine because we only have this feeling to feed the passion and attraction necessary to provoke reproduction of our species (Schopenhauer).

So machines doesn't need to play sex to reproduce they will generate themselves in assemblage lines.

In this way what we need from robots is not a development of structures to generate emotions in robots we don't need them to be feeling robots.

They will need to be able to generate representations of human emotions - expressing emotions - and also able to recognize human emotions and having algorithms able to response to humans.
Robots will need this only in communication between robot-human not robot-robot.

What is ART?
 
Three theories or perspectives to look at art.

a)      emotivist theory - artistic expression of emotion, artist-based emotional theory of art
b)      sentimentalist theory - emotional responses of the viewer/listener, audience-based emotional theory of art
c)      rationalist theory - rational communication + rational production

Using the three perspectives we can try to look at art as:

Something created intentionally, being able to communicate anything that will elicit a determined set of emotions in the receiver
 

In the beginning I was looking for the convergence between film and virtual reality. I had a path to follow, to do a profound film study of 3-4 movies related to virtual reality, analysing the future consequences of VR in reality, in life, society, film - in the end a cultural analysis.
 
Then I turned my interests to storytelling - narratives, the art of storytelling, stereotypes of narrative - more versed in art theories.
 
Then I looked at emotions, the way to build emotions in a videogame through storytelling - psychology is over my head. 

In some way i've returned back to my prior quests before the PhD - Why is interactive content communication (like virtual reality) not as emotional as film?
 
Don't know where I'll be in the next couple of months. I feel that I need to focus, and above all to find a strong link between these works already done. I'll need to bang my head in looking for answers. Where is the goal, what is my final target, what am I trying to prove that nobody else have already tried?
 
Just know that I want something that mixes emotions, film, VR and storytelling. I also know that there are some people out there working in the same area and that probably already found some solutions. Maybe that I need to focus a bit more in a sub-area of all this 4 mixed areas... at the same time I'm afraid to go deeper because every time I've to write something (papers) on the area, I've to justify and explain everything in the beginning because not everybody is aware of the state-of-the-art in the 4 areas.
 
So, that's where I'm now. I'm working in the building of a straight line for this multidisciplinary PhD. Hope to find some answers till the end of the summer, in the between I'll have to finish the chapter on film study.  

I, Robot (2004)

I, Robot by Alex Proyas

«Asimov... making machines an example of how the world could be bettered through the mastery of technology. ..The robot was artificial intelligence in a man's shape, a foil for asking what it means to be human and what rules should govern us.
...
The idea that we can take our social interactions and code them with an Asimovian algorithm ("allow no harm, obey all orders, protect yourself") is at odds with the messy, unpredictable world.
...
This need for people to behave in a predictable, rational, measurable way recalls Mr. Spock's autistic inability to understand human emotion without counting dimples to discern happiness or frown lines to identify sorrow.»

in http://www.wired.com/wired/archive/12.07/machines_pr.html

Today is a happy day :)

Yesterday I've won the prize for the best paper presented at the conference

Games2004 - Digital Entertainment and Interactive Games, Faculdade de Ciências da Universidade de Lisboa, 12 July

I presented the paper "Emotion and Suspense in the Interactive Storytelling".

I argued in this paper that we can develop suspense in both cinema and interactive storytelling but the emotional result from suspense diverge completely. Then I presented some new perspectives to allow some convergence in the emotional experiences in order to raise the amount of significance of the experience to the user.

I'll post soon the slides from the conference at my homepage.

T-Shirt TV Screen

"A model at Loews Theatres on Boston Common invites onlookers to watch a movie trailer on the flat-screen TV embedded in her T-shirt."

http://www.csmonitor.com/2004/0708/p11s01-wmgn.htm

ICO new analysis paths



Myst + D. Quixote + Conan, Boy in the Future + ...

CONAN, the boy in the Future by Hayao Miyazaki (1978)

It's amazing how Miyazaki has been present since my infancy in my life. I had 4 years when he made this fabulous TV series. I saw it only when I was about 8 and since then i never forgot it.

Conan produced and huge memory mark in me. In the sunny days when I'm relaxed and have time to look at my infancy it's hard to avoid the Conan feelings. Visuals, music, voices, character movement, the apocalyptic scenario, Conan happiness with the little things, the loneliness.

Today when i look at a movie like "Spirited Away" i just feel this as to come from the same genius mind.

I came to the conclusion that "ICO" in some way, used some of these artistic atmospheres from Conan and that's maybe one of the reasons for ICO having touched me so profoundly.

Sex, Technology, Storytelling and Emotions

Today "Wired for Sex" [1] was about the new tech sex toys for the elicitation of human orgasms, male or female. Products are getting better and better in touch/sense simulation, but the interviewed people were not yet satisfied. They said they could get orgasms but they don't get their sex needs, completely fulfilled. They didn't come to feel the intensity they await from a normal sex relation.

The real problem I'm seeing for technological sex is the same I see for digital storytelling. Human Emotions.

Everybody that has experienced sex with a stranger knows that the only thing he gets from that act is the orgasm deprived of feelings.

If you don't care about the other being you'll never get the top climax that a sex relation can provide.

In sex you need to feel that there's another being there. Besides the one and only "love" feeling that is so vast and difficult to define we can talk about the self-esteem. To get in a sex relation you need the sexual organs from your opposite [2], be it technological or in flesh but to feel it intense you need complicity, respect, admiration, etc. from the other being.

Feelings emanated from the other being will intensify the mechanical act, will elicit brain activity of self-esteem that bond with the act in itself will finally explode in a powerful outcome.

In a story is the same thing. If you don't come to care, feel angry, sorrow or happiness about the characters in the story, you'll never feel the emotional intensity and so the feelings of the story will come to be boredom and a waste of time.

I believe that sex technology could be a very good framework to show the peculiarity of human sensations, the human need of the other. Relations.

So if we want to provide anything technological that can someday elicit emotional intensity we'll need to get to emotional machines, which in some way can reach a point of emotion simulation, illusionary enough to fool our mind.

[1] TV Docummentary: Episode 4 - VIRTUAL REALITY SEX
[2] hetro relation

"ICO" (2001) by Fumito Ueda

ICO is the first game with the ability to reproduce an area of fully diverse and undefined emotions in the user. The diversity could have been more intense if the narrative experience had a shorter duration, some story elements were lost while we're traversing the enormous castle (puzzle/maze). We need ten hours to finish the game, and between the moment we begin and the moment we come to closure some intensities are lost in time. I justify these losses by the pressings the authors must have felt in having to extend the game in order to fulfil player's desire for infinity playing.

I believe shadow monsters were almost unnecessary. In the most parts of the game they serve only the purpose to raise tension and anxiety in the player, they don't have a real goal, at least that justifies the intense fights in some parts of the game. Also through the game the relation between Yorda and Ico is developed but it would have been much stronger and much emotional if we had some facial close ups of Ico and Yorda. Anyway the fact of having two characters, one controlled by AI and the other by the user is amazing. We really come to a point when we feel responsible for Yorda and where we feel real empathy for Ico. The hands connection, emphasized through vibration of the game pad, is pure emotion design innovation.

The last hour of the game is truly poignant, tension and anxiety is raised in a balanced way taking us to a final cut scene that we receive as a delicious gift. This final cutscene has no typical "victory dynamics" euphoria but it's filled with a truly story reward full of "diverse and undefined emotions". When Yorda comes to get us, our emotions are completely diverse: happiness, sadness, surprise, adoration, kindness. We feel an enormous tenderness for both characters. The epilogue is plain magnificent, oneiric and very positive.

Think about Myst and add the universal theme "boy meets girl", relationships are the key for game emotions.


UPDATE
In 2005 I've made an extensive analysis of ICO and comparison with Myst for an european research project. You can read it here.

In 2009 I've published a book on interactive emotions where I've extended the analysis of Ico. The book was written in portuguese and is accessible free online, "Emoções Interactivas. Do cinema para os Videojogos", pages 244-253)

A.I. by Steven Spielberg

As an emotion thought-provoking movie we can analyse A.I. by Steven Spielberg as a true guide for everyone interested in emotion and computers.

A.I. makes us believe that the final frontier really separating us from "meca" stands in EMOTION. We can even evolve this to cloning arguments and assume the same about clones versus originals.

Emotion is the real and only identification of human individualism.

About the end of the movie, I’ve now thought about two different depicted situations:

a) We can see the final characters as aliens from outer space. Explorers in search of new forms of life, historians of the universe. The planet Earth is showed as a dead planet without anything alive. And these characters could be only reconstructing our planet history through DNA. David was the only delegate of our existence, and the unique persistent specimen able to unveil the secret of how a human mind worked in the past. In the end we were not alone; we weren’t unique in the universe and neither the most important thing alive in the universe but only a part of it, a race that have been extinct like all the animals that are now disappearing from the face of the earth.


b) The characters as the third generation of 'meca', 'living' alone on planet earth. No humans left. Human bodies were suppressed from the earth and only the minds transferred to 'meca' could have survived. Mind without organic bodies, the total liberation of mind could survive and became immortal. In the end, the 'meca' minds could be seen as the 'organa' minds. We can even think that our minds without our body could become peaceful, comprehensive, intelligent, sublime, supra-human as the depicted characters. Therefore our body is portrayed as the one to eliminate, our enemy, the great obstacle to human evolution. Against (a) we can see 'meca' as our unconscious development in order to avoid, even as mind only, the human race extinction nothing more than the most important biological function of species.

In (b) perspective we have a paradox, if we follow the state of the art of emotions studies. Emotions can’t exist in humans without body because emotions aren’t cerebral but bodily dependent.

This leads us to think that in the future AI must develop to a point in were it can be doable to create 'representations' of human body emotions and inject them in some container/memory inside a 'meca'.

But even if we do that, how can a 'meca' elaborate connections between representations o human body defences of life (emotions) and an artificial mind that doesn't need the body to survive or avoid extinction?

And now I arrived at my first thought about all this that in a way is in consonance with Schopenhauer philosophy.

Without death at the end of the lifeline, why exist?

If your tired of refining your investigation searches in google.com, try scirus.com.

http://www.scirus.com

because it enables you to quickly:

• Pinpoint scientific, scholarly, technical and medical data on the Web.
• Find the latest reports, peer-reviewed articles and journals that other search engines miss.
• Offer unique functionalities designed for scientists and researchers.

Artificial Emotion
Sherry Turkle interview to Boston Globe

She says, we become attached to sophisticated machines not for their smarts but their emotional reach.

"They seduce us by asking for human nurturance, not intelligence," she says.

"We're suckers not for realism but for relationships."

Kids, she has found, define aliveness in terms of emotion: "In meeting objects as simple as Furbys, their conversations of aliveness have to do with if a computer loves them and if they love the computer."

The Love Machine

the 3 distinct areas to go when delving into emotion and computers

. emotion perception
. emotion interpretation
. emotion expression

Picard (1997)

Today arrived

Nico Frijda, The Emotions, 1986

John LeDoux, The Emotional Brain, 1998

I’ll present a paper about emotion and virtual illusion, next week, day 21 of April at University of Covilhã, Portugal.

"Emotion Elements in Virtual Interactive Entertainment"

abstract: The convergence of narrative forms and directing/editing techniques between videogames and film are becoming a reality. Although the emotion intensity created by film is far superior to videogames. This paper presents a specification of emotion elements in interactive entertainment captured through neuroscience, cognitive science and film theory areas of knowledge...

it will be presented in the conference LUSOCOM 2004 (http://www.lusocom2004.ubi.pt)

John Barlow about cyberspace

«. diversity is as essential to healthy community as it is to healthy ecosystems... I believe that the principal reason for the almost universal failure of the intentional communities ... is lack of diversity in their members.
It's rare a commune with people fundamentally out of philosophical agreement with the majority.»

«Like Frankenstein... is the usual problem when we try to build something which can only be grown.»

«. Cyberspace missing elements
a) absence of alternatives
b) sense of genuine adversity, generally shared» [we don't have this at movies also]

The Philosophy of Horror, by Noel Carroll

Image and Mind : Film, Philosophy and Cognitive Science, by Gregory Currie

Suspense: Conceptualizations, Theoretical Analyses, and Empirical Explorations (Lea's Communication Series) eds. Peter Vorderer

Affective Interactions: Toward a New Generation of Computer Interfaces (Lecture Notes in Computer Science, 1814.) eds. Ana Paiva

The Nature of Emotion: Fundamental Questions (Series in Affective Science)
eds. Paul Ekman, Richard J. Davidson, Davidson Ekman

MelbourneDAC, the 5th International Digital Arts and Culture Conference.
May 19 - 23, 2003

Ph. D. Panel Session

The main advice from the experienced Doctorate Candidates:

. don't mess up your life just before you submit the thesis. (keep your boyfriend or husband around until after you have submitted.)
. write about something you like.
. spend time on preparing your work.
. focus on theory - whether you want to use it or develop it.
. write, write, write, present and publish.
. get a good supervisor.