Dec 06, 2005
Ana Paiva, Rui Prada, Ricardo Chaves, Marco Vala, Adrian Bullock, Gerd Andersson, and Kristina Hook, ICMI’03, November 5–7, 2003, Vancouver, British Columbia, Canada.
In this paper, we describe a way of controlling the emotional states of a synthetic character in a game (FantasyA) through a tangible interface named SenToy. SenToy is a doll with sensors in the arms, legs and body, allowing the user to influence the emotions of her character in the game. The user performs gestures and movements with SenToy, which are picked up by the sensors and interpreted according to a scheme found through an intial Wizard of Oz study. Different gestures are used to express each of the following emotions: anger, fear, happiness, surprise, sadness and gloating. Depending on the expressed emotion, the synthetic character in FantasyA will, in turn, perform different actions. The evaluation of SenToy acting as the interface to the computer game FantasyA has shown that users were able to express most of the desired emotions to influence the synthetic characters, and that overall, players, especially children, really liked the doll as an interface."
Read full paper:
Oct 21, 2005
Having a sense of well-being or euphoria while at the computer
Inability to stop the activity
Craving more and more time at the computer
Neglect of family and friends
Feeling empty, depressed, irritable when not at the computer
Lying to employers and family about activities
Problems with school or job
Carpal tunnel syndrome
Eating irregularities, such as skipping meals
Failure to attend to personal hygiene
Sleep disturbances, change in sleep pattern
(According to Maressa Hecht Orack, Ph.D.)
Sep 14, 2005
Sep 01, 2005
Via Siggraph 2005 Emerging Technologies web-site
Color-Enhanced Emotion is a system that recognizes facial expressions and controls skin-pigment components using a real-time processor to enhance them.
Developed by Toshiya Nakaguchi and coll. from Chiba University, the installation uses computer vision techniques to recognize feelings expressed in facial images, and then implements a hardware-accelerated real-time processing system to control the pigment components of the skin by replicating a broad range of conditions with color enhancements: fair, suntanned, pale, red-faced, etc. Registration cameras decompose the surface reflection of the face to enhance it with the colors associated with commonly observed emotions.
According to Nakaguchi and his colleagues, the Color-Enhanced Emotion project may impulse the development of a new communication paradigm and a lead to the creation of new generation of video-editing technology.
Aug 05, 2005
As designers and social innovators, should we take any notice of technology policy? Wouldn't it be best to ignore the think-tanks and telcos, and concentrate on doing great projects in the real world? A 90% focus on projects would probably be healthy. But we also need to keep half an eye on policy making because that's where priorities for research spending - and hence the projects we are able to do - are made.
Tech policy is not a pretty picture right now. After a few years in which social issues were visible on the agenda, tech-push is fighting back. In the European Union, for example, the Information Society Technologies (IST) programme contains a lot of tech but not much soc. The IST's aim is to 'master technology and its applications, and help strengthen industrial competitiveness'. Documents mention the need to 'address the main European societal challenges' - but the advisory group that interprets that statement, ISTAG, consists wholly of Big Tech and Big Research interests. (To compound the imbalance, ISTAG comprises 29 men and just four women). There once existed a panel of High-Level Socio-Economic Experts but they quietly disappeared in 2003, supplanted by an entity called eEurope. The main job of eEurope is to 'develop modern public services and a dynamic environment for e-business through widespread availability of broadband access at competitive prices and a secure information infrastructure'. Once again: a lot about tech and not much about soc. >>MORE
Jul 25, 2005
The EU-funded project Myself focuses on the enrichment of the simulations and the affective computing methodologies traditionally carried out in didactic classroom.
The project's idea is to integrate affective computing into virtual tutors to enhance distance learning and training applications. Affective computing is aimed at giving computers skills of emotional intelligence, including the ability to recognize and express emotions, and to respond to them effectively.
Started in Semptember 2004 and running 24 months, the project gathers industries and research performers from seven European countries (Italy, Spain, United Kingdom, Estonia, Switzerland, Poland, Holland). The project's coordinator is ACSE, an Italian company that supplies information technology services.
We have interviewed Dr. Fabrizia Mantovani, psychologist, who is working on the research workpackage on Affective Computing for the Center for Studies in Communication Sciences of University Milano-Bicocca, headed by Prof. Luigi Anolli.
PTJ - Dr. Mantovani, what are the potential applications of Myself project?
The target application of the project was the development of an e-learning platform endowed with affective computing capabilities (recognition of, and adaptation to, the emotional and motivational states of the users during the learning path). The second objective is development of interactive simulations for the training of communication and emotional skills in professional relationships. In this project, two main target-end users are finance and banking and healthcare professionals.
PTJ - Interesting. But how this approach can be used in healthcare?
The idea is to use 3D-agents based simulations to train physicians and caregivers on how to improve the relationships with patients and their families. Traditionally, soft-skill training is mainly carried out in classroom and face-to-face settings. In Myself, the goal is to use experiential e-learning training to support the acquisition of complex communication and procedural skills (i.e. how to deliver bad health news to the patient, enhance compliance, to be in trust in the relationship etc.), which are useful to improve the relationship between doctors and patients.
More to explore
Affective Computing Portal - a web site devoted to affective computing research
Affective Computing Lab at MIT Medialab - web site of the lab headed by Rosalind Picard
Virtual Reality Lab (VRlab) at the Swiss Federal Institute of Technology - the leading lab for virtual humans research
Jul 07, 2005
Via the Presence Listserv
Mercy Therapeutic Radiology Associates use manmade skyscape to provide a comfortable ambience during radiation therapy. The images help patients relax and lie still during treatments.
The manmade skyscape, called a SkyCeiling, is produced by Fairfield-based company called the Sky Factory. The circular SkyCeiling features a pale sliver of a moon hanging against a deep blue sky, tree branches speckled with bright green leaves and gauzy white clouds.
Mercy Therapeutic Radiology has three SkyCeilings, the first installed in late 2003. The hospital also invested in three virtual windows made by the Sky Factory, with landscapes that use a window as the point of reference.
Each virtual window costs $2,000 to $3,000. The ceiling murals cost from $3,000 to $13,000.
Jan 10, 2005
Experience Design And the Design of Experience
by Erik Davis
This piece appeard in Arcadia: Writings on Theology and Technology (Australia, 2001)
There is no creation ex nihilo. We always work from pre-existing material, both literal substances (wood, a language, the resonance of strings and reeds) and the existing cultural organization of those materials within history, tradition, and contemporary networks of influence. So as we survey the expanding and converging landscape of electronic, virtual, and immersive production, we might ask ourselves: what material is being worked here? Is it simply new organizations of photons, sound waves, and haptic cues? Or does the "holistic" fusion of different media and the construction of more immersive technologies actually suggest another, perhaps more fundamental material?
I’d wager that the new material is indeed rather fundamental: human experience itself. Of course, "human experience" is a vague and historically loaded concept, and a thorough hashing out of the term would require, at the very least, lengthy excursions into Jamesian pragmatism, psychobiology, and Buddhist phenomenology. But for the moment let’s just think of human experience as the phenomenal unfolding of awareness in real time, a movement which tugs against the network of concepts and significations while tending toward the condition of more direct sensation or intuitive perception. In other words, experience may not be able to escape the prisonhouse of language, but it willingly sticks its nose out the barred window and inhales.
Many semiotic and structuralist arguments suggest that we are creatures of language, that nothing, either sensations or intuitions, escapes the domain of signs. But one can just as easily argue that everything that arises in consciousness is experience — that memory, analysis, and reflection all arise in the phenomenal stream, the loops and twists, of James’ "stream of consciousness." As a compromise between these two positions, imagine that you possess an analog Consciousness slider that runs from the nearly totally linguistic on one side to the pure intensities of sensation on the other. While avoiding a strict divide, your thought gizmo allows you to make a gradual though clear distinction between meaning and sensation.
Take the example of a rollercoaster. Obviously roller-coasters exist within a network of symbolic associations. But the act of subjectively submitting your bodymind to a rollercoaster ride, and undergoing the resulting thrills of adrenaline, fear, and gut-fluttering sensation, cannot be directly assimilated to the network of significations that constitute the meaning of rollercoasters. The same point can be made about recreational drugs. Obviously the drug experience is mediated by culture, by expectations, rituals, and social stories about the meaning and value of certain compounds. But we are sticking our heads in the sand if we insist that the evident physiological changes induced by drugs do not correspond with real psychological changes, not only in the content of subjective experience (its images, pleasure fluxes and meanings), but in the more fundamental cognitive parameters which structure experience in the first place.
At the same time, these non-semiotic elements of experience can also function as passageways to new regimes of signs. Fans of psychedelics often find themselves plunging into incoherent or abstract deformations of perception and sensation, only to "break through" into very strange, but nonetheless solid and coherent, worlds of meaning. Immersive works of art or entertainment are also rarely content to simply produce a new range of sensations. Instead, they often function as portals into "other worlds." Following a similar development, the rollercoaster grows into themed adventure rides like Universal Studio’s Back to the Future attraction. These other worlds, of course, are composed of the same sorts of signs that make up our shared human construct — indeed, they often detach those signs from conventional reality, recombining or morphing them within the more malleable zone of the virtual. But in order to successfully boot up these new semiotic universes within a users’ consciousness, the media technology must directly engage the machinery of human perception (proprioception, 3d audio, etc) on a "subliminal" level. In other words, immersive worlds are constructed on a platform of rejiggered experience.
And so we enter the era of what I’m calling Experience Design. A quick scan of our sociocultural landscape suggests that, in terms of artistic practices, mass entertainment, sports, and emerging technologies of pleasure, productive forces are increasingly targeting experience itself — that evanescent flux of sensation and perception that is, in some sense, all we have and all we are.
Let’s begin with the rise of the so-called "experience economy." On one level, this describes an apparent shift within the consumption patters of the younger, more technologically savvy elite, a shift away from the hoarding of material goods and status symbols to the hoarding of novel, exciting, and challenging experiences. (Dennis Tito’s 20 million-dollar space holiday on MIR is the paragon here). The experience economy of the super-rich also dovetails with broader cultural trends, including the dramatic intensification of tourism over the last few decades — a process which offers us increasingly specialized, adventurous, and exotic packages (guzzling ayahuasca with Peruvian shamans, caving in Belize, visiting real live monks in Bhutan). We have also seen a heightened interest in technologically-mediated outdoor activities like rock climbing or wind surfing, along with the rise of "extreme sports," which have little to do with sports as contest and much to do with the production of subjective intensity. The extreme example here is the Bungie jump, which requires neither skill nor exertion beyond the passive willingness to undergo the death-defying neurotransmitter and adrenaline rush that hits the nervous system.
The turn towards increasingly raw experience also marks a number of developments within media and entertainment, including the often-remarked descent to the lowest common denominator of sex, violence and the gross-out stunts of MTV’s Jackass ("Don’t try this at home!"). Over the last ten or fifteen years we have also seen the rise of a new kind of film, one which features amazing special effects, but which otherwise sucks. Whether or not we judge such films to be good, or even worthwhile, depends on how much we accept the new regime of special effects as a semi-autonomous component of cinema whose art is largely devoted to stimulating immediate sensations and visceral — rather than symbolic or narrative — emotion. A similar logic comes to the fore in many computer games and mass applications of virtual reality technologies in amusement parks and arcades, all of which strive for the quality of "immersion" — which is often just another word for simulated experience. Meanwhile, the language of "experience" has become thoroughly integrated into multimedia design, even in the relatively low-bandwidth tricks and offerings that commercial websites use to capture sticky eyeballs.
In the musical sphere, we can see a similar shift in the rise of raves, where intense lights, sounds, and projection screens combine to create a visceral, often collective experience of intensity and atmosphere. These effects (and affects) differ markedly from the more traditional identifications available through even the most Dionysian rock concerts. Another crucial ingredient to the rave experience is drugs. As Simon Reynolds has argued, many elements of electronic dance music, including non-sonic elements like light sticks and Vicks Vap-o-rub inhalers, emerged because of the particular effects they produce in a suitably tweaked mind. In fact, psychoactive drugs are in some ways the ultimate "technology of experience," and establish a basic model for Experience Design. The rapid advances in psycho-pharmacology, both corporate and underground, have given rise to a flood of consciousness-modifying substances which promise to both suppress unwanted dimensions of human experience (depression, anxiety) and open up novel spaces of perceptual and cognitive effects to immediate exploration.
Obviously, a startlingly broad range of phenomena can be placed under the umbrella of Experience Design, and such breadth is often suspicious. But despite a number of crucial problems that are overlooked in this acute generalization, it seems crucial to recognize and emphasize the continuity, rather than the divergence, between contemporary practices that target the human sensorium. Across the fields of art, architecture, media, music, pharmacology, even spirituality, we are moving towards the intentional and multi-dimensional stimulation and production of a complex range of increasingly immediate human responses, including the direct induction of classic "altered states of consciousness." These responses extend far beyond (and below) the traditional object of communication: the conscious human subject conceived as a rational agent and a reader of meanings. In other words, as science, pharmacology, and media technology deepen their understanding of how the human nervous system joins with the ever mercurial psyche to produce a lived sense of reality, these knowledges are becoming integrated into the engines of cultural production. To put it crudely, our cultural technologies are becoming less like books and songs, and more like rollercoasters or drugs.
Media art, with its connections to both alternative culture and critical theory, is curiously arrayed when it comes to Experience Design. On the one hand, its theoretical savvy frees it from the traditional notion of the subject as an autonomous agent of meaning, while also increasing the willingness to play with the construction of subjectivity. Yet many artists, writers and theorists remain queasy about the more technoscientific knowledges and practices involved in understanding and producing human subjectivity within the increasingly technical domain of psychology. This reflexively critical response to technoscientific discourses is mirrored in a similarly pervasive set of responses to spiritual or ritual discourses, which also fundamentally engage experience and the production of altered states of consciousness. While artists are becoming more overtly concerned with bioscience and spirituality alike, both of these flinches continue to result in art which stresses critical distance over the direct mobilization of effects and novel zones of becoming. This distance is valuable, but insufficient. However legitimate, skepticism and distrust of hard psychology, neural science, and psychobiology should not cut the (post)humanistic world off from direct engagement with the proliferating technologies of subjectivity.
For one thing, many other sectors of society are perfectly happy to employ these same tools to far more chilling ends. Advertising and marketing are only the most obvious examples on what I would call the right wing of Experience Design. ("Black magic" would perhaps be a more appropriate term, but I will leave the occult dimension of Experience Design aside for now). Here the target is often demonstrably irrational: an instinctive, un-self-aware subject whose inchoate fears and desires are organized around commodities or institutions. Though one must always beware of excessive fears over "subliminal" advertising, mallrats with sensitive noses will also recognize the pine and spice scents pumped into malls around Christmas time. Some slot machines are now equipped with high-tech smell emitters because certain scents have proven to keep individuals at the machines longer. Whether or not these cues are culturally determined is beside the point. What’s important is that at the moment, these stimulants aim for a technical zone of influence below "propaganda," which is still linked explicitly to a field of meanings. Instead they directly attack the limbic system, drawing the subject into a deeper, more immersive activity. As the West embarks on a Shadow War against terrorism, the tools of propaganda and psychic management alike can only proliferate.
By embracing the tools of Experience Design, media artists have and can continue to critique our expanding technosphere while also probing its capacity for beauty, pleasure, and novel perceptions -- even wisdom. Artists are uniquely placed to interrogate the production of technological experience, and to question the dominant experiences which are being engineered and renormalized by massive commercial engines of subjectivity. But this critical function must be coupled with experiment, with the willingness to creatively participate in the larger cultural process of re-engineering subjectivity, of pushing the envelope of experience. This is not necessarily a matter of becoming high tech -- relatively low-tech artists like Gary Hill and Bill Viola have made great strides in this direction. But it is a matter of directly engaging, not simply the new technologies, but the underlying technical "material" of subjectivity itself.
Finally, I believe this turn towards experience, in art and technology, is related to the growing embrace of the discourses and practices of spirituality. Whether or not it is defined or encountered within the context of faith traditions or not, "spirituality" largely emphasizes the use of subtle "psychological" techniques and practices to open up and transform our existential, personal, subjective encounter with the world and the self. At its best, the global turn towards meditation, yoga, healing prayer, trance dancing, and practices of loving-kindness reflects a search for a higher tone of experience itself, not a hunger for new consoling beliefs. The secular spirituality of self-help books, brain machines, and leadership seminars can also be seen as a species of "Experience Design" in that it emphasizes changing, or reprogramming, your direct experience of your self in the world. However, here the underlying intentions — which in some sense make or break spiritual aspiration — often leave much to be desired.
Media artists are uniquely placed to explore this emerging world of spirituality without falling into the dogmatic or New Age traps that swallow up so many true believers. Altered states of consciousness are real, and as our media technologies get better at drawing us in and out of them, artists and other non-coercive proponents of the human spirit (or whatever you want to call it) need to become familiar with these states, not simply as a source of inspiration, but as modes of expression, communication, and confrontation itself. By recognizing that the material that we are now focused on is not technology but human experience itself, then we take a step closer to that strange plateau where our inner lives unfold into an almost collective surface of shared sensation and reframed perception — a surface on which we may feel exposed and vulnerable, but beginning to awake.
Dec 09, 2004
Flow is an optimal experience characterized by:
* a sense of playfulness
* a feeling of being in control
* concentration and highly focused attention
* mental enjoyment of the activity for its own sake
* a distorted sense of time
* a match between the challenge at hand and one's skills
Info and Quotes from Flow: The Psychology of Optimal Experience By Mihaly Csikszentmihalyi:
"Contrary to expectation, "flow" usually happens not during relaxing moments of leisure and entertainment, but rather when we are actively involved in a difficult enterprise, in a task that stretches our mental and physical abilities." - Mihaly Csikszentmihalyi
"People seem to get more flow from what they do on their jobs than from leisure activities in free time." - Mihaly Csikszentmihalyi
It turns out that watching TV is not at all a flowful activity. People generally report higher levels of stress, depression, and tension after watching TV. It seems that TV's main virtue is that it occupies the mind undemandingly. Flow is hard to achieve without effort. Flow is not "wasting time".
"It is by being fully involved with every detail of our lives, whether good or bad, that we find happiness, not by trying to look for it directly. - Mihaly Csikszentmihalyi
"People who learn to control inner experience will be able to determine the quality of their lives, which is as close as any of us can come to being happy." - Mihaly Csikszentmihalyi
"Getting control of life is never easy, and sometimes it can be definitely painful." - Mihaly Csikszentmihalyi
"Since what we experience is reality, as far as we are concerned, we can transform reality to the extent that we influence what happens in consciousness and thus free ourselves from the threats and blandishments of the outside world." - Mihaly Csikszentmihalyi