Jun 21, 2016
Two good news for Positive Technology followers.
1) Our new book on Human Computer Confluence is out!
2) It can be downloaded for free here
Human-computer confluence refers to an invisible, implicit, embodied or even implanted interaction between humans and system components. New classes of user interfaces are emerging that make use of several sensors and are able to adapt their physical properties to the current situational context of users.
A key aspect of human-computer confluence is its potential for transforming human experience in the sense of bending, breaking and blending the barriers between the real, the virtual and the augmented, to allow users to experience their body and their world in new ways. Research on Presence, Embodiment and Brain-Computer Interface is already exploring these boundaries and asking questions such as: Can we seamlessly move between the virtual and the real? Can we assimilate fundamentally new senses through confluence?
The aim of this book is to explore the boundaries and intersections of the multidisciplinary field of HCC and discuss its potential applications in different domains, including healthcare, education, training and even arts.
Please cite as follows:
Andrea Gaggioli, Alois Ferscha, Giuseppe Riva, Stephen Dunne, Isabell Viaud-Delmon (2016). Human computer confluence: transforming human experience through symbiotic technologies. Warsaw: De Gruyter. ISBN 9783110471120.
May 26, 2016
In 1999, Joseph Pine and James Gilmore wrote a seminal book titled “The Experience Economy” (Harvard Business School Press, Boston, MA) that theorized the shift from a service-based economy to an experience-based economy.
According to these authors, in the new experience economy the goal of the purchase is no longer to own a product (be it a good or service), but to use it in order to enjoy a compelling experience. An experience, thus, is a whole-new type of offer: in contrast to commodities, goods and services, it is designed to be as personal and memorable as possible. Just as in a theatrical representation, companies stage meaningful events to engage customers in a memorable and personal way, by offering activities that provide engaging and rewarding experiences.
Indeed, if one looks back at the past ten years, the concept of experience has become more central to several fields, including tourism, architecture, and – perhaps more relevant for this column – to human-computer interaction, with the rise of “User Experience” (UX).
The concept of UX was introduced by Donald Norman in a 1995 article published on the CHI proceedings (D. Norman, J. Miller, A. Henderson: What You See, Some of What's in the Future, And How We Go About Doing It: HI at Apple Computer. Proceedings of CHI 1995, Denver, Colorado, USA). Norman argued that focusing exclusively on usability attribute (i.e. easy of use, efficacy, effectiveness) when designing an interactive product is not enough; one should take into account the whole experience of the user with the system, including users’ emotional and contextual needs. Since then, the UX concept has assumed an increasing importance in HCI. As McCarthy and Wright emphasized in their book “Technology as Experience” (MIT Press, 2004):
“In order to do justice to the wide range of influences that technology has in our lives, we should try to interpret the relationship between people and technology in terms of the felt life and the felt or emotional quality of action and interaction.” (p. 12).
However, according to Pine and Gilmore experience may not be the last step of what they call as “Progression of Economic Value”. They speculated further into the future, by identifying the “Transformation Economy” as the likely next phase. In their view, while experiences are essentially memorable events which stimulate the sensorial and emotional levels, transformations go much further in that they are the result of a series of experiences staged by companies to guide customers learning, taking action and eventually achieving their aspirations and goals.
In Pine and Gilmore terms, an aspirant is the individual who seeks advice for personal change (i.e. a better figure, a new career, and so forth), while the provider of this change (a dietist, a university) is an elictor. The elictor guide the aspirant through a series of experiences which are designed with certain purpose and goals. According to Pine and Gilmore, the main difference between an experience and a transformation is that the latter occurs when an experience is customized:
“When you customize an experience to make it just right for an individual - providing exactly what he needs right now - you cannot help changing that individual. When you customize an experience, you automatically turn it into a transformation, which companies create on top of experiences (recall that phrase: “a life-transforming experience”), just as they create experiences on top of services and so forth” (p. 244).
A further key difference between experiences and transformations concerns their effects: because an experience is inherently personal, no two people can have the same one. Likewise, no individual can undergo the same transformation twice: the second time it’s attempted, the individual would no longer be the same person (p. 254-255).
But what will be the impact of this upcoming, “transformation economy” on how people relate with technology? If in the experience economy the buzzword is “User Experience”, in the next stage the new buzzword might be “User Transformation”.
Indeed, we can see some initial signs of this shift. For example, FitBit and similar self-tracking gadgets are starting to offer personalized advices to foster enduring changes in users’ lifestyle; another example is from the fields of ambient intelligence and domotics, where there is an increasing focus towards designing systems that are able to learn from the user’s behaviour (i.e. by tracking the movement of an elderly in his home) to provide context-aware adaptive services (i.e. sending an alert when the user is at risk of falling).
But likely, the most important ICT step towards the transformation economy could take place with the introduction of next-generation immersive virtual reality systems. Since these new systems are based on mobile devices (an example is the recent partnership between Oculus and Samsung), they are able to deliver VR experiences that incorporate information on the external/internal context of the user (i.e. time, location, temperature, mood etc) by using the sensors incapsulated in the mobile phone.
By personalizing the immersive experience with context-based information, it might be possibile to induce higher levels of involvement and presence in the virtual environment. In case of cyber-therapeutic applications, this could translate into the development of more effective, transformative virtual healing experiences.
Furthermore, the emergence of "symbiotic technologies", such as neuroprosthetic devices and neuro-biofeedback, is enabling a direct connection between the computer and the brain. Increasingly, these neural interfaces are moving from the biomedical domain to become consumer products. But unlike existing digital experiential products, symbiotic technologies have the potential to transform more radically basic human experiences.
Brain-computer interfaces, immersive virtual reality and augmented reality and their various combinations will allow users to create “personalized alterations” of experience. Just as nowadays we can download and install a number of “plug-ins”, i.e. apps to personalize our experience with hardware and software products, so very soon we may download and install new “extensions of the self”, or “experiential plug-ins” which will provide us with a number of options for altering/replacing/simulating our sensorial, emotional and cognitive processes.
Such mediated recombinations of human experience will result from of the application of existing neuro-technologies in completely new domains. Although virtual reality and brain-computer interface were originally developed for applications in specific domains (i.e. military simulations, neurorehabilitation, etc), today the use of these technologies has been extended to other fields of application, ranging from entertainment to education.
In the field of biology, Stephen Jay Gould and Elizabeth Vrba (Paleobiology, 8, 4-15, 1982) have defined “exaptation” the process in which a feature acquires a function that was not acquired through natural selection. Likewise, the exaptation of neurotechnologies to the digital consumer market may lead to the rise of a novel “neuro-experience economy”, in which technology-mediated transformation of experience is the main product.
Just as a Genetically-Modified Organism (GMO) is an organism whose genetic material is altered using genetic-engineering techniques, so we could define aTechnologically-Modified Experience (ETM) a re-engineered experience resulting from the artificial manipulation of neurobiological bases of sensorial, affective, and cognitive processes.
Clearly, the emergence of the transformative neuro-experience economy will not happen in weeks or months but rather in years. It will take some time before people will find brain-computer devices on the shelves of electronic stores: most of these tools are still in the pre-commercial phase at best, and some are found only in laboratories.
Nevertheless, the mere possibility that such scenario will sooner or later come to pass, raises important questions that should be addressed before symbiotic technologies will enter our lives: does technological alteration of human experience threaten the autonomy of individuals, or the authenticity of their lives? How can we help individuals decide which transformations are good or bad for them?
Answering these important issues will require the collaboration of many disciplines, including philosophy, computer ethics and, of course, cyberpsychology.
May 24, 2016
Feb 14, 2016
As many analysts predict, next-generation virtual reality technology promises to change our lives.
From manufacturing to medicine, from entertainment to learning, there is no economic or cultural sector that is immune from the VR revolution.
According to a recent report from Digi-Capital, the augmented/virtual reality market could hit $150B revenue by 2020, with augmented reality projected to reach $120B and virtual reality $30B.
Still, there are a lot of unanswered questions concerning the potential negative effects of virtual reality on the human brain. For example, we know very little about the consequences of prolonged immersion in a virtual world.
Most of scientific virtual reality experiments carried out so far have lasted for short time intervals (typically, less than an hour). However, we don’t know what are the potential side effects of being “immersed” for a 12-hour virtual marathon. When one looks at today’s headsets, it might seem unlikely that people will spend so much time wearing them, because they are still ergonomically poor. Furthermore, most virtual reality contents available on the market are not exploiting the full narrative potential of the medium, which can go well beyond a “virtual Manhattan skyride”.
But as soon as usability problems will be fixed, and 3D contents will be compelling and engaging enough, the risk of “3D addiction” may be around the corner. Most importantly, risks of virtual reality exposure are not limited to adults, but especially endanger adolescents’ and children’s health. Given the widespread use of smartphones among kids, it is likely that virtual reality games will become very popular within this segment.
Given that Zuckerberg regards virtual reality as the next big thing after video for Facebook (in March 2014 his corporation bought Oculus VR in a deal worth $2 billion), perhaps he might also consider investing some of these resources for supporting research on the health risks that are potentially associated with this amazing and life-changing technology.
Dec 26, 2015
Via Road to VR
The Manus VR team demonstrate their latest experiment, utilising Valve’s laser-based Lighthouse system to track their in-development VR glove.
Manus VR (previously Manus Machina), the company from Eindhoven, Netherlands dedicated to building VR input devices, seem have gained momentum in 2015. They secured their first round of seed funding and have shipped early units to developers and now, their R&D efforts have extended to Valve’s laser based tracking solution Lighthouse, as used in the forthcoming HTC Vive headset and SteamVR controllers.
The Manus VR team seem to have canibalised a set of SteamVR controllers, leveraging the positional tracking of wrist mounted units to augment Manus VR’s existing glove-mounted IMUs. Last time I tried the system, the finger joint detection was pretty good, but the Samsung Gear VR camera-based positional tracking struggled understandably with latency and accuracy. The experience on show seems immeasurably better, perhaps unsurprisingly.
Oct 12, 2014
From New Scientist
The latest prototype of virtual reality headset Oculus Rift allows you to step right into the movies you watch and the games you play
AN OLD man sits across the fire from me, telling a story. An inky dome of star-flecked sky arcs overhead as his words mingle with the crackling of the flames. I am entranced.
This isn't really happening, but it feels as if it is. This is a program called Storyteller – Fireside Tales that runs on the latest version of the Oculus Rift headset, unveiled last month. The audiobook software harnesses the headset's virtual reality capabilities to deepen immersion in the story. (See also "Plot bots")
Fireside Tales is just one example of a new kind of entertainment that delivers convincing true-to-life experiences. Soon films will get a similar treatment.
Movie company 8i, based in Wellington, New Zealand, plans to make films specifically for Oculus Rift. These will be more immersive than just mimicking a real screen in virtual reality because viewers will be able to step inside and explore the movie while they are watching it.
"We are able to synthesise photorealistic views in real-time from positions and directions that were not directly captured," says Eugene d'Eon, chief scientist at 8i. "[Viewers] can not only look around a recorded experience, but also walk or fly. You can re-watch something you love from many different perspectives."
The latest generation of games for Oculus Rift are more innovative, too. Black Hat Oculus is a two-player, cooperative game designed by Mark Sullivan and Adalberto Garza, both graduates of MIT's Game Lab. One headset is for the spy, sneaking through guarded buildings on missions where detection means death. The other player is the overseer, with a God-like view of the world, warning the spy of hidden traps, guards and passageways.
Deep immersion is now possible because the latest Oculus Rift prototype – known as Crescent Bay – finally delivers full positional tracking. This means the images that you see in the headset move in sync with your own movements.
This is the key to unlocking the potential of virtual reality, says Hannes Kaufmann at the Technical University of Vienna in Austria. The headset's high-definition display and wraparound field of view are nice additions, he says, but they aren't essential.
The next step, says Kaufmann is to allow people to see their own virtual limbs, not just empty space, in the places where their brain expects them to be. That's why Beijing-based motion capture company Perception, which raised more than $500,000 on Kickstarter in September, is working on a full body suit that gathers body pose estimation and gives haptic feedback – a sense of touch – to the wearer. Software like Fireside Tales will then be able to take your body position into account.
In the future, humans will be able to direct live virtual experiences themselves, says Kaufmann. "Imagine you're meeting an alien in the virtual reality, and you want to shake hands. You could have a real person go in there and shake hands with you, but for you only the alien is present."
Oculus, which was bought by Facebook in July for $2 billion, has not yet announced when the headset will be available to buy.
This article appeared in print under the headline "Deep and meaningful"
Oct 06, 2014
In the last decade, online virtual worlds such as Second Life and alike have become enormously popular. Since their appearance on the technology landscape, many analysts regarded shared 3D virtual spaces as a disruptive innovation, which would have rendered the Web itself obsolete.
This high expectation attracted significant investments from large corporations such as IBM, which started building their virtual spaces and offices in the metaverse. Then, when it became clear that these promises would not be kept, disillusionment set in and virtual worlds started losing their edge. However, this is not a new phenomenon in high-tech, happening over and over again.
The US consulting company Gartner has developed a very popular model to describe this effect, called the “Hype Cycle”. The Hype Cycle provides a graphic representation of the maturity and adoption of technologies and applications.
It consists of five phases, which show how emerging technologies will evolve.
In the first, “technology trigger” phase, a new technology is launched which attracts the interest of media. This is followed by the “peak of inflated expectations”, characterized by a proliferation of positive articles and comments, which generate overexpectations among users and stakeholders.
In the next, “trough of disillusionment” phase, these exaggerated expectations are not fulfilled, resulting in a growing number of negative comments generally followed by a progressive indifference.
In the “slope of enlightenment” the technology potential for further applications becomes more broadly understood and an increasing number of companies start using it.
In the final, “plateau of productivity” stage, the emerging technology established itself as an effective tool and mainstream adoption takes off.
So what stage in the hype cycle are virtual worlds now?
After the 2006-2007 peak, metaverses entered the downward phase of the hype cycle, progressively loosing media interest, investments and users. Many high-tech analysts still consider this decline an irreversible process.
However, the negative outlook that headed shared virtual worlds into the trough of disillusionment maybe soon reversed. This is thanks to the new interest in virtual reality raised by the Oculus Rift (recently acquired by Facebook for $2 billion), Sony’s Project Morpheus and alike immersive displays, which are still at the takeoff stage in the hype cycle.
Oculus Rift's chief scientist Michael Abrash makes no mystery of the fact that his main ambition has always been to build a metaverse such the one described in Neal Stephenson's (1992) cyberpunk novel Snow Crash. As he writes on the Oculus blog
"Sometime in 1993 or 1994, I read Snow Crash and for the first time thought something like the Metaverse might be possible in my lifetime."
Furthermore, despite the negative comments and deluded expectations, the metaverse keeps attracting new users: in its 10th anniversary on June 23rd 2013, an infographic reported that Second Life had over 1 million users visit around the world monthly, more than 400,000 new accounts per month, and 36 million registered users.
So will Michael Abrash’s metaverse dream come true? Even if one looks into the crystal ball of the hype cycle, the answer is not easily found.
Sep 25, 2014
Sep 21, 2014
I still can't close my mouth.
The demo lasted about 10 min, during which several scenes were presented. The resolution and framerate are astounding, you can turn completely around. I can say this is the first time in my life I can really say I was there.
I believe this is really the begin of a new era for VR, and I am sure I won't sleep tonight thinking about the infinite possibilities and applications of this technology. and I don't think I am exaggerating - if anything, I am underestimating
Aug 05, 2014
A Japanese company has recreated a tropical beach in the very reception area they also use as their employee meeting space and staff lounge.
Aug 03, 2014
Modulation of functional network with real-time fMRI feedback training of right premotor cortex activity
Modulation of functional network with real-time fMRI feedback training of right premotor cortex activity.
Neuropsychologia. 2014 Jul 21;
Authors: Hui M, Zhang H, Ge R, Yao L, Long Z
Abstract. Although the neurofeedback of real-time fMRI can reportedly enable people to gain control of the activity in the premotor cortex (PMA) during motor imagery, it is unclear how the neurofeedback training of PMA affect the motor network engaged in the motor execution (ME) and imagery (MI) task. In this study, we investigated the changes in the motor network engaged in both ME and MI task induced by real-time neurofeedback training of the right PMA. The neurofeedback training induced changes in activity of the ME-related motor network as well as alterations in the functional connectivity of both the ME-related and MI-related motor networks. Especially, the percent signal change of the right PMA in the last training run was found to be significantly correlated with the connectivity between the right PMA and the left posterior parietal lobe (PPL) during the pre-training MI run, post-training MI run and the last training run. Moreover, the increase in the tapping frequency was significantly correlated with the increase of connectivity between the right cerebellum and the primary motor area / primary sensory area (M1/S1) of the ME-related motor network after neurofeedback training. These findings show the importance of the connectivity between the right PMA and left PPL of the MI network for the up-regulation of the right PMA as well as the critical role of connectivity between the right cerebellum and M1/S1 of the ME network in improving the behavioral performance.
Birdly is a full body, fully immersive, Virtual Reality flight simulator developed at the Zurich University of the Arts (ZHdK). With Birdly, you can embody an avian creature, the Red Kite, visualized through Oculus Rift, as it soars over the 3D virtual city of San Francisco, heightened by sonic, olfactory, and wind feedback.
Jul 30, 2014
A virtual rehabilitation program after amputation: a phenomenological exploration.
Disabil Rehabil Assist Technol. 2013 Nov;8(6):511-5
Authors: Moraal M, Slatman J, Pieters T, Mert A, Widdershoven G
Abstract. PURPOSE: This study provides an analysis of bodily experiences of a man with a lower leg amputation who used a virtual rehabilitation program. METHOD: The study reports data from semi-structured interviews with a 32-year veteran who used a virtual environment during rehabilitation. The interviews were analyzed using interpretative phenomenological analysis (IPA). RESULTS: During this rehabilitation program, he initially experienced his body as an object, which he had to handle carefully. As he went along with the training sessions, however, he was more stimulated to react directly without being aware of the body's position. In order to allow himself to react spontaneously, he needed to gain trust in the device. This was fostered by his narrative, in which he stressed how the device mechanically interacts with his movements. CONCLUSION: The use of a virtual environment facilitated the process of re-inserting one's body into the flow of one's experience in two opposite, but complementary ways: (1) it invited this person to move automatically without taking into account his body; (2) it invited him to take an instrumental or rational view on his body. Both processes fostered his trust in the device, and ultimately in his body. IMPLICATIONS FOR REHABILITATION: Providing (more) technological explanation of the technological device (i.e. the virtual environment), may facilitate a rehabilitation process. Providing (more) explicit technological feedback, during training sessions in a virtual environment, may facilitate a rehabilitation process.
Jul 09, 2014
Experiential Virtual Scenarios With Real-Time Monitoring (Interreality) for the Management of Psychological Stress: A Block Randomized Controlled Trial
The recent convergence between technology and medicine is offering innovative methods and tools for behavioral health care. Among these, an emerging approach is the use of virtual reality (VR) within exposure-based protocols for anxiety disorders, and in particular posttraumatic stress disorder. However, no systematically tested VR protocols are available for the management of psychological stress. Objective: Our goal was to evaluate the efficacy of a new technological paradigm, Interreality, for the management and prevention of psychological stress. The main feature of Interreality is a twofold link between the virtual and the real world achieved through experiential virtual scenarios (fully controlled by the therapist, used to learn coping skills and improve self-efficacy) with real-time monitoring and support (identifying critical situations and assessing clinical change) using advanced technologies (virtual worlds, wearable biosensors, and smartphones).
Full text paper available at: http://www.jmir.org/2014/7/e167/
Jun 30, 2014
Apr 15, 2014
A post-stroke rehabilitation system integrating robotics, VR and high-resolution EEG imaging.
IEEE Trans Neural Syst Rehabil Eng. 2013 Sep;21(5):849-59
Authors: Steinisch M, Tana MG, Comani S
We propose a system for the neuro-motor rehabilitation of upper limbs in stroke survivors. The system is composed of a passive robotic device (Trackhold) for kinematic tracking and gravity compensation, five dedicated virtual reality (VR) applications for training of distinct movement patterns, and high-resolution EEG for synchronous monitoring of cortical activity. In contrast to active devices, the Trackhold omits actuators for increased patient safety and acceptance levels, and for reduced complexity and costs. VR applications present all relevant information for task execution as easy-to-understand graphics that do not need any written or verbal instructions. High-resolution electroencephalography (HR-EEG) is synchronized with kinematic data acquisition, allowing for the epoching of EEG signals on the basis of movement-related temporal events. Two healthy volunteers participated in a feasibility study and performed a protocol suggested for the rehabilitation of post-stroke patients. Kinematic data were analyzed by means of in-house code. Open source packages (EEGLAB, SPM, and GMAC) and in-house code were used to process the neurological data. Results from kinematic and EEG data analysis are in line with knowledge from currently available literature and theoretical predictions, and demonstrate the feasibility and potential usefulness of the proposed rehabilitation system to monitor neuro-motor recovery.
Glyph looks like a normal headset and operates like one, too. That is, until you move the headband down over your eyes and it becomes a fully-functional visual visor that displays movies, television shows, video games or any other media connected via the attached HDMI cable.
Using Virtual Retinal Display (VRD), a technology that mimics the way we see light, the Glyph projects images directly onto your retina using one million micromirrors in each eye piece. These micromirrors reflect the images back to the retina, producing a reportedly crisp and vivid quality.
Apr 06, 2014
Measuring the effects through time of the influence of visuomotor and visuotactile synchronous stimulation on a virtual body ownership illusion
Measuring the effects through time of the influence of visuomotor and visuotactile synchronous stimulation on a virtual body ownership illusion.
Authors: Kokkinara E, Slater M
Abstract. Previous studies have examined the experience of owning a virtual surrogate body or body part through specific combinations of cross-modal multisensory stimulation. Both visuomotor (VM) and visuotactile (VT) synchronous stimulation have been shown to be important for inducing a body ownership illusion, each tested separately or both in combination. In this study we compared the relative importance of these two cross-modal correlations, when both are provided in the same immersive virtual reality setup and the same experiment. We systematically manipulated VT and VM contingencies in order to assess their relative role and mutual interaction. Moreover, we present a new method for measuring the induced body ownership illusion through time, by recording reports of breaks in the illusion of ownership ('breaks') throughout the experimental phase. The balance of the evidence, from both questionnaires and analysis of the breaks, suggests that while VM synchronous stimulation contributes the greatest to the attainment of the illusion, a disruption of either (through asynchronous stimulation) contributes equally to the probability of a break in the illusion.
Researchers have developed a new way to explore the human brain in virtual reality. The system, called Glass Brain, which is developed by Philip Rosedale, creator of the famous game Second Life, and Adam Gazzaley, a neuroscientist at the University of California San Francisco, combines brain scanning, brain recording and virtual reality to allow a user to journey through a person’s brain in real-time.
Read the full story on Neurogadget
Mar 03, 2014
Virtual reality for the assessment of frontotemporal dementia, a feasibility study.
Disabil Rehabil Assist Technol. 2014 Feb 14;
Authors: Mendez MF, Joshi A, Jimenez E
Abstract Purpose: Behavioral variant frontotemporal dementia (bvFTD) is a non-Alzheimer dementia characterized by difficulty in documenting social-emotional changes. Few investigations have used virtual reality (VR) for documentation and rehabilitation of non-Alzheimer dementias. Methods: Five bvFTD patients underwent insight interviews while immersed in a virtual environment. They were interviewed by avatars, their answers were recorded, and their heart rates were monitored. They were asked to give ratings of their stress immediately at the beginning and at the end of the session. Results: The patients tolerated the head-mounted display and VR without nausea or disorientation, heart rate changes, or worsening stress ratings. Their insight responses were comparable to real world interviews. All bvFTD patients showed their presence in the VR environment as they moved their heads to face and respond to each avatar's questions. The bvFTD patients tended to greater verbal elaboration of answers with larger mean length of utterances compared to their real world interviews. Conclusions: VR is feasible and well-tolerated in bvFTD. These patients may have VR responses comparable to real world performance and they may display a presence in the virtual environment which could even facilitate assessment. Further research can explore the promise of VR for the evaluation and rehabilitation of dementias beyond Alzheimer's disease. Implications for Rehabilitation Clinicians need effective evaluation and rehabilitation strategies for dementia, a neurological syndrome of epidemic proportions and a leading cause of disability. Memory and cognitive deficits are the major disabilities and targets for rehabilitation in Alzheimer's disease, the most common dementia. In contrast, social and emotional disturbances are the major disabilities and targets for rehabilitation in behavioral variant frontotemporal dementia (bvFTD), an incompletely understood non-Alzheimer dementia. Virtual reality is a technology that holds great promise for the evaluation and rehabilitation of patients with bvFTD and other non-Alzheimer dementias, and preliminary evidence suggests that this technology is feasible in patients with bvFTD.