Positive Technology Journal

The convergence between ubiquitous computing and wearable microsensors is enabling a new trend in human-computer interaction: “self-tracking”. In general terms, self-tracking refers to the measurement of data about the self, through personal informatics tools. Data collected can be aggregated, visualized, collated into reports and shared. For example, by wearing a small sensor wirelessly connected with a smartphone, a runner can automatically track distance, speed, calories burned and share her performance on the web. But virtually any aspect of one’s life can be potentially registered, at the biological, behavioral or contextual levels. Programs already exist to monitor blood pressure, heart rate, pain, happiness, mood, and location, just to name few examples.

Conceptually, self-tracking is not a new idea. Since the invention of writing, human beings have used different tools to record daily life (i.e. the diary). However, the emergence of pervasive computing tools has made this process easier. In addition, the increasing memory capacity of portable media devices allow to store massive amounts of personal data, making it possible to record an entire life of events.

This is the idea behind MyLifeBits, a project by Microsoft’s Media Presence Research Group, which has created a system able to digitally chronicle every aspect of a person's life. The development of this “lifetime personal store” was possible thanks to the perseverance of Gordon Bell, a Microsoft’s senior researcher and computer scientist, who for eleven years has captured every moment of his personal and professional lives. Bell’s experiment was inspired by the Memex system, a “configurable storehouse of knowledge” envisioned by engineer Vannevar Bush as early as 1945. The database created by Bell includes articles, books, correspondence, photos, telephone calls, video files, web pages visited and is completely searchable. Another pioneer of “life-logging” is Steve Mann, a tenured professor at the Department of Electrical and Computer Engineering at the University of Toronto, who in 1994 started continuously transmitting his everyday life 24 hours a day, 7 days a week on his website: visitors could see what Mann was looking at, as well as communicate with him in real time. 

But besides the intriguing idea of logging one’s life, what is the real promise of self-tracking? Personal informatics offers the possibility to store and access data from our daily life and improve self-knowledge. Insights gained by performing these measurements can be used, for example, to change life-threatening habits, adopt healthier lifestyle, or take more informed treatment decisions. This is why a growing number of websites offer services to track and compare health data. For example, CureTogether allows users to anonimously track various health measures (including symptoms, treatment plans and medication schedules) and share them with other individuals having the same conditions. Aggregated data can be then analyzed to identify trends and eventually highlight the most effective treatments. The founder of the website, Alexandra Carmichael, is herself a self-tracker. She reported monitoring more than 40 different categories of information about her health and personal habits, from caloric intake to daily mood.

The potential of self-tracking has not been overlooked by psychologists. Conceptually, this approach was developed by Mihaly Csikszentmihalyi and Reed Larson in 1983, much before the advent of personal informatics. They created a paper-and-pencil methodology, called Experience Sampling Method (ESM), which requires participants to fill out multiple brief questionnaires about their current activities and feelings by responding to random alerts throughout the day. ESM has been used effectively with adolescent and adult populations for decades to understand areas such as mood, social interactions and time use. This approach has also proven to be helpful in defining therapeutic interventions that are optimally suited for an individual patient.

Used in combination with physiological and contextual measures, computerized version of the ESM may provide a powerful tool to study person-environment interactions. Researchers interested in experimenting with this approach, but lacking programming skills, can use MyExperience, a BSD-licensed open source mobile data collection tool developed for Windows Mobile devices. MyExperience allows the combination of sensing and self-report to collect both quantitative and qualitative data on experience and activity. The beta release of MyExperience supports 50 built-in sensors including GPS, GSM-based motion sensors and device usage information. The sensor events themselves can be used to trigger custom actions such as to initiate wireless database synchronization, send SMS messages to the research team and/or present in situ self-report surveys. Other external sensors (i.e. physiological) can be added via MyExperience plug-in architecture. MyExperience can also be used for designing innovative cybertherapies. For example, Dr. Margaret Morris from the Digital Health Group at Intel and colleagues from Oregon Health and Sciences and Columbia University have recently developed and tested a mobile phone application for mood reporting, which also provides therapeutic exercises for cognitive reappraisal and physical relaxation (M. E. Morris et al, Mobile Therapy: Case Study Evaluations of a Cell Phone Application for Emotional Self-Awareness, Journal of Medical Internet Research, 12(2):e10, 2010).

In summary, self-tracking is an emerging trend in personal informatics, with potentially interesting applications in the fields of cyberpsychology and cybertherapy. However, more research is needed to determine the real benefits (and risks) of this approach.

More to explore:

• Physiological Computing is a blog run byrun by Stephen Fairclough and Kiel Gilleade aimed at providing the latest news and research in the field of physiological computing.
• The Quantified Self is a group engaged in self-tracking activities started by Kevin Kelly and Gary Isaac Wolf. The group also organizes meetups around the world.
• Patients like Me is a social networking health site that enables its members to share treatment and symptom information in order to track and to learn from real-world outcomes.
• The Body Computing Conference is an international conference devoted to the nascent field of physiological computing.
• PsychLog is an open source, smartphone-based experience sampling tool that allows to collect psychological, physiological (via wireless ECG) and behavioral activity data (from wireless in-built accelerometer). 
  

Human Computer Confluence (HC-CO) is an ambitious initiative recently launched by the European Commission under the Future and Emerging Technologies (FET) program, which fosters projects that investigate and demonstrate new possibilities “emerging at the confluence between the human and technological realms” (source: HC-CO website, EU Commission).

Such projects will examine new modalities for individual and group perception, actions and experience in augmented, virtual spaces. In particular, such virtual spaces would span the virtual reality continuum, also extending to purely synthetic but believable representation of massive, complex and dynamic data. HC-CO also fosters inter-disciplinary research (such as Presence, neuroscience, psychophysics, prosthetics, machine learning, computer science and engineering) towards delivering unified experiences and inventing radically new forms of perception/action.

HC-CO brings together ideas stemming from two series of Presence projects (the complete list is available here) with a vision of new forms of interaction and of new types of information spaces to interact with. It will develop the science and technologies necessary to ensure an effective, even transparent, bidirectional communication between humans and computers, which will in turn deliver a huge set of applications: from today's Presence concepts to new senses, to new perceptive capabilities dealing with more abstract information spaces to the social impact of such communication enabling technologies. Inevitably, these technologies question the notion of interface between the human and the technological realm, and thus, also in a fundamental way, put into question the nature of both.

The long-term implications can be profound and need to be considered from an ethical/societal point of view. HC-CO is, however, not a programme on human augmentation. It does not aim to create a super-human. The idea of confluence is to study what can be done by bringing new types of technologically enabled interaction modalities in between the human and a range of virtual (not necessarily naturalistic) realms. Its ambition is to bring our best understanding from human sciences into future and emerging technologies for a new and purposeful human computer symbiosis.

HC-CO is conceptually broken down into the following themes:

• HC-CO Data. On-line perception and interaction with massive volumes of data: new methods to stimulate and use human sensory perception and cognition to interpret massive volumes of data in real time to enable assimilation, understanding and interaction with informational spaces. Research should find new ways to exploit human factors (sensory, perceptual and cognitive aspects), including the selection of the most effective sensory modalities, for data exploration. Although not explicitly mentioned, non-sensorial pathways, i.e., direct brain to computer and computer to brain communication could be explored.
• HC-CO Transit. Unified experience, emerging from the unnoticeable transition from physical to augmented or virtual reality: new methods and concepts towards unobtrusive mixed or virtual reality environment (multi-modal displays, tracking systems, virtual representations...), and scenarios to support entirely unobtrusive interaction. Unobtrusiveness also applies to virtual representations, their dynamics, and the feedback received. Here the challenge is both technological and scientific, spanning human cognition, human machine interaction and machine intelligence disciplines.
• HC-CO Sense. New forms of perception and action: invent and demonstrate new forms of interaction with the real world, virtual models or abstract information by provoking a mapping from an artificial medium to appropriate sensory modalities or brain regions. This research should reinforce data perception and unified experience by augmenting the human interaction capabilities and awareness in virtual spaces.

In sum, HC-CO is an emerging r&d field that holds the potential to revolutionize the way we interact with computers. Standing at the crossroad between cognitive science, computer science and artificial intelligence, HC-CO can provide the cyberpsychology and cybertherapy community with fresh concepts and interesting new tools to apply in both research and clinical domains.

More to explore:

• HC-CO initiative: The official EU website the HC-CO initiative, which describes the broad objectives of this emerging research field. 
• HC2 Project: The horizontal character of HC-CO makes it a fascinating and fertile interdisciplinary field, but it can also compromise its growth, with researchers scattered across disciplines and groups worldwide. For this reason a coordination activity promoting discipline connect, identity building and integration while defining future research, education and policy directions at the regional, national, European and international level has been created. This project is HC2, a three-year Coordination Action funded by the FP7 FET Proactive scheme. The consortium will draw on a wide network of researchers and stakeholders to achieve four key objectives: a) stimulate, structure and support the research community, promoting identity building; b) to consolidate research agendas with special attention to the interdisciplinary aspects of HC-CO; c) enhance the Public Understanding of HC-CO and foster the early contact of researchers with high-tech SMEs and other industry players; d) establish guidelines for the definition of new educational curricula to prepare the next generation of HC-CO researchers.
• CEED Project: Funded by the HC-CO initiative, the Collective Experience of Empathic Data Systems (CEEDs) project aims to develop “novel, integrated technologies to support human experience, analysis and understanding of very large datasets”. CEEDS will develop innovative tools to exploit theories showing that discovery is the identification of patterns in complex data sets by the implicit information processing capabilities of the human brain. Implicit human responses will be identified by the CEEDs system’s analysis of its sensing systems, tuned to users’ bio-signals and non-verbal behaviours. By associating these implicit responses with different features of massive datasets, the CEEDs system will guide users’ discovery of patterns and meaning within the datasets.
• VERE Project: VERE - Virtual Embodiment and Robotic Re-Embodiment – is another large project funded by the HC-CO initiative, which aims at “dissolving the boundary between the human body and surrogate representations in immersive virtual reality and physical reality”. Dissolving the boundary means that people have the illusion that their surrogate representation is their own body, and act and have thoughts that correspond to this. The work in VERE may be thought of as applied presence research and applied cognitive neuroscience.