Positive Technology Journal

Technology does not naturally sense nonverbal cues such as facial expressions and tone of voice, and does not easily acquire common sense knowledge about people. These "mindreading" functions also do not come naturally for some people, such as those diagnosed with autism. ESP is an affective wearable system that explores ways to augment and enhance the wearer's emotional-social intelligence. ESP's computational model of mind-reading infers in real time affective-cognitive mental states from nonverbal cues such as head and facial displays of people, and communicates these inferences to the wearer via visual, sound, and tactile feedback.

Our work leverages the advances in affect sensing and perception to (1) develop technologies that are sensitive to people's affective-cognitive states; (2) advance autism research and (3) create new technologies that enhance the social-emotional intelligence of people diagnosed with autism, as well as those who are not.

The project addresses open research challenges pertaining to whether machines can augment social interactions in a way that improves human to human communication. A longer term aim is to use the prosthesis as an assistive and therapeutic device for people with Autism Spectrum Disorder.