Visions of Emotion: Decoding Human Bodily Expressions

Abstract
The emergence of artificial emotional intelligence technology is revolutionizing the
fields of computers and robotics, allowing for a new level of communication and
understanding of human behaviour that was once thought impossible. Whereas recent advancements in deep learning have transformed the field of computer vision, automated understanding of evoked or expressed emotions in visual media remains in its infancy. This foundering stems from the absence of a universally accepted definition of "emotion," coupled with the inherently subjective nature of emotions and their intricate nuances. In this talk, I will provide a brief overview of our affective computing research and then focus on our recent work on bodily expressed emotion
understanding in the wild. Our multidisciplinary effort among computer and information sciences, psychology, and statistics, proposed a scalable and reliable crowdsourcing approach for collecting in-the-wild perceived emotion data for computers to learn to recognize body languages of humans.

A large and growing annotated dataset with about 10,000 body movements video clips and over 13,000 human characters, named BoLD (Body Language Dataset), has been created. Comprehensive statistical analysis revealed many interesting insights from the dataset. A system to model emotional expressions based on bodily movements, named ARBEE (Automated Recognition of Bodily Expression of Emotion), has also been developed and evaluated. Besides, we have developed a high-quality human motor element dataset based on the Laban Movement Analysis movement coding system and utilized that to jointly learn about motor elements and emotions. Our long-term goal is to integrate knowledge from computing, psychology, and performing arts to enable automated understanding and analysis of emotion through body language.

Speaker Bio
James Wang is a Distinguished Professor in the Artificial Intelligence and
Data Science, and Human-Computer Interaction sections of the College of Information Sciences and Technology at The Pennsylvania State University. He received the bachelor's degree in Mathematics summa cum laude from the University of Minnesota, Twin Cities, MN, and the MS degree in Mathematics, the MS degree in Computer Science, and the Ph.D. degree in Medical Information Sciences, all from Stanford University, Stanford, CA. His current research focuses on the modelling of objects, concepts, aesthetics, and emotions in big visual data, with applications in robotics, psychology, biomedicine, and visual art. His publications have been cited about 28,000 times (h=63). He has supervised 29 doctoral students.

More information about his research group can be found here.