Designing Virtual Humans in the Metaverse that Move, Express, and Feel Like Us!

The creation of intelligent virtual agents (IVAs) or digital humans is vital for many virtual and augmented reality systems. As the world increasingly uses digital and virtual platforms for everyday communication and interactions, there is a heightened need to create human-like virtual avatars and agents endowed with social and emotional intelligence. Interactions between humans and virtual agents are being used in different areas including, VR, games and story-telling, computer-aided design, social robotics, and healthcare. Designing and building intelligent agents that can communicate and connect with people is necessary but not sufficient. Researchers must also consider how these IVAs will inspire trust and desire among humans. Knowing the perceived affective states and social-psychological constructs (such as behavior, emotions, psychology, motivations, and beliefs) of humans in such scenarios allows the agents to make more informed decisions and navigate and interact in a socially intelligent manner. In this talk, I will give an overview of our recent work on simulating intelligent, interactive, and immersive human-like agents who can also learn, understand and be sentient to the world around them using a combination of emotive gestures, gaits, and expressions. Finally, I will also talk about our many ongoing projects which use our AI-driven IVAs, including intelligent digital humans for urban simulation, mental health and therapy applications, and social robotics.