Human-AI Systems for Creating and Understanding Videos

Video is becoming a core medium for communicating a wide range of content, including educational lectures, vlogs, and how-to tutorials. While videos are engaging and informative, they lack the familiar and useful affordances of text for browsing, skimming,and flexibly transforming information. This severely limits who can interact with video content and how they can interact with it, makes editing a laborious process, and means that much of the information in videos is not accessible to everyone. But, what future systems will make videos useful for all users? In this talk, I will share my work creating interactive Human-AI systems that combine the benefits of multiple mediums of communication (e.g., text, video, and audio) in two key areas: 1) helping domain experts find content of interest in videos, and 2) making videos accessible to people who are blind or have visual impairments. First, I will discuss core challenges of finding information in videos from interviews with domain experts and people with disabilities. Then, I will introduce new systems that leverage AI, and the results of technical and user evaluations that demonstrate system efficacy. I will conclude with how hybrid HCI-AI breakthroughs will make digital communication more effective and accessible in the future, and how new interactions can help us to realize the full potential of recent AI/ML advances.