Achieving Codec Telepresence

Imagine two people living in different parts of the world. Wouldn’t it be amazing if they could communicate and interact with each other as if they were co-present in the same room? Enabling such an experience virtually, i.e., building a Codec Telepresence system that is indistinguishable from reality is one of the goals of Reality Labs Research (RL-R) in Pittsburgh. To this end, we develop key technology that combines fundamental computer vision, machine learning, and graphics research based on a novel neural reconstruction and rendering paradigm. In this talk, I will explain what a Codec Telepresence system is, how it works, as well as cover recent research advances towards achieving our goal. In the future, this system will bring the world closer together by enabling anybody to communicate and interact with anyone, anywhere, at any time, as if everyone were sharing the same physical space.