Foundation Models and 3D Computer Vision

In this partly speculative talk, I will share my thoughts on Foundation Models (aka Large Models) and their implications for object-centric 3D computer vision. To do this, I will first discuss some of our recent work on learning to generate, edit, arrange, and capture 3D objects and humans. This will include our work on (1) recursively generating and modifying 3D shapes using natural language descriptions; (2) arranging 3D shapes and re-arranging collections of shapes; and (3) capturing real-world objects and human hands. Next, using our and others' work as examples, I will speculate on how Foundation Models could provide new perspectives for addressing the same problems. I will conclude by identifying open opportunities and challenges.