From Pixels to Physics: Learning to Reason, Act, and Explore
Foundation models have transformed perception and language, but true intelligence requires embodiment. Agents must not only interpret the world, but also predict dynamics, plan actions, and explore autonomously.In this talk, I present a new paradigm for embodied foundation models built on three pillars: reasoning through simulation, learning through abstraction, and exploration through imagination. I will show how (1) physics simulators can be integrated into VLM reasoning, (2) transferable world models can be learned from 3D motion traces distilled from video, and (3) curiosity-driven exploration can be achieved via imagination and verification. Together, these approaches enable agents to acquire physical intuition, adapt across embodiments, and learn from interaction—bringing us closer to general-purpose embodied intelligence.