From Pixels to Physics: Learning to Reason, Act, and Explore

Talk
Jia-Bin Huang
Talk Series: 
Time: 
10.23.2026 11:00 to 12:00

Foundation models have transformed perception and language, but true intelligence requires embodiment. Agents must not only interpret the world, but also predict dynamics, plan actions, and explore autonomously.In this talk, I present a new paradigm for embodied foundation models built on three pillars: reasoning through simulation, learning through abstraction, and exploration through imagination. I will show how (1) physics simulators can be integrated into VLM reasoning, (2) transferable world models can be learned from 3D motion traces distilled from video, and (3) curiosity-driven exploration can be achieved via imagination and verification. Together, these approaches enable agents to acquire physical intuition, adapt across embodiments, and learn from interaction—bringing us closer to general-purpose embodied intelligence.