Ruoshi Liu Explores How Robots Can Learn Through Play

Humans learn how the physical world works long before they can explain it. Through play, they pick up objects, move them around, try different actions and learn from what happens next. Ruoshi Liu is exploring whether robots can learn in a similar way.

Liu joined the University of Maryland’s Department of Computer Science as an assistant professor in fall 2026. His research focuses on robot learning, computer vision and machine learning, with a broader goal of developing systems that can understand their surroundings and use that knowledge to perform physical tasks.

“I’m super excited overall about joining the CS department,” Liu said. “I have been having really good experiences interacting with faculty, students and staff at the department. Really excited and optimistic about the future!”

Liu also holds appointments with the University of Maryland Institute for Advanced Computer Studies and the Center for Automation Research. His lab studies robot manipulation using machine learning, geometric processing and foundation models, drawing on information from the internet, simulations and real-world environments to help robots learn behaviors that can transfer to new situations.

One of Liu’s current projects approaches that larger goal through a familiar human activity: play.

“Humans learn a lot about the physical world from playing, and we are asking whether this process can teach robots about the world as well,” Liu said.

For a robot, learning through play could mean interacting with objects and observing the consequences rather than relying only on instructions for a specific task. That question connects to Liu’s broader work in reinforcement learning, world modeling, dexterous manipulation and human-robot interaction, all of which examine different aspects of how machines can learn to act in changing physical environments.

His previous research provides examples of that approach. In PaperBot, Liu and his collaborators developed a system that learns through real-world experimentation to make and use paper tools. The robot tested designs for tasks including folding and throwing paper airplanes and creating paper grippers, allowing it to learn from the results of its own physical trials rather than relying entirely on simulation.

Another project, AquaBot, examined learning in a less predictable environment: underwater. The autonomous system first learned manipulation skills from human demonstrations and then used self-learning to improve its performance. Researchers tested it on tasks such as grasping rocks, sorting objects and retrieving items underwater.

More recently, Liu has contributed to research examining whether the knowledge contained in generative AI models can also help robots learn. One project uses video generation as part of robot policy learning, allowing a system to predict how a task should unfold visually and translate that information into physical actions. Tests included opening drawers, stacking cups and handling objects the system had not encountered during training.

Together, the projects address a recurring challenge in robotics: building machines that can do more than repeat behaviors demonstrated under fixed conditions. Liu’s research instead examines how robots can learn, adapt and apply what they have learned when their surroundings or tasks change.

That ability could become more relevant as robots move into settings beyond research laboratories. Liu points to potential applications in manufacturing, consumer electronics and health care.

“As robots become more and more ubiquitous in our lives, how can we make them smarter at doing things, adapt to different environments, and interact well with us humans?” Liu said.

Before joining UMD, Liu worked as a research scientist at Meta FAIR and Amazon FAR and remains affiliated with Amazon FAR as an Amazon Scholar. He earned his Ph.D. in computer science from Columbia University in 2025. His earlier computer vision research included work on 3D reconstruction and large-scale collections of three-dimensional objects.

At UMD, Liu plans to continue exploring the connection between advances in artificial intelligence and machines that must operate in the physical world. He is particularly interested in how technologies such as large language models could shape how robots learn and interact with people and their environments.

“I’m really excited and curious about how the advances of AI will change robotics as a field both in academic research and in the industry,” Liu said.

—Story by Samuel Malede Zewdu, CS Communications