Colloquium Details
Learning Structured World Models From and For Physical Interactions
Speaker: Yunzhu Li, MIT
Location: 60 Fifth Avenue 150
Date: March 31, 2022, 11:30 a.m.
Host: Rob Fergus
Synopsis:
Humans have a strong intuitive understanding of the physical world. We observe and interact with the environment through multiple sensory modalities and build a mental model that predicts how the world would change if we applied a specific action (i.e., intuitive physics). My research draws on insights from humans and develops model-based reinforcement learning (RL) agents that learn from their interactions and build predictive models of the environment that generalize widely across a range of objects made with different materials. The core idea behind my research is to introduce novel representations and integrate structural priors into the learning systems to model the dynamics at different levels of abstraction. I will discuss how such structures can make model-based planning algorithms more effective and help robots to accomplish complicated manipulation tasks (e.g., manipulating an object pile, pouring a cup of water, and shaping deformable foam into a target configuration). Beyond visual perception, I will also discuss how we built multi-modal sensing platforms with dense tactile sensors in various forms (e.g., gloves, socks, vests, and robot sleeves) and how they can lead to more structured and physically grounded models of the world.
Speaker Bio:
Yunzhu Li is a Ph.D. student at MIT, advised by Prof. Antonio Torralba and Prof. Russ Tedrake. His work stands at the intersection of robotics, computer vision, and machine learning, with the goal of helping robots perceive and interact with the physical world as dexterously and effectively as humans do. Yunzhu received the Adobe Research Fellowship and was selected as the First Place Recipient of the Ernst A. Guillemin Master's Thesis Award in Artificial Intelligence and Decision Making at MIT. His research has been published in top journals and conferences, including Nature, NeurIPS, CVPR, and CoRL, and featured by major media outlets, including CNN, BBC, The Wall Street Journal, Forbes, The Economist, and MIT Technology Review. Before coming to MIT, he received a B.S. Degree from Peking University. He has also spent time at the Stanford AI Lab and the NVIDIA Robotics Research Lab.
Notes:
In-person attendance only available to those with active NYU ID cards.