Bridging the Sim2real Gap in Robotics with Marius Memmel - #695

Update: 2024-07-30

Digest

This podcast episode delves into the challenges of building AI for robotic agents, particularly in unstructured environments like kitchens. The guest, Marius Memel, a PhD student at the University of Washington, discusses his research on using simulation to bridge the gap between real-world robotics and traditional models. He introduces the concept of SimTeriel, which involves incorporating simulation into solving robotics problems, and highlights the importance of data collection and the limitations of traditional approaches. Marius delves into the challenges of collecting data for robot learning, emphasizing the cost and limitations of real-world data collection. He explains how simulation offers a cost-effective solution for generating high-quality data, but highlights the "reality gap" between simulation and the real world. He introduces SimTeriel as a way to address this gap by creating more accurate simulations. Marius introduces Acid, a Sim-to-Real approach that aims to close the gap between simulation and the real world. Acid involves an exploration phase where the robot learns to explore the real world to improve the simulator, followed by an exploitation phase where the robot uses the improved simulator to solve tasks. He explains how Acid uses domain randomization and system identification to achieve this. The discussion explores the challenges of defining tasks and selecting the appropriate granularity for robot learning. Marius explains how Acid focuses on the longest horizon possible without adding additional methods, allowing for flexibility in task definition. He emphasizes that Acid provides a simulator that can be used for various tasks involving the parameters and objects of interest. Marius elaborates on Acid's two-phase approach: exploration and exploitation. He explains how the exploration phase in the real world complements the exploration done in simulation, using domain randomization to train the robot to identify a variety of parameters. This information is then used to improve the simulator, enabling more accurate exploitation in the real world. Marius discusses the objective function used in Acid, which is inspired by Fisher information. He explains how this function encourages the robot to collect data that is informative about the underlying physics parameters, leading to more accurate simulations. He highlights the novelty of using Fisher information in a Sim-to-Real loop. Marius discusses the challenges of reconstructing both the geometry and dynamics of a scene for simulation. He explains that Acid focuses on reconstructing the geometry and kinematic structure, assuming the robot has a tracker for the object of interest. The robot then learns to identify the dynamics of the object, such as its center of mass. Marius emphasizes that the Fisher information objective function used in Acid is not limited to the Acid framework. He suggests that it can be used as a general-purpose reward function for any task involving parameter identification, particularly in Sim-to-Real scenarios. Marius compares Acid to other approaches that integrate exploration and exploitation, such as motor adaptation and iterative approaches. He argues that Acid's approach is superior for tasks where a single shot in the real world is crucial, as it uses a safer exploration strategy that doesn't directly execute the task. Marius introduces URD Former, a project that focuses on reconstructing the kinematic structure or geometry of a scene from a single RGB observation. He explains how URD Former uses depth information to create a URDF document, which can then be used to construct a simulator for training robots. Marius discusses the challenges of training an inverse model to go from an RGB image to a URDF document. He explains how URD Former uses synthetically generated data, created through procedural generation and re-skinned with Stable Diffusion, to train the model. This approach allows for the creation of a large dataset of realistic-looking images. Marius discusses the process of bootstrapping URD Former, which involves generating a large dataset of synthetic images and training a transformer-based model. He explains how the model uses bounding box detection to provide additional information, making it easier to reconstruct the URDF document. Marius discusses the potential for combining Acid and URD Former to create a more comprehensive system for constructing simulators on the fly. He highlights the potential for using VLMs to initialize simulations and the challenges of managing large datasets generated by these simulators. Marius shares his perspective on the future of robotics and Sim-to-Real approaches. He believes that constructing simulations on the fly, informed by the real world, is a promising path. He highlights the potential for leveraging VLMs to improve simulation construction and the need for better data management strategies as simulators become more powerful.

Outlines