Robots learn to play with play dough
tech innovation 2022
The inner child in many of us feels an overwhelming sense of joy when stumbling across a heap of a fluorescent, rubbery mix of water, salt, and flour that puts the goo on the map: play dough. (Even though it rarely happens in adulthood.)
While it’s fun and easy for 2-year-olds to manipulate play dough, it’s harder for robots to handle shapeless slime. Machines with hard objects have become increasingly reliable, but manipulating soft, deformable objects comes with a laundry list of technical challenges, and most importantly, with the most flexible structures, if you can handle a part. If you move, you’re probably affecting everything else.
Scientists at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) and Stanford University recently let robots get a hand in playing with modeling compound, but not for nostalgia. Their new system learns directly from visual input so that the robot can see, simulate and shape dough objects with a two-fingered gripper. “RoboCraft” can reliably plan the robot’s behavior by pinching and releasing play dough to make different characters, including ones it’s never seen before. With only 10 minutes of data, the two-finger gripper rivaled human counterparts that teleoperated the machine—and sometimes outperformed, on the tasks tested.
“Modeling and manipulating objects with a high degree of freedom are essential capabilities for robots to enable complex industrial and household interaction tasks, such as filling dumplings, rolling sushi, and making pottery,” says Yunzhou Li, CSAL PHD. student and author on a new paper about RoboCraft. “While there have been recent advances in manipulating clothing and ropes, we found that objects with high plasticity, such as dough or plasticine – despite their ubiquity in those domestic and industrial settings – were a largely unexplored area of robocraft. Also, we learn to model dynamics directly from high-dimensional sensory data, which provides a promising data-driven avenue for us to make effective planning.”
With undefined, smooth materials, the entire structure needs to be taken into account before any kind of efficient and effective modeling and planning can be done. By transforming the images into graphs of tiny particles, with an algorithm, RoboCraft makes more accurate predictions about changes in the shape of materials, using graph neural networks as the dynamics model.
Typically, researchers use complex physics simulators to model and understand the forces and dynamics that apply to objects, but RoboCraft only uses visual data. The internal working of the system relies on three parts to shape the soft material into an “R”.
The first part—perception—is all about learning to “see.” It uses cameras to collect raw, visible sensor data from the environment, which are then turned into tiny clouds of particles to represent shapes. A graph-based neural network then uses said particle data to “simulate” the object’s dynamics, or learn how it moves. Then, the algorithm helps plan the robot’s behavior so it learns to “shape” a blob of dough armed with training data from several pinches. While the letters are a bit loose, they are certainly representative.
Apart from the attractive shapes, the team is (actually) working on making dumplings out of dough and ready-made fillings. Right now, with only a two finger gripper, that’s a big question. RoboCraft will require additional equipment (a baker needs many tools to cook; so do robots)—a rolling pin, a stamp, and a mold.
Scientists envision a future in the field of using robocraft to assist with household tasks and chores, which could be especially helpful for the elderly or those with limited mobility. To accomplish this, a more adaptive representation of the dough or object would be needed, given several constraints, and at the same time finding out which class of models might be appropriate to capture the underlying structural systems.
“RoboCraft essentially shows that this predictive model for planning motion can be learned in very data-efficient ways. In the long run, we’re about to use different tools to manipulate materials.” Thinking,” Lee says. “If you think about making dumplings or dough, just a gripper won’t be able to solve it. Helping the model understand and complete long-horizon planning tasks, e.g., present tools, movements, and tasks.” Looking at how the dough will deform is the next step for future work.”
Solving the challenges of robotic pizza making
This story is republished courtesy of MIT News (web.mit.edu/newsoffice/), is a popular site that covers news about MIT research, innovation and teaching.
Citation: Robots learn to play with play dough (2022, 23 June) Retrieved 23 June 2022
This document is subject to copyright. No part may be reproduced without written permission, except for any fair use for the purpose of personal study or research. The content is provided for information purposes only.
Robots learn to play with play dough
#Robots #learn #play #play #dough