Robots on the way to becoming household assistants

Robots on the way to becoming household assistants

San Francisco, June 30, 2022

Robots have quickly learned how to deal with rigid objects. But objects that can be deformed at will or shapeless mud cause them problems. Even with soft, easily deformable objects, the machines can’t do much at first.

Scientists at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) and Stanford University recently had robots play with modeling clay, but not for reasons of nostalgia.

Their robot, „RoboCraft,“ learns to simulate and shape doughy objects directly from visual input using a two-finger gripper. After just 10 minutes of data collection, the two-fingered gripper was able to keep up with human counterparts controlling the machine remotely – it was on par, and sometimes better, at the tasks tested.

„Modeling and manipulating objects with high degrees of freedom are essential skills for robots to learn complex interaction tasks in industrial and household settings, such as filling dumplings, rolling sushi, and making pottery,“ says Yunzhu Li, a doctoral student at CSAIL and author of a new article on RoboCraft.

With RoboCraft, we learn the dynamic models directly from high-dimensional sensory data, which gives us a promising data-driven path for effective planning.“

One far-future area the researchers envision is using RoboCraft to assist with household tasks and chores, which could be especially helpful for the elderly or people with limited mobility. Achieving this would require a much more adaptive representation of the dough or object, given the many obstacles that can be encountered, and would also require research into what class of models might be appropriate to capture the underlying structural systems.

„RoboCraft shows that this predictive model can be learned in a very data-efficient way to plan movements. In the long term, we are thinking about using different tools to manipulate materials,“ Li says. „If you think about making dumplings or dough, a single gripper is not enough to do the job. The next step for future work will help the model understand and handle longer-term planning tasks, such as how the dough will deform given the current tool, movements and actions.“