NSF Grant 1217904: Dexterous Manipulation Using Predictive Thin-Shell Modeling

Peter Allen, Eitan Grinspun PI's
Department of Computer Science
Columbia University
Grasping and manipulation of deformable objects presents a host of new research challenges that are much more demanding than for rigid objects. A particular challenge is to fully understand the physics of deformation and to model deformable objects in a way that can be used by real robotic systems in the presence of noise and uncertainty and with real-time constraints. This project will use offline simulation to predict states of deformable objects modeled as thin-shells (i.e. cloth, fabric, clothing) that can then be recognized by a robotic vision/grasping system to pick up and manipulate these objects. The Intellectual Merit includes a significant step forward in the synthesis of state-of-the-art numerical computation of plasto-elastica with a database-driven manipulation approach to allow robots to manipulate deformable objects. This includes fabric simulation technology that provides the requisite level of accuracy at speeds amenable to online computation parallel to the robotic grasping task. The Broader Impacts of this research include 1) creation of an open source, extensible, 3-D database of deformable objects for dissemination to the robotics and graphics communities, 2) extending the range of working environments for the emerging field of personal robotic assistants, and 3) developing and educating a new class of scientists who bridge the fields of computer graphics, computational mechanics, and robotics.
Recognition of Deformable Object Category and Pose [slides] [videos] [bibtex]

Y. Li, C-F Chen, and P. K. Allen

Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Hong Kong, June 2014

Real-time Pose Estimation of Deformable Objects Using a Volumetric Approach [slides] [videos] [bibtex] [source code]

Y. Li*, Y. Wang*, M. Case, S-F Chang, and P. K. Allen (* equal contribution)

Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Chicago, Sept. 2014

Regrasping and Unfolding of Garments Using Predictive Thin Shell Modeling [slides] [video] [bibtex]

Y. Li, D. Xu, Y. Yue, Y. Wang, S-F Chang, E. Grinspun, and P. K. Allen

To appear in the Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Seattle, May 2015

Folding Deformable Objects using Predictive Simulation and Trajectory Optimization by Y. Li, Y. Yue, Danfei Xu, Eitan Grinspun, Peter K. Allen, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2015, Hamburg. Video

Multi-Sensor Surface Analysis for Robotic Ironing by Yinxiao Li, Xiuhan Hu, Danfei Xu, Yonghao Yue, Eitan Grinspun, Peter Allen, IEEE International Conference on Robotics and Automation, May 2016, Stockholm. Video

Laundry Robot Page

Pipeline for folding deformable objects with trajectory optimization