Shuran Song

I am an assistant professor in computer science department at Columbia University, where I direct the Columbia Artificial Intelligence and Robotics (CAIR) Lab . My research focuses on computer vision and robotics. I’m interested in developing algorithms that enable intelligent systems to learn from their interactions with the physical world, and autonomously acquire the perception and manipulation skills necessary to execute complex tasks and assist people.

Prospective Students & Post-docs: Prospective post-docs, please email me your CV directly. Current students at Columbia, please fill out this form. PhD applicants, please apply directly to the PhD program and mention my name in your application.

Email: shurans at cs.columbia.edu    Office: 624 CEPSR, 530 West 120th St, New York, NY 10027

News

Recent Talks

Publications

Semantic Abstraction: Open-World 3D Scene Understanding from 2D Vision-Language Models

Huy Ha, Shuran Song
Arxiv Preprint
Webpage  •   Paper   •   Code Coming Soon

CLIP on Wheels: Open-Vocabulary Models are (Almost) Zero-Shot Object Navigators

Samir Yitzhak Gadre, Mitchell Wortsman, Gabriel Ilharco, Ludwig Schmidt, Shuran Song
Arxiv Preprint
Webpage  •   Paper   •   Code Coming Soon

BusyBot: Learning to Interact, Reason, and Plan in a BusyBoard Environment

Zeyi Liu, Zhenjia Xu, Shuran Song
Arxiv Preprint
Webpage  •   Paper   •   Code Coming Soon

Structure From Action: Learning Interactions for Articulated Object 3D Structure Discovery

Neil Nie, Samir Yitzhak Gadre, Kiana Ehsani, Shuran Song
Arxiv Preprint
Webpage  •   Paper   •   Code Coming Soon

Iterative Residual Policy for Goal-Conditioned Dynamic Manipulation of Deformable Objects

Cheng Chi, Benjamin Burchfiel, Eric Cousineau, Siyuan Feng, Shuran Song
Robotics: Science and Systems (RSS) 2022
Best Paper Award   •   Best Student Paper Finalist  •   Webpage  •   Paper   •   Code

DextAIRity: Deformable Manipulation Can be a Breeze

Zhenjia Xu, Cheng Chi, Benjamin Burchfiel, Eric Cousineau, Siyuan Feng, Shuran Song
Robotics: Science and Systems (RSS) 2022
Best System Paper Finalist   •   Webpage  •   Paper   •   Code

Learning Pneumatic Non-Prehensile Manipulation with a Mobile Blower

Jimmy Wu, Xingyuan Sun, Andy Zeng, Shuran Song, Szymon Rusinkiewicz, Thomas Funkhouser
Robotics and Automation Letters (RA-L) 2022
Intelligent Robots and Systems (IROS) 2022
Webpage  •   Paper   •   Code

TANDEM: Learning Joint Exploration and Decision Making with Tactile Sensors

Jingxi Xu, Shuran Song, Matei Ciocarlie
Robotics and Automation Letters (RA-L) 2022
Intelligent Robots and Systems (IROS) 2022
Webpage  •   Paper

Scene Editing as Teleoperation: A Case Study in 6DoF Kit Assembly

Shubham Agrawal*, Yulong Li*, Jen-Shuo Liu, Steven K. Feiner, Shuran Song
Intelligent Robots and Systems (IROS) 2022
Webpage  •   Paper

Continuous Scene Representations for Embodied AI

Samir Yitzhak Gadre, Kiana Ehsani, Shuran Song, Roozbeh Mottaghi,
Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2022)
Webpage  •   Paper

UMPNet: Universal Manipulation Policy Network for Articulated Objects

Zhenjia Xu, Zhanpeng He, Shuran Song
Robotics and Automation Letters (RA-L) and ICRA 2022
Webpage  •   Paper

FishGym: A High-Performance Physics-based Simulation Framework for Underwater Robot Learning

Wenji Liu, Kai Bai, Xuming He, Shuran Song, Changxi Zheng, and Xiaopei Liu
International Conference on Robotics and Automation (ICRA 2022)
Code  •   Paper

Leveraging SE(3) Equivariance for Self-supervised Category-Level Object Pose Estimation from Point Clouds

Xiaolong Li, Yijia Weng, Li Yi, Leonidas Guibas, A. Lynn Abbott, Shuran Song, He Wang
NeurIPS, 2021
Webpage  •   Paper  •   Code

FlingBot: The Unreasonable Effectiveness of Dynamic Manipulations for Cloth Unfolding

Huy Ha, Shuran Song
Conference on Robotic Learning (CoRL2021)
Best System Paper Award   •   Webpage  •   Paper  •   Code

GarmentNets: Category-Level Pose Estimation for Garments via Canonical Space Shape Completion

Cheng Chi, Shuran Song
IEEE International Conference on Computer Vision (ICCV2021)
Webpage  •   Paper  •   Code

Act the Part: Learning Interaction Strategies for Articulated Object Part Discovery

Samir Yitzhak Gadre, Kiana Ehsani, Shuran Song
IEEE International Conference on Computer Vision (ICCV2021)
Webpage (with online Demo!)  •   Paper  •   Code

Dynamic Grasping with Reachability and Motion Awareness

Iretiayo Akinola*, Jingxi Xu*, Shuran Song, and Peter Allen
International Conference on Intelligent Robots and Systems (IROS) 2021
Webpage  •   Paper

AdaGrasp: Learning an Adaptive Gripper-Aware Grasping Policy

Zhenjia Xu, Beichun Qi, Shubham Agrawal, Shuran Song
International Conference on Robotics and Automation (ICRA 2021)
Webpage  •   Paper  •   Code

Spatial Intention Maps for Multi-Agent Mobile Manipulation

Jimmy Wu, Xingyuan Sun, Andy Zeng, Shuran Song, Szymon Rusinkiewicz, Thomas Funkhouser
International Conference on Robotics and Automation (ICRA 2021)
Webpage  •   Paper  •   Code

Visual Perspective Taking for Opponent Behavior Modeling

Boyuan Chen, Yuhang Hu, Robert Kwiatkowski, Shuran Song, Hod Lipson
International Conference on Robotics and Automation (ICRA 2021)
Webpage  •   Paper  •   Code

SSCNav: Confidence-Aware Semantic Scene Completion for Visual Semantic Navigation

Yiqing Liang, Boyuan Chen, Shuran Song
International Conference on Robotics and Automation (ICRA 2021)
Webpage  •   Paper  •   Code

Learning 3D Dynamic Scene Representations for Robot Manipulation

Zhenjia Xu*, Zhanpeng He*, Jiajun Wu, Shuran Song
Conference on Robotic Learning (CoRL) 2020
Webpage  •   Paper  •   Code

Fit2Form: 3D Generative Model for Robot Gripper Form Design

Huy Ha*, Shubham Agrawal*, Shuran Song
Conference on Robotic Learning (CoRL) 2020
Webpage  •   Paper Code

Learning a Decentralized Multi-arm Motion Planner

Huy Ha, Jingxi Xu, Shuran Song
Conference on Robotic Learning (CoRL) 2020
Webpage  •   Paper  •   Code

Multi-task Learning Increases Adversarial Robustness

Chengzhi Mao, Amogh Gupta, Vikram Nitin, Baishakhi Ray, Shuran Song, Junfeng Yang, Carl Vondrick
European Conference on Computer Vision (ECCV) 2020
Oral Presentation  •   PDF

Grasping in the Wild: Learning 6DoF Closed-Loop Grasping from Low-Cost Demonstrations

Shuran Song, Andy Zeng, Johnny Lee, Thomas Funkhouser
Intelligent Robots and Systems (IROS) 2020
Robotics and Automation Letters (RA-L) 2020
Webpage  •   PDF

Spatial Action Maps for Mobile Manipulation

Jimmy Wu, Xingyuan Sun, Andy Zeng, Shuran Song, Johnny Lee, Szymon Rusinkiewicz, Thomas Funkhouser
Robotics: Science and Systems (RSS) 2020
Webpage  •   PDF

Category-Level Articulated Object Pose Estimation

Xiaolong Li, He Wang, Li Yi, Leonidas Guibas, A. Lynn Abbott, Shuran Song
Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2020
Oral Presentation  •   Webpage  •   PDF

Form2Fit: Learning Shape Priors for Generalizable Assembly from Disassembly

Kevin Zakka, Andy Zeng, Johnny Lee, Shuran Song
International Conference on Robotics and Automation (ICRA 2020)
Best Paper Award in Automation Finalist  •   Webpage  •   PDF

ClearGrasp: 3D Shape Estimation of Transparent Objects for Manipulation

Shreeyak S. Sajjan, Matthew Moore, Mike Pan, Ganesh Nagaraja, Johnny Lee, Andy Zeng, Shuran Song
International Conference on Robotics and Automation (ICRA 2020)
Webpage  •   PDF

Learning to See before Learning to Act: Visual Pre-training for Manipulation

Lin Yen-Chen, Andy Zeng, Shuran Song, Phillip Isola, Tsung-Yi Lin
International Conference on Robotics and Automation (ICRA 2020)
Webpage  •   PDF

TossingBot: Learning to Throw Arbitrary Objects with Residual Physics

Andy Zeng, Shuran Song, Stefan Welker, Johnny Lee, Alberto Rodriguez, Thomas Funkhouser
Robotics: Science and Systems 2019 (RSS 2019)
IEEE Transactions on Robotics (T-RO 2020)
Best System Paper Award    Best Student Paper Finalist  •   Webpage  •   PDF

DensePhysNet: Learning Dense Physical Object Representations via Multi-step Dynamic Interactions

Zhenjia Xu, Jiajun Wu, Andy Zeng, Joshua Tenenbaum, Shuran Song
Robotics: Science and Systems 2019 (RSS 2019)
Webpage  •   PDF

Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimation

He Wang, Srinath Sridhar, Jingwei Huang, Julien Valentin, Shuran Song , Leonidas J. Guibasi
Proceedings of 32th IEEE Conference on Computer Vision and Pattern Recognition (CVPR2019)
Oral Presentation   •   Webpage  •   PDF

Neural Illumination: Lighting Prediction for Indoor Environments

Shuran Song and Thomas Funkhouser
Proceedings of 32th IEEE Conference on Computer Vision and Pattern Recognition (CVPR2019)
Oral Presentation   •   Webpage  •   PDF

Neural Graph Matching Networks for Fewshot 3D Action Recognition

Michelle Guo, Edward Chou, Shuran Song, De-An Huang, Serena Yeung, Li Fei-Fei
European Conference on Computer Vision (ECCV2018)
PDF

Learning Synergies between Pushing and Grasping with Self-supervised Deep Reinforcement Learning

A. Zeng, S. Song, S. Welker, J. Lee, A. Rodriguez, T. Funkhouser
Intelligent Robots and Systems (IROS) 2020
Best Cognitive Robotics Paper Award Finalist   •   Webpage  •   PDF

Im2Pano3D: Extrapolating 360° Structure and Semantics Beyond the Field of View

S. Song, A. Zeng, A. X. Chang, M. Savva, S. Savarese, T. Funkhouser
Proceedings of 31th IEEE Conference on Computer Vision and Pattern Recognition CVPR2018
Oral Presentation   •   Webpage  •   PDF

Robotic Pick-and-Place of Novel Objects in Clutter with Multi-Affordance Grasping and Cross-Domain Image Matching

A. Zeng, S. Song, K. Yu, E. Donlon, F. R. Hogan, M. Bauza, D. Ma, O. Taylor, M. Liu, E. Romo, N. Fazeli, F. Alet, N. C. Dafle, R. Holladay, I. Morona, P. Q. Nair, D. Green, I. Taylor, W. Liu, T. Funkhouser, A. Rodriguez (ICRA2018)
Amazon Robotics Best Systems Paper Award   •   Webpage  •   PDF

Semantic Scene Completion from a Single Depth Image

S. Song, F. Yu, A. Zeng, A. Chang, M. Savva, T. Funkhouser
Proceedings of 30th IEEE Conference on Computer Vision and Pattern Recognition (CVPR2017)
Oral Presentation   •   Webpage  •   PDF

3DMatch: Learning the Matching of Local 3D Geometry in Range Scans

A. Zeng, S. Song, M. Nießner, M. Fisher, J. Xiao and T. Funkhouser.
Proceedings of 30th IEEE Conference on Computer Vision and Pattern Recognition (CVPR2017)
Oral Presentation   •   Webpage  •   PDF


Physically-Based Rendering for Indoor Scene Understanding Using Convolutional Neural Networks

Y. Zhang*, S. Song*, E. Yumer, M. Savva, J. Lee, H. Jin, T. Funkhouser.
Proceedings of 30th IEEE Conference on Computer Vision and Pattern Recognition (CVPR2017)
Oral Presentation   •   Webpage  •   PDF

Matterport3D: Learning from RGB-D Data in Indoor Environments

A. X. Chang, A. Dai, T. Funkhouser, M. Halber, M. Nießner, M. Savva, S. Song , A. Zeng, Y. Zhang
IEEE International Conference on 3D Vision (3DV 2017)
Webpage  •   PDF


Multi-view Self-supervised Deep Learning for 6D Pose Estimation in the Amazon Picking Challenge

A. Zeng, K.T. Yu, S. Song, D. Suo, E. Walker Jr., A. Rodriguez, and J. Xiao
International Conference on Robotics and Automation (ICRA2017)
Webpage  •   PDF

Deep Sliding Shapes for Amodal 3D Object Detection in RGB-D Images

S. Song, and J. Xiao.
Proceedings of 29th IEEE Conference on Computer Vision and Pattern Recognition (CVPR2016)
Webpage  •   PDF


ShapeNet: An Information-Rich 3D Model Repository

A. Chang, T. Funkhouser, L. Guibas, P. Hanrahan, Q. Huang, Z. Li, S. Savarese, M. Savva, S. Song, H. Su, J. Xiao, L. Yi, and F. Yu.
arXiv:1512.03012 [cs.CV] 9 Dec 2015
Webpage  •   PDF

SUN RGB-D: A RGB-D Scene Understanding Benchmark Suite

S. Song, S. Lichtenberg and J. Xiao
Proceedings of 28th IEEE Conference on Computer Vision and Pattern Recognition (CVPR2015)
Oral Presentation [Watch it on Techtalks]   •   Webpage  •   PDF

SUN RGB-D: A RGB-D Scene Understanding Benchmark Suite

Z. Wu, S. Song, A. Khosla, F. Yu, L. Zhang, X. Tang and J. Xiao
Proceedings of 28th IEEE Conference on Computer Vision and Pattern Recognition (CVPR2015)
Oral Presentation [Watch it on Techtalks]   •   Webpage  •   PDF

Robot In a Room: Toward Perfect Object Recognition in Closed Environments

S. Song, L. Zhang, and J. Xiao.
arXiv:1507.02703 [cs.CV] 9 Jul 2015
Webpage  •   PDF

Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop

F. Yu, A. Seff, Y. Zhang, S. Song and J. Xiao.
arXiv:1506.03365 [cs.CV] 10 Jun 2015
Webpage  •   PDF

Sliding Shapes for 3D Object Detection in Depth Images

S. Song and J. Xiao
Proceedings of the 13th European Conference on Computer Vision (ECCV2014)
Oral Presentation [Watch it on Videolectures]   •   Webpage  •   PDF


PanoContext: A Whole-room 3D Context Model for Panoramic Scene Understanding

Y. Zhang, S. Song, P. Tan, and J. Xiao
Proceedings of the 13th European Conference on Computer Vision (ECCV2014)
Oral Presentation [Watch it on Videolectures]   •   Webpage  •   PDF


Tracking Revisited using RGBD Camera: Unified Benchmark and Baselines

S. Song and J. Xiao
Proceedings of 14th IEEE International Conference on Computer Vision (ICCV2013)
Webpage  •   PDF

Teaching