PhD Proposal: Seeing Behind the Scene: Using Symmetry to Reason About Objects In Cluttered Environments

Rapid advances in the robotic technology are bringing robots out of the controlled environments of assembly lines and factories into the unstructured and unpredictable "real-life" workspaces of human beings. One of the prerequisites for operating in such environments is the ability to interact with previously unobserved physical objects. To achieve this individual objects have to be delineated from the rest of the environment and their shape properties estimated from partial 3D pointcloud views of the scene. This remains a challenging task due to the lack of prior information about the shape and pose of the object as well as occlusions in cluttered scenes.
We attempt to solve this problem by utilizing the powerful concept of symmetry. Symmetry is ubiquitous in both natural and man-made environments. It reveals redundancies in the structure of the world around us and thus can be used in a variety of visual processing tasks. In this thesis we propose to use symmetry to analyze scenes from partial 3D pointcloud views. We introduce an approach to scene segmentation that consists of two novel algorithms that work in tandem. The first algorithm is used to find 3D bilateral symmetries of the scene. Symmetry plane hypotheses are detected efficiently by matching normal edge curves in the pointcloud The second algorithm uses the detected symmetries to initialize a segmentation process that finds points of the scene that are consistent with the symmetry. The key intuition is that only symmetries corresponding to objects will produce good segmentations. Preliminary results show that this algorithm can successfully segment a variety of common objects in heavily cluttered tabletop scenes and compares favourably to the state of the art approaches.
We propose several extensions to our approach. Firstly symmetry detection and segmentation algorithms will be generalized to handle rotationally symmetric objects. Secondly the results of the segmentation process will be used to reconstruct the occluded points of the scene. The efficacy of our approach will be tested by implementing a robotic system capable of picking apart a pile of objects on a table. Scene interpretations delivered by our algorithms will be used to find objects in the scene and to generate object grasps for a robotic manipulator.
Examining Committee:
Committee Chair: - Dr. Yiannis Aloimonos
Dept's Representative - Dr. David Mount
Committee Member(s): - Dr. Cornelia Fermuller
- Dr. David Jacobs