Planar mirrors for image-based robot localization and 3-D reconstruction

Planar catadioptric vision sensors consist of a pinhole camera observing a scene being reflected on two (or more) planar mirrors. These systems have recently received an increasing attention because, unlike stereo cameras, can capture two views of the same scene without the need of hardware multi-camera synchronization and calibration. In this paper we explore the original scenario in which a robot manipulator, equipped with a pinhole camera on its end-effector, observes an unknown 3-D scene both directly and reflected through multiple mirrors. We present new multiple-view properties for this scenario and, based on these theoretical results, we present new image-based camera localization and new 3-D scene reconstruction algorithms. Extensive simulation and real-data experiments illustrate the theory and show the effectiveness of the proposed designs.