Coverage Planning for Robotic Vision Applications in Complex 3D Environment

Using robots to perform the vision-based coverage tasks such as inspection, shape reconstruction and surveillance has attracted increasing attentions from both academia and industry in past few years. These tasks require the robot to carry vision sensors such as RGB camera, laser scanner, thermal camera to take surface measurement of the desired target objects, with a required surface coverage constraint. Due to the high demands and repetitive natures of these tasks, automatically generating efficient robotic motion plans could significantly reduce cost and improve productivity, which is highly desirable. Several planning approaches have been proposed for the vision-based coverage planning problems with robots in the past. However, these planning methods either only focused on coverage problems in 2D environment, or found less optimal results, or were specific to limited scenarios. In this thesis, we proposed the novel planning algorithms for vision-based coverage planning problems with industrial manipulators and Unmanned Aerial Vehicles (UAVs) in complex 3D environment. Different sampling and optimization methods have been used in the proposed planning algorithms to achieve better planning results. The very first and important step of these coverage planning tasks is to identify a suitable viewpoint set that satisfies the application requirements. This is considered as a view planning problem. The second step is to plan collision-free paths, as well as the visiting sequence of the viewpoints. This step can be formulated as a sequential path planning problem, or path planning problem for short. In this thesis, we developed view planning methods that generate candidate viewpoints using randomized sampling-based and Medial Object-based methods. The view planning methods led to better results with fewer required viewpoints and higher coverage ratios. Moreover, the proposed view planning methods were also applied to practical application in which the detailed 3D building model needs to be reconstructed when only 2D public map data is available. In addition to the proposed view planning algorithms, we also combined the view planning and path planning problems as a single coverage planning problem; and solved the combined problem in a single optimization process to achieve better results. The proposed planning method was applied to industrial shape inspection application with robotic manipulators. Additionally, we also extended the planning method to a industrial robotic inspection system with kinematic redundancy to enlarge the workspace and to reduce the required inspection time. Moreover, a learning-based robotic calibration method was also developed in order to accurately position vision sensors to desired viewpoints in these instances with industrial manipulators.