Understanding the Visual Appearance of Road Scenes Using a Monocular Camera

Over the past several decades, research efforts in the development of self-driving vehicles have drastically improved accompanying technologies. Since the challenges held by Defense Advanced Research Projects Agency, the autonomous driving industry has increased significantly, and almost all the automotive companies have started to develop the technologies to deploy autonomous driving vehicles in the real world. Even though a lot of companies have been making efforts to achieve fully automated vehicles, the current technologies are not mature enough to be deployed in the real world yet, because self-driving vehicles need to respond to uncontrolled environments, such as moving objects, pedestrians, traffic lights, and unexpected work-zones. Among these uncontrolled environments, this thesis focuses on understanding road information and estimating states of traffic lights. Given that all of the traffic control devices are regularized in colors, color is one of the most significant features to be recognized. In order to accomplish such necessary a vision task, self-driving vehicles must incorporate cameras. Despite the fact that traffic control devices have their own regularized color and cameras can see those devices, they are still difficult to detect and recognize by autonomous vehicles. One of the biggest problems is that the color of those devices can be captured differently based on illumination. In this thesis, we investigate the problem of recognizing static objects using a monocular camera to assist self-driving vehicles in perceiving traffic control devices. The perception system, specifically a camera, should recognize the objects robustly regardless of the environment. Throughout this thesis, we exploit different color spaces and apply machine learning to reduce color variance. Also, we develop algorithms which compensate for illumination changes by considering the Sun position, to further improve the road sign recognition. Furthermore, we improve a traffic light state estimation which performs robustly under various illumination conditions. We deploy and demonstrate all of the algorithms in an autonomous vehicle.