This thesis examines the detection of vanishing points and the horizon line and their application to visual localization tasks in urban environments. The thesis proposes new deep learning methods to overcome the limitations of existing approaches to vanishing point detection. The first key contribution introduces a novel approach for horizon line and vanishing point detection. Unlike most existing methods, our method directly infers both the HL and an unlimited number of horizontal VPs, even those extending beyond the image frame. The second key contribution of this thesis is a structure-enhanced vanishing point detector. This method utilizes a multi-task learning framework to estimate multiple horizontal vanishing points from a single image. It goes beyond simple vanishing point detection by generating masks that identify vertical planar structures corresponding to each vanishing point, providing valuable scene layout information. Experimental results demonstrate that our method outperforms traditional line-based methods and modern deep learning-based methods. The thesis then explores the use of vanishing points for image matching and registration, particularly in cases where images are captured from vastly different viewpoints. Despite continuous progress in feature extractors and descriptors, these methods often fail in the presence of significant scale or viewpoint variations. The proposed methods address this challenge by incorporating vanishing points and scene structures. One major challenge in using vanishing points for registration is establishing reliable correspondences, especially in large-scale scenarios. This work addresses this challenge by proposing a vanishing point matching method aided by the detection of masks of vertical scene structures corresponding to these vanishing points. To our knowledge, this is the first implementation of a method for vanishing point matching that exploits image content rather than just detected segments. This vanishing point correspondence facilitates the estimation of the camera’s relative rotation, particularly in large-scale scenarios. Additionally, incorporating information from scene structures enables more reliable keypoint correspondence within these structures. Consequently, the method facilitates the estimation of relative translation, which is itself constrained by the rotation derived from the vanishing points. The quality of rotation can sometimes be impacted by the imprecision of detected vanishing points. Therefore, we propose a vanishing point-guided image matching method that is much less sensitive to the accuracy of vanishing point detection.