Introduction to Large-Scale Visual Geo-localization
Advances in computer vision and pattern recognitionPublished 1 January 2016
Amir Zamir, Asaad Hakeem, Luc Van Gool, Mubarak Shah, Richard Szeliski
Citations11
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This chapter discusses how the availability of web-scale geo-referenced data affects visual geo-localization, what role semantic information plays in this problem, and how precise localization can be achieved using large-scale textured (RGB) and untextured (non-RGB) 3D models.
Abstract
Despite recent advances in computer vision and large-scale indexing techniques, automatic geo-localization of images and videos remains a challenging task. The majority of existing computer vision solutions for geo-localization are limited to highly-visited urban regions for which a significant amount of geo-tagged imagery is available, and therefore, do not scale well to large and ordinary geo-spatialGeo-spatial regions. In this chapter, we provide an overview of the major research themes in visual geo-localization, investigate the challenges, and point to problem areas that will benefit from common synthesis of perspectives from these research themes. In particular, we discuss how the availability of web-scale geo-referenced data affects visual geo-localization, what role semantic information plays in this problem, and how precise localization can be achieved using large-scale textured (RGB) and untextured (non-RGB) 3D models. We also introduce a few real-world applications which became feasible as a result of the capability of estimating an image's geo-location. We conclude this chapter by providing an overview of the emerging trends in visual geo-localization and a summary of the rest of the chapters of the book.
Keywords
Computer ScienceEngineering
Communications of the ACMImageNet classification with deep convolutional neural networks
75,709 Citations2017Alex Krizhevsky, Ilya Sutskever +1 more
A large, deep convolutional neural network was trained to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes and employed a recently developed regularization method called "dropout" that proved to be very effective.
NeurocomputingAdvances in neural information processing systems 7
22,300 Citations1997Krzysztof J. Cios, Mark E. Shields
Image and Vision ComputingImage registration methods: a survey
6,326 Citations2003Barbara Zitová, Jan Flusser
A review of recent as well as classic image registration methods to provide a comprehensive reference source for the researchers involved in image registration, regardless of particular application areas.
arXiv (Cornell University)Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
1,827 Citations2014David Eigen, Christian Puhrsch +1 more
Building Rome in a day
1,055 Citations2009Sameer Agarwal, Noah Snavely +3 more
IM2GPS: estimating geographic information from a single image
890 Citations2008James Hays, Alexei A. Efros
This paper proposes a simple algorithm for estimating a distribution over geographic locations from a single image using a purely data-driven scene matching approach and shows that geolocation estimates can provide the basis for numerous other image understanding tasks such as population density estimation, land cover estimation or urban/rural classification.
ACM Transactions on GraphicsWhat makes Paris look like Paris?
687 Citations2012Carl Doersch, Saurabh Singh +3 more
It is shown that geographically representative image elements can be discovered automatically from Google Street View imagery in a discriminative manner and it is demonstrated that these elements are visually interpretable and perceptually geo-informative.
City-Scale Location Recognition
626 Citations2007Grant Schindler, Matthew A. Brown +1 more
It is shown that by carefully selecting the vocabulary using the most informative features, retrieval performance is significantly improved, allowing us to increase the number of database images by a factor of 10.
24/7 place recognition by view synthesis
520 Citations2015Akihiko Torii, Relja Arandjelović +3 more
A new place recognition approach is developed that combines an efficient synthesis of novel views with an efficient synthesis of novel views with a compact indexable image representation and significantly outperforms other large-scale place recognition techniques on this challenging data.
Lecture notes in computer scienceLocation Recognition Using Prioritized Feature Matching
446 Citations2010Yunpeng Li, Noah Snavely +1 more
This work devise an adaptive, prioritized algorithm for matching a representative set of SIFT features covering a large scene to a query image for efficient localization, based on considering features in the scene database, and matching them to query image features, as opposed to more conventional methods that match image features to visual words or database features.
City-scale landmark identification on mobile devices
398 Citations2011David M. Chen, Georges Baatz +10 more
Two popular representations of street-level image data are fused and show that they contain complementary information that can be exploited to significantly improve the recall rates on the city scale and improve feature detection in low contrast parts of the street-level data.
Learning deep representations for ground-to-aerial geolocalization
396 Citations2015Tsung-Yi Lin, Yin Cui +2 more
This work localizes a ground-level query image by matching it to a reference database of aerial imagery and shows the effectiveness of Where-CNN in finding matches between street view and aerial view imagery and the ability of the learned features to generalize to novel locations.
Lecture notes in computer scienceAccurate Image Localization Based on Google Maps Street View
368 Citations2010Amir Zamir, Mubarak Shah
A localization method in which the SIFT descriptors of the detected SIFT interest points in the reference images are indexed using a tree in order to localize a query image and a novel GPS-tag-based pruning method removes the less reliable descriptors.
Lecture notes in computer scienceWorldwide Pose Estimation Using 3D Point Clouds
367 Citations2012Yunpeng Li, Noah Snavely +2 more
This work addresses the problem of determining where a photo was taken by estimating a full 6-DOF-plus-intrincs camera pose with respect to a large geo-registered 3D point cloud with state-of-the-art results on landmark recognition as well as the ability to locate cameras to within meters, requiring only seconds per query.
Visual Place Recognition with Repetitive Structures
354 Citations2013Akihiko Torii, Josef Šivic +2 more
This work describes a representation of repeated structures suitable for scalable retrieval based on robust detection of repeated image structures and a simple modification of weights in the bag-of-visual-word model.
Wide-Area Image Geolocalization with Aerial Reference Imagery
332 Citations2015Scott Workman, Richard Souvenir +1 more
This work proposes to use deep convolutional neural networks to address the problem of cross-view image geolocalization, in which the geolocation of a ground-level query image is estimated by matching to georeferenced aerial images.
Discrete-continuous optimization for large-scale structure from motion
313 Citations2011David Crandall, Andrew Owens +2 more
This work presents an alternative formulation for SfM based on finding a coarse initial solution using a hybrid discrete-continuous optimization, and then improving that solution using bundle adjustment, and shows that it can produce models that are similar to or better than those produced with incremental bundles adjustment, but more robustly and in a fraction of the time.
Cross-View Image Geolocalization
263 Citations2013Tsung-Yi Lin, Serge Belongie +1 more
A cross-view feature translation approach to greatly extend the reach of image geolocalization methods by learning the relationship between ground level appearance and overhead appearance and land cover attributes from sparsely available geotagged ground-level images.
IEEE Transactions on Pattern Analysis and Machine IntelligenceImage Geo-Localization Based on MultipleNearest Neighbor Feature Matching UsingGeneralized Graphs
215 Citations2014Amir Zamir, Mubarak Shah
A robust distance function based on the Gaussian Radial Basis Function (G-RBF) is proposed and evaluated on a new data set of 102k street view images; the experiments show it outperforms the state of the art by 10 percent.
ACM Transactions on GraphicsFinding paths through the world's photos
207 Citations2008Noah Snavely, Rahul Garg +2 more
Lecture notes in computer scienceAvoiding Confusing Features in Place Recognition
205 Citations2010Jan Knopp, Josef Šivic +1 more
A method for automatic detection of image-specific and spatially-localized groups of confusing features is developed, and it is demonstrated that suppressing them significantly improves place recognition performance while reducing the database size.
Lecture notes in computer scienceLarge Scale Visual Geo-Localization of Images in Mountainous Terrain
158 Citations2012Georges Baatz, Olivier Saurer +2 more
This work targets mountainous terrain and uses digital elevation models to extract representations for fast visual database lookup and validate the system on the scale of a whole country Switzerland, 40 000km2 using a new dataset of more than 200 landscape query pictures with ground truth.
Proceedings of the IEEEScene Reconstruction and Visualization From Community Photo Collections
131 Citations2010Noah Snavely, Ian Simon +3 more
This paper reviews recent progress on these challenging computer vision problems, and describes how it can be used to turn community photo collections into immersive, interactive 3-D experiences.
Geo-localization of street views with aerial image databases
116 Citations2011Mayank Bansal, Harpreet Sawhney +2 more
The feasibility of solving the challenging problem of geolocalizing ground level images in urban areas with respect to a database of images captured from the air such as satellite and oblique aerial images is studied.
Geolocating Static Cameras
112 Citations2007Nathan Jacobs, Scott Satkin +3 more
It is found that simple summary statistics are sufficient to geolocate cameras without determining correspondences between cameras or explicitly reasoning about weather in the scene, and most cameras can be localized to within 50 miles of the known location.
Learning and Calibrating Per-Location Classifiers for Visual Place Recognition
107 Citations2013Petr Gronát, Guillaume Obozinski +2 more
Style-Aware Mid-level Representation for Discovering Visual Connections in Space and Time
100 Citations2013Yong Jae Lee, Alexei A. Efros +1 more
This work presents a weakly-supervised visual data mining approach that discovers connections between recurring mid-level visual elements in historic and geographic image collections, and attempts to capture the underlying visual style.
International Journal of Computer VisionWhat Do the Sun and the Sky Tell Us About the Camera?
88 Citations2009Jean‐François Lalonde, Srinivasa G. Narasimhan +1 more
The sun and the sky serve as geometric calibration targets, which can be used to annotate a large database of image sequences and are demonstrated to achieve a mean localization error of 110 km on real, low-quality Internet webcams.
Registration of video to geo-referenced imagery
83 Citations2002Raman Kumar, Harpreet Sawhney +3 more
This work presents the key algorithms for the problem of accurate mapping between camera coordinates and geo-coordinates, called geo-spatial registration, and utilizes the imagery and terrain information contained in the geo- Spatial database to precisely align dynamic videos with the reference imagery and thus achieves a much higher accuracy.
Automatic photo-to-terrain alignment for the annotation of mountain pictures
80 Citations2011Lionel Baboud, Martin Čadík +2 more
A novel automatic technique is proposed to avoid the burden of manual registration of a given photograph with a 3D geo-referenced terrain model, able to automatically derive the pose of the camera relative to the geometric terrain model.
Lecture notes in computer scienceGIS-Assisted Object Detection and Geospatial Localization
65 Citations2014Shervin Ardeshir, Amir Zamir +2 more
This paper proposes a method for improving object detection using a set of priors acquired from GIS databases, and demonstrates that this approach outperforms well established object detectors, such as DPM, with a large margin.
Journal of Multimedia4D Cities: Analyzing, Visualizing, and Interacting with Historical Urban Photo Collections
63 Citations2012Grant Schindler, Frank Dellaert
This work uses computer vision techniques to tie together large sets of historical photographs of a given city into a consistent 4D model of the city: a 3D model with time as an additional dimension.
Finding paths through the world's photos
55 Citations2008Noah Snavely, Rahul Garg +2 more
This work takes as input a large set of community or personal photos, reconstructs camera viewpoints, and automatically computes orbits, panoramas, canonical views, and optimal paths between views.
City scale geo-spatial trajectory estimation of a moving camera
50 Citations2012Gonzalo Vaca-Castano, Amir Zamir +1 more
A novel method for estimating the geospatial trajectory of a moving camera with unknown intrinsic parameters, in a city-scale urban environment based on a three step process that includes finding the best visual matches of individual images to a dataset of geo-referenced street view images, Bayesian tracking to estimate the frame localization and its temporal evolution, and a trajectory reconstruction algorithm to eliminate inconsistent estimations.
Lecture notes in computer scienceUltra-wide Baseline Facade Matching for Geo-localization
44 Citations2012Mayank Bansal, Kostas Daniilidis +1 more
A new method for scale selection is introduced which enables the extraction and segmentation of facades as well and shows experimental results on retrieval accuracy on a challenging set of publicly available imagery and compare with standard SIFT-based techniques.
Lecture notes in computer scienceScene Chronology
31 Citations2014Kevin Matzen, Noah Snavely
A new method for taking an urban scene reconstructed from a large Internet photo collection and reasoning about its change in appearance through time, which is robust and scalable to scenes with hundreds of thousands of images and billions of noisy, individual point observations.
User-Driven Geolocation of Untagged Desert Imagery Using Digital Elevation Models
31 Citations2013Eric Tzeng, Andrew Zhai +3 more
This system makes use only of publicly available digital elevation models (DEMs) to rapidly and accurately locate photographs in non-urban environments such as deserts and applies a variety of geometrically constrained matching techniques to efficiently and accurately match the query skyline to a database skyline, thereby localizing the query image.
Webcam geo-localization using aggregate light levels
19 Citations2011Nathan Jacobs, Kylia Miskell +1 more
A simple, robust cue that relates overall image intensity to the zenith angle of the sun (which need not be visible) is explored, and the accuracy of geolocation based on this cue as a function of different models of theZenith-intensity relationship and the amount of imagery available is characterized.
Creating compact architectural models by geo-registering image collections
19 Citations2009Radek Grzeszczuk, Jana Košecká +2 more
An optimal view-selection algorithm for selecting a small set of views for texture mapping that best describe the structure, while minimizing warping and stitching artifacts, and producing a consistent visual representation is proposed.
GeoSensor NetworksFeature-Based Georegistration of Aerial Images
15 Citations2004Sohaib Khan, Yaser Sheikh +1 more
The telemetry information is used to orthorectify the aerial images, to bring both im-ageries into a common projection space, and then applied to apply the registration technique to achieve accurate alignment.
Visual business recognition
11 Citations2013Amir Zamir, Afshin Dehghan +1 more
This paper proposes a multimodal approach which incorporates business directories, textual information, and web images in a unified framework for visual business recognition and employs a text processing method customized for business recognition assisted by nearby business names.
Relaxation Matching for Georegistration of Aerial and Satellite Imagery
10 Citations2007Caixia Wang, Anthony Stefanidis +1 more
In this paper, a novel approach for image-to-map registration through graph matching is presented, which relies on its invariance to translation, rotation and scale differences, and is optimized through the use of continuous relaxation labeling.
Proceedings of ... IEEE International Conference on Fuzzy SystemsImage-to-X Registration using Linear Features
4 Citations2007Caixia Wang, Arie Croitoru +2 more
Automated algorithms for the registration of sequences of aerial imagery to vector map data using linear features (primarily roads) as control information are introduced and both exact and inexact matching schemes are presented.