The New Data and New Challenges in Multimedia Research.
arXiv (Cornell University)Published 5 March 2015Open access
Bart Thomée, David A. Shamma, Gerald Friedland, Benjamin Elizalde, Karl Ni, Douglas N. Poland
Citations249
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
The rationale behind the creation of the YFCC100M, the largest public multimedia collection that has ever been released, is explained, as well as the implications the dataset has for science, research, engineering, and development.
Abstract
We present the Yahoo Flickr Creative Commons 100 Million Dataset (YFCC100M), the largest public multimedia collection that has ever been released. The dataset contains a total of 100 million media objects, of which approximately 99.2 million are photos and 0.8 million are videos, all of which carry a Creative Commons license. Each media object in the dataset is represented by several pieces of metadata, e.g. Flickr identifier, owner name, camera, title, tags, geo, media source. The collection provides a comprehensive snapshot of how photos and videos were taken, described, and shared over the years, from the inception of Flickr in 2004 until early 2014. In this article we explain the rationale behind its creation, as well as the implications the dataset has for science, research, engineering, and development. We further present several new challenges in multimedia research that can now be expanded upon with our dataset.
Keywords
Computer Science
2009 IEEE Conference on Computer Vision and Pattern RecognitionImageNet: A large-scale hierarchical image database
61,841 Citations2009Jia Deng, Wei Dong +4 more
A new database called “ImageNet” is introduced, a large-scale ontology of images built upon the backbone of the WordNet structure, much larger in scale and diversity and much more accurate than the current image datasets.
Caffe
11,194 Citations2014Yangqing Jia, Evan Shelhamer +6 more
Caffe provides multimedia scientists and practitioners with a clean and modifiable framework for state-of-the-art deep learning algorithms and a collection of reference models for training and deploying general-purpose convolutional neural networks and other deep models efficiently on commodity architectures.
International Journal of Computer VisionThe Pascal Visual Object Classes Challenge: A Retrospective
7,304 Citations2014Mark Everingham, S. M. Ali Eslami +4 more
A review of the Pascal Visual Object Classes challenge from 2008–2012 is provided, with an appraisal of the aspects of the challenge that worked well, and those that could be improved in future challenges.
The MIR flickr retrieval evaluation
1,811 Citations2008Mark J. Huiskes, Michael S. Lew
This paper presents a collection for the MIR community comprising 25000 images from the Flickr website which are redistributable for research purposes and represent a real community of users both in the image content and image tags.
Evaluation campaigns and TRECVid
1,224 Citations2006Alan F. Smeaton, Paul Over +1 more
An introduction to information retrieval (IR) evaluation from both a user and a system perspective is given, high-lighting that system evaluation is by far the most prevalent type of evaluation carried out.
IM2GPS: estimating geographic information from a single image
890 Citations2008James Hays, Alexei A. Efros
This paper proposes a simple algorithm for estimating a distribution over geographic locations from a single image using a purely data-driven scene matching approach and shows that geolocation estimates can provide the basis for numerous other image understanding tasks such as population density estimation, land cover estimation or urban/rural classification.
Journal of the American Society for Information Science and TechnologyThe conundrum of sharing research data
879 Citations2012Christine L. Borgman
Four rationales for sharing data are examined, drawing examples from the sciences, social sciences, and humanities: to reproduce or to verify research, to make results of publicly funded research available to the public, to enable others to ask new questions of extant data, and to advance the state of research and innovation.
Mapping the world's photos
777 Citations2009David Crandall, Lars Bäckström +2 more
This work uses the spatial distribution of where people take photos to define a relational structure between the photos that are taken at popular places, and finds that visual and temporal features improve the ability to estimate the location of a photo, compared to using just textual features.
Visual CommunicationDigital photography: communication, identity, memory
506 Citations2008José van Dijck
How technical changes (digitization) combined with growing insights in cognitive science and socio-cultural transformations have affected personal photography is explored.
2009 IEEE Conference on Computer Vision and Pattern RecognitionTowards total scene understanding: Classification, annotation and segmentation in an automatic framework
496 Citations2009Li-Jia Li, Richard Socher +1 more
A fully automatic learning framework that is able to learn robust scene models from noisy Web data such as images and user tags from Flickr.com that significantly outperforms state-of-the-art algorithms.
Towards automatic extraction of event and place semantics from flickr tags
475 Citations2007Tye Rattenbury, Nathaniel Good +1 more
An approach for extracting semantics of tags, unstructured text-labels assigned to resources on the Web, based on each tag's usage patterns, and shows that the Scale-structure Identification method outperforms the existing techniques.
New trends and ideas in visual concept detection
345 Citations2010Mark J. Huiskes, Bart Thomée +1 more
This paper provides an overview of the various strategies that were devised for automatic visual concept detection using the MIR Flickr collection, and discusses results from various experiments in combining social data and low-level content-based descriptors to improve the accuracy of visual concept classifiers.
International Journal of Computer VisionSUN Database: Exploring a Large Collection of Scene Categories
336 Citations2014Jianxiong Xiao, Krista A. Ehinger +3 more
The Scene Understanding database is proposed, a nearly exhaustive collection of scenes categorized at the same level of specificity as human discourse that contains 908 distinct scene categories and 131,072 images.
RepliCHI
42 Citations2014Max L. Wilson, Ed H. +2 more
The aim of this workshop is to continue to facilitate a cultural shift towards the community naturally adopting replication techniques in situations that are considered worth investigating.
The Placing Task
31 Citations2014Jaeyoung Choi, Bart Thomée +9 more
A recent development of a new standardized web-scale geo-tagged dataset for Placing Task 2014, which contains 5.5 million photos and 35,000 videos, allows research community to easily evaluate new algorithms and to analyze their performance with respect to the state-of-the-art approaches.
arXiv (Cornell University)The YLI-MED Corpus: Characteristics, Procedures, and Plans
18 Citations2015Julia Bernd, Damian Borth +8 more
The procedures used to collect the corpus are described; detailed descriptive statistics about the corpus makeup are given (and how video attributes affected annotators' judgments); possible biases in the corpus introduced by the authors' procedural choices are discussed; it is compared with the most similar existing dataset, TRECVID MED's HAVIC corpus; and gives an overview of the future plans for expanding the annotation effort.