Learning to link with wikipedia

Published 26 October 2008

David Milne, Ian H. Witten

Citations1,266

TL;DR

This paper explains how machine learning can be used to identify significant terms within unstructured text, and enrich it with links to the appropriate Wikipedia articles, and performs very well, with recall and precision of almost 75%.

Abstract

This paper describes how to automatically cross-reference documents with Wikipedia: the largest knowledge base ever known. It explains how machine learning can be used to identify significant terms within unstructured text, and enrich it with links to the appropriate Wikipedia articles. The resulting link detector and disambiguator performs very well, with recall and precision of almost 75%. This performance is constant whether the system is evaluated on Wikipedia articles or "real world" documents.

Keywords

Computer ScienceSocial Sciences

C4.5: Programs for Machine Learning

23,665 Citations1992J. R. Quinlan

A complete guide to the C4.5 system as implemented in C for the UNIX environment, which starts from simple core learning methods and shows how they can be elaborated and extended to deal with typical problems such as missing data and over hitting.

Programs for Machine Learning

5,794 Citations1994Steven L. Salzberg, Alberto M. Segre

In his new book, C4.5: Programs for Machine Learning, Quinlan has put together a definitive, much needed description of his complete system, including the latest developments, which will be a welcome addition to the library of many researchers and students.

Lecture notes in computer scienceDBpedia: A Nucleus for a Web of Open Data

4,675 Citations2007Sören Auer, Christian Bizer +4 more

The extraction of the DBpedia datasets is described, and how the resulting information is published on the Web for human-andmachine-consumption and how DBpedia could serve as a nucleus for an emerging Web of open data.

View PDF

Yago

3,927 Citations2007Fabian M. Suchanek, Gjergji Kasneci +1 more

YAGO builds on entities and relations and currently contains more than 1 million entities and 5 million facts, which includes the Is-A hierarchy as well as non-taxonomic relations between entities (such as hasWonPrize).

View PDF

Wikify!

921 Citations2007Rada Mihalcea, Andras Csomai

This paper introduces the use of Wikipedia as a resource for automatic keyword extraction and word sense disambiguation, and shows how this online encyclopedia can be used to achieve state-of-the-art results on both these tasks.

An effective, low-cost measure of semantic relatedness obtained from Wikipedia links

677 Citations2008David Milne, Ian H. Witten

This paper describes a new technique for obtaining measures of semantic relatedness that uses Wikipedia to provide structured world knowledge about the terms of interest using the hyperlink structure of Wikipedia rather than its category hierarchy or textual content.

Semantic Wikipedia

479 Citations2006Max Völkel, Markus Krötzsch +3 more

This paper provides an extension to be integrated in Wikipedia, that allows the typing of links between articles and the specification of typed data inside the articles in an easy-to-use manner, and presents the design, implementation, and possible uses of this extension.

MADOC (University of Mannheim)Deriving a large scale taxonomy from Wikipedia

447 Citations2007Simone Paolo Ponzetto, Michael Strube

A large scale taxonomy containing a large amount of subsumption is derived using methods based on connectivity in the network and lexicosyntactic matching to label the semantic relations between categories in Wikipedia.

Overcoming the brittleness bottleneck using wikipedia: enhancing text categorization with encyclopedic knowledge

404 Citations2006Evgeniy Gabrilovich, Shaul Markovitch

It is proposed to enrich document representation through automatic use of a vast compendium of human knowledge--an encyclopedia, and empirical results confirm that this knowledge-intensive representation brings text categorization to a qualitatively new level of performance across a diverse collection of datasets.

Clustering short texts using wikipedia

335 Citations2007Somnath Banerjee, Krishnan Ramanathan +1 more

A method of improving the accuracy of clustering short texts by enriching their representation with additional features from Wikipedia is proposed and empirical results indicate that this enriched representation of text items can substantially improve the clustering accuracy when compared to the conventional bag of words representation.