Setting up a competition framework for the evaluation of structure extraction from OCR-ed books

International Journal on Document Analysis and Recognition (IJDAR)Published 6 September 2010

Antoine Doucet, Gabriella Kazai, Bodin Dresevic, Aleksandar Uzelac, Bogdan Radakovic, Nikola Todic

DOI10.1007/s10032-010-0127-3

Source URL

Citations29

SJR quartileQ1

SJR score0.83

SNIP1.96

TL;DR

The setup of the Book Structure Extraction competition run at ICDAR 2009 is described, the book collection used in the task, the collaborative construction of the ground truth, the evaluation measures, and the evaluation results are discussed.

Abstract

This paper describes the setup of the Book Structure Extraction competition run at ICDAR 2009. The goal of the competition was to evaluate and compare automatic techniques for deriving structure information from digitized books, which could then be used to aid navigation inside the books. More specifically, the task that participants faced was to construct hyperlinked tables of contents for a collection of 1,000 digitized books. This paper describes the setup of the competition and its challenges. It introduces and discusses the book collection used in the task, the collaborative construction of the ground truth, the evaluation measures, and the evaluation results. The paper also introduces a data set to be used freely for research evaluation purposes.

Keywords

Computer ScienceArts and Humanities

The Journal of Academic LibrarianshipMass Digitization of Books

120 Citations2006Karen Coyle

The Open Content Alliance, a library initiative formed after Google announced its library book digitization project, has brought library digitization projects into the public eye, even though libraries were experimenting with digitization for at least a decade.

Lecture notes in computer scienceAdvances in Focused Retrieval

68 Citations2009Shlomo Geva, Jaap Kamps

Experiments with Proximity-Aware Scoring for XML Retrieval at INEX 2008 and a Content-Based Link Detection Approach Using the Vector Space Model are reported on.

View PDF

Lecture notes in computer scienceFocused Retrieval and Evaluation

61 Citations2010Christine Largeron, Christophe Moulin +1 more

View PDF

ACM SIGIR ForumOverview of the INEX 2007 Book Search track

31 Citations2008Gabriella Kazai, Antoine Doucet

This paper provides an overview of the newly launched Book Search Track at INEX 2007 (BookSearch '07), its participants, tasks, book corpus, test topics and relevance assessments, as well as some results.

ICDAR 2009 Book Structure Extraction Competition

21 Citations2009Antoine Doucet, Gabriella Kazai +4 more

Lecture notes in computer scienceBook Layout Analysis: TOC Structure Extraction Engine

20 Citations2009Bodin Dresevic, Aleksandar Uzelac +2 more

This work presents a book specific layout analysis system used to extract TOC structure information from the scanned and OCRed books, and provides labeling scheme for the TOC sections of the books, high level overview for the book layouts analysis system, as well as TOC Structure Extraction Engine.

Lecture notes in computer scienceEffective Use of Semantic Structure in XML Retrieval

16 Citations2007Roelof van Zwol, Tim van Loosbroek

This article discusses how the structure of an XML document can be enriched using the Reuters 21587 news collection, and studies the effect of the additional semantic structure on the retrieval performance of the XSee search engine for XML documents.

ACM SIGIR ForumReport on the SIGIR 2008 workshop on focused retrieval

16 Citations2008Jaap Kamps, Shlomo Geva +1 more

The report outlines the events of the SIGIR Workshop on Focused Retrieval and summarizes the major outcomes.

Lecture notes in computer scienceOverview of the INEX 2009 Book Track

13 Citations2010Gabriella Kazai, Antoine Doucet +2 more

The goal of the INEX 2009 Book Track is to evaluate approaches for supporting users in reading, searching, and navigating the full texts of digitized books.

Lecture notes in computer scienceXRCE Participation to the Book Structure Task

8 Citations2009Hervé Déjean, Jean-Luc Meunier

The XRCE participation to the Structure Extraction task of the INEX Book track 2009 is presented and the four methods used for detecting the book structure in the book body are explained.