Provenance in databases
Published 11 June 2007
Peter Buneman, Wang-Chiew Tan
Citations115
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This tutorial will attempt to provide an overview of research in provenance in databases with a focus on recent database research and technology in this area.
Abstract
The provenance of data has recently been recognized as central to the trust one places in data. It is also important to annotation, to data integration and to probabilistic databases. Three workshops have been held on the topic, and it has been the focus of several research projects and prototype systems. This tutorial will attempt to provide an overview of research in provenance in databases with a focus on recent database research and technology in this area. This tutorial is aimed at a general database research audience and at people who work with scientific data.
Keywords
Computer ScienceDecision Sciences
Lecture notes in computer scienceWhy and Where: A Characterization of Data Provenance
1,310 Citations2001Peter Buneman, Sanjeev Khanna +1 more
An approach to computing provenance when the data of interest has been created by a database query is described, adopting a syntactic approach and present results for a general data model that applies to relational databases as well as to hierarchical data such as XML.
ACM SIGMOD RecordA survey of data provenance in e-science
1,157 Citations2005Yogesh Simmhan, Beth Plale +1 more
The main aspect of the taxonomy categorizes provenance systems based on why they record provenance, what they describe, how they represent and storeprovenance, and ways to disseminate it.
Trio: A System for Integrated Management of Data, Accuracy, and Lineage
477 Citations2004Jennifer Widom
This paper provides numerous motivating applications for Trio and lays out preliminary plans for the data model, query language, and prototype system.
Very Large Data BasesULDBs: databases with uncertainty and lineage
459 Citations2006Omar Benjelloun, Anish Das Sarma +2 more
It is shown that the ULDB representation is complete, and that it permits straightforward implementation of many relational operations, and how ULDBs enable a new approach to query processing in probabilistic databases.
ACM Transactions on Database SystemsTracing the lineage of view data in a warehousing environment
436 Citations2000Yingwei Cui, Jennifer Widom +1 more
The lineage problem is formally defined, lineage tracing algorithms for relational views with aggregation are developed, and mechanisms for performing consistent lineage tracing in a multisource data warehousing environment are proposed.
ACM Computing SurveysLineage retrieval for scientific data processing: a survey
415 Citations2005Rajendra Bose, James Frew
Based on a comprehensive survey of lineage research and previous prototypes, a metamodel is presented to help identify and assess the basic components of systems that provide lineage retrieval for scientific data products.
Provenance management in curated databases
296 Citations2006Peter Buneman, Adriane Chapman +1 more
This paper describes an approach in which the user's actions are tracked while browsing source databases and copying data into a curated database, in order to record theuser's actions in a convenient, queryable form and uses it to evaluate the feasibility of database support for provenance management.
The VLDB JournalAn annotation management system for relational databases
277 Citations2005Deepavali Bhagwat, Laura Chiticariu +2 more
An annotation management system for relational databases where every piece of data in a relation is assumed to have zero or more annotations associated with it and annotations are propagated along, from the source to the output, as data is being transformed through a query.
On propagation of deletions and annotations through views
219 Citations2002Peter Buneman, Sanjeev Khanna +1 more
The two problems correspond to two fundamentally distinct notions of provenance, why and where-provenance, which gives important insights into computational issues involved in data provenance or lineage --- the process by which data moves through databases.
Supporting fine-grained data lineage in a database visualization environment
200 Citations2002Allison Woodruff, Michael Stonebraker
This paper proposes a novel method to support fine-grained data lineage that lazily computes the lineage using a limited amount of information about the processing operators and the base data, and introduces the notions of weak inversion and verification.
RePEc: Research Papers in EconomicsA Polygen Model for Heterogeneous Database Systems: The Source Tagging Perspective
172 Citations1990Y. Richard Wang, Stuart Madnick
A polygen model for resolving the Data Source Tagging and Intermediate source Tagging problems is presented and a data-driven query translation mechanism for mapping a polygen query into a set of local queries dynamically is presented.
Lecture notes in computer scienceUsing Semantic Web Technologies for Representing E-science Provenance
153 Citations2004Jun Zhao, Chris Wroe +4 more
This work explores the use of Semantic Web technologies such as RDF, and ontologies to support its representation and used existing initiatives such as Jena and LSID, to generate and store such material.
MONDRIAN: Annotating and Querying Databases through Colors and Blocks
121 Citations2006Floris Geerts, Anastasios Kementsietsidis +1 more
An annotation-oriented data model for the manipulation and querying of both data and annotations, which allows for the specification of annotations on sets of values and for effectively querying the information on their association is introduced.
A Framework for Collecting Provenance in Data-Centric Scientific Workflows
119 Citations2006Yogesh Simmhan, Beth Plale +1 more
A framework, based on a loosely-coupled publish-subscribe architecture for propagating provenance activities, satisfies the needs of detailed provenance collection while a performance evaluation of a prototype finds a minimal performance overhead.
Lecture notes in computer scienceRecording and Reasoning over Data Provenance in Web and Grid Services
104 Citations2003Martin Szomszor, Luc Moreau
This work proposes an infrastructure level support for a provenance recording capability for service-oriented architectures such as the Grid and Web Services and provides a mechanism by which provenance is used to determine whether previous computed results are still up to date.
Lecture notes in computer scienceA Model for User-Oriented Data Provenance in Pipelined Scientific Workflows
99 Citations2006Shawn Bowers, Timothy McPhillips +3 more
This work develops a simple provenance model capable of supporting a wide range of scientific use cases even for complex models of computation such as process networks, and is capable of reconstructing complete data and invocation dependency graphs for a workflow run.
Reconciling while tolerating disagreement in collaborative data sharing
93 Citations2006Nicholas E. Taylor, Zachary G. Ives
This paper develops and presents a fully decentralized model of collaborative data sharing, in which participants publish their data on an ad hoc basis and simultaneously reconcile updates with those published by others, and defines a consistency semantics for database instances under this model of disagreement.
ORCHESTRA : Rapid, collaborative sharing of dynamic data
92 Citations2005Zachary G. Ives, Nitin Khandelwal +2 more
This paper describes the initial design and prototype of the ORCHESTRA system, which focuses on managing disagreement among multiple data representations and instances and represents an important evolution of the concepts of peer-to-peer data sharing, which considers revision, disagreement, authority, and intermittent participation.
Lecture notes in computer scienceContainment of Relational Queries with Annotation Propagation
45 Citations2004Wang-Chiew Tan
This work says that a query is annotation-contained in another if the annotated output of the former is contained in the latter on every possible annotated input databases, and shows that annotation-containment is a more refined notion in general.
Lecture notes in computer scienceProvenance-Based Validation of E-Science Experiments
42 Citations2005Sylvia C. Wong, Simon Miles +3 more
Lecture notes in computer scienceOn the Expressiveness of Implicit Provenance in Query and Update Languages
41 Citations2006Peter Buneman, James Cheney +1 more