Planning meets Data Cleansing
Proceedings of the International Conference on Automated Planning and SchedulingPublished 11 May 2014Open access
Roberto Boselli, Mirko Cesarini, Fabio Mercorio, Mario Mezzanzanica
Citations24
SJR score0.60
SNIP1.39
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
The concept of cost-optimal Universal Cleanser — a collection of cleansing actions for each data inconsistency — is formalised as a planning problem and a motivating government application in which it has been used is presented.
Abstract
One of the motivations for research in data quality is to automatically identify cleansing activities, namely a sequence of actions able to cleanse a dirty dataset, which today are often developed manually by domain-experts. Here we explore the idea that AI Planning can contribute to identify data inconsistencies and automatically fix them. To this end, we formalise the concept of cost-optimal Universal Cleanser — a collection of cleansing actions for each data inconsistency — as a planning problem. We present then a motivating government application in which it has be used.
Keywords
Computer ScienceDecision Sciences
Applied Longitudinal Data Analysis
10,585 Citations2003J. David Singer, John B. Willett
Journal of the American Statistical AssociationApplied Longitudinal Data Analysis: Modeling Change and Event Occurrence
1,958 Citations2005Carolyn J. Anderson
Data Quality : Concepts, Methodologies and Techniques
666 Citations2006Carlo Batini, Monica Scannapieco
NADEEF
300 Citations2013Michele Dallachiesa, Amr Ebaid +5 more
NADEEF is presented, an extensible, generalized and easy-to-deploy data cleaning platform that is designed in a way to allow cleaning algorithms to cope with multiple rules holistically, i.e. detecting and repairing data errors without differentiating between various types of rules.
AI MagazineFF: The Fast-Forward Planning System
241 Citations2001Jöerg Hoffmann
The algorithmic techniques used in FF in comparison to hsp are described and their benefits in terms of run-time and solution-length behavior are evaluated.
On approximating optimum repairs for functional dependency violations
209 Citations2009Solmaz Kolahi, Laks V. S. Lakshmanan
An approximation algorithm is presented that for a fixed set of functional dependencies and an arbitrary input inconsistent database, produces a repair whose distance to the database is within a constant factor of the optimum repair distance.
Proceedings of the VLDB EndowmentTowards certain fixes with editing rules and master data
198 Citations2010Wenfei Fan, Jianzhong Li +3 more
ACM Transactions on Database SystemsEfficient checking of temporal integrity constraints using bounded history encoding
191 Citations1995Jan Chomicki
An efficient implementation method for temporal integrity constraints formulated in Past Temporal Logic is presented, where every database state is extended with auxiliary relations that contain the historical information necessary for checking constraints.
Latent Markov Models for Longitudinal Data
182 Citations2012Francesco Bartolucci, Alessio Farcomeni +1 more
This book discusses Latent Markov Modeling as a guide to Bayesian inference via reversible jump, and its applications include selection and hypothesis testing, and modeling and inference of latent variable models and their applications.
Proceedings of the VLDB EndowmentThe LLUNATIC data-cleaning framework
178 Citations2013Floris Geerts, Giansalvatore Mecca +2 more
This paper proposes a new semantics for repairs, and a chase-based algorithm to compute minimal solutions, and implemented the framework in a DBMS-based prototype, and results confirm its good scalability and superior quality in computing repairs.
Don't be SCAREd
142 Citations2013Mohamed Yakout, Laure Berti‐Équille +1 more
A novel approach combining machine learning and likelihood methods for cleaning dirty databases by value modification based on maximizing the likelihood of replacement data given the data distribution, which can be modeled using statistical machine learning techniques.
The VLDB JournalTowards certain fixes with editing rules and master data
124 Citations2011Wenfei Fan, Jianzhong Li +3 more
A framework and an algorithm to find certain fixes, based on master data, a notion of certain regions, and a class of editing rules are presented, by interacting with the users to ensure that one of the certain regions is correct.
Proceedings of the International Conference on Automated Planning and SchedulingUPMurphi: A Tool for Universal Planning on PDDL+ Problems
107 Citations2009Giuseppe Della Penna, Daniele Magazzeni +2 more
This paper proposes the use of explicit model checking-based techniques to solve universal planning problems on hardly-approachable domains, including hybrid systems with possibly a nonlinear continuous dynamics.
Introduction to information quality
67 Citations2016Ron S. Kenett, Galit Shmueli
Lecture notes in computer scienceOn the Computational Complexity of Minimal-Change Integrity Maintenance in Relational Databases
59 Citations2005Jan Chomicki, Jerzy Marcinkowski
This work focuses on two basic computational issues: repair checking and consistent query answers, delineating the boundary between the tractable and the intractable, and sheds light on the computational feasibility of minimal-change integrity maintenance.
Data-centric systems and applicationsIntroduction to Information Quality
46 Citations2016Carlo Batini, Monica Scannapieco
The goal of this chapter is to show and discuss the perspectives that make data and information (D&I) quality an issue worth being investigated and understood.
Lecture notes in computer scienceData Quality through Model Checking Techniques
17 Citations2011Mario Mezzanzanica, Roberto Boselli +2 more
The Robust Data Quality Analysis is introduced, which exploits formal methods to support Data Quality Improvement Processes and has proved successful, by giving insights on the data quality levels and by providing suggestions on how to ameliorate the overall data quality process.
Quality & QuantityClassification of longitudinal career paths
15 Citations2011Pietro Giorgio Lovaglio, Mario Mezzanzanica
Lecture notes in computer scienceInconsistency Knowledge Discovery for Longitudinal Data Management: A Model-Based Approach
15 Citations2013Roberto Boselli, Mirko Cesarini +2 more
In the last years, the growing diffusion of IT-based services has given a rise to the use of huge masses of data, however, using data for analytical and decision making purposes requires to perform several tasks, e.g. data cleansing, data filtering, data aggregation and synthesis, etc.
AI CommunicationsModel checking for universal planning in deterministic and non-deterministic domains
11 Citations2013Fabio Mercorio
The V-UPMurphi tool is developed, which applies a disk-based planning approach for efficient storage during the plan synthesis, and an algorithm to solve cost-optimal strong planning problems in non-deterministic contexts is developed.