Mining Complex Data Streams: Discretization, Attribute Selection and Classification
Journal of Advances in Information TechnologyPublished 1 August 2013Open access
Dewan Md. Farid, Chowdhury Mofizur Rahman
Citations42
SJR quartileQ3
SJR score0.30
SNIP0.63
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
The proposed discretization algorithm finds the possible cut points in continuous attributes using information gain heuristic and naive Bayesian classifier that can separate the class distributions.
Abstract
Due to the large volume of data set as well as complex and dynamic properties of data instances, several data mining algorithms have been applied for mining complex data streams in the last decades. Now a day, knowledge extraction from data streams is getting more complex because the structure of the data instance does not match the attribute values when considering the tabulated data, texts, web, images or videos etc. In this paper, we address some difficulties of mining complex data streams such as dealing with continuous attributes, input attribute selection, and classifier construction. The proposed discretization algorithm finds the possible cut points in continuous attributes using information gain heuristic and naïve Bayesian classifier that can separate the class distributions. We evaluate the proposed algorithms on several benchmark data sets from UCI machine learning repository. The experimental results demonstrate that the proposed method improves the quality of discretization of continuous attributes and scales up the classification accuracy for different types of classification problem.
Keywords
Computer Science
Machine LearningInduction of Decision Trees
14,815 Citations1986J. R. Quinlan
This paper summarizes an approach to synthesizing decision trees that has been used in a variety of systems, and it describes one such system, ID3, in detail, which is described in detail.
International Joint Conference on Artificial IntelligenceMulti-Interval Discretization of Continuous-Valued Attributes for Classification Learning
2,832 Citations1993Usama M. Fayyad, Keki B. Irani
This paper addresses the use of the entropy minimization heuristic for discretizing the range of a continuous-valued attribute into multiple intervals.
Elsevier eBooksSupervised and Unsupervised Discretization of Continuous Features
1,931 Citations1995James Dougherty, Ron Kohavi +1 more
Binning, an unsupervised discretization method, is compared to entropy-based and purity-based methods, which are supervised algorithms, and it is found that the performance of the Naive-Bayes algorithm significantly improved when features were discretized using an entropy- based method.
Journal of Artificial Intelligence ResearchImproved Use of Continuous Attributes in C4.5
1,847 Citations1996J. R. Quinlan
A reported weakness of C4.5 in domains with continuous attributes is addressed by modifying the formation and evaluation of tests on continuous attributes with an MDL-inspired penalty, leading to smaller decision trees with higher predictive accuracies.
Data Mining and Knowledge DiscoveryDiscretization: An Enabling Technique
979 Citations2002Huan Liu, Farhad Hussain +2 more
This paper aims at a systematic study of discretization methods with their history of development, effect on classification, and trade-off between speed and accuracy.
ACM Transactions on Information and System SecurityA framework for constructing features and models for intrusion detection systems
964 Citations2000Wenke Lee, Salvatore J. Stolfo
A novel framework, MADAM ID, for Mining Audit Data for Automated Models for Instrusion Detection, which uses data mining algorithms to compute activity patterns from system audit data and extracts predictive features from the patterns.
Machine LearningOn the Handling of Continuous-Valued Attributes in Decision Tree Generation
680 Citations1992Usama M. Fayyad, Keki B. Irani
The result serves to give a better understanding of the entropy measure, to point out that the behavior of the information entropy heuristic possesses desirable properties that justify its usage in a formal sense, and to improve the efficiency of evaluating continuous-valued attributes for cut value selection.
Computers & SecurityFeature deduction and ensemble design of intrusion detection systems
602 Citations2004Srilatha Chebrolu, Ajith Abraham +1 more
This study investigated the performance of two feature selection algorithms involving Bayesian networks and Classification and Regression Trees and an ensemble of BN and CART and proposed an hybrid architecture for combining different feature selection algorithm for real world intrusion detection.
Identifying important features for intrusion detection using support vector machines and neural networks
390 Citations2003Andrew H. Sung, Srinivas Mukkamala
This paper applies the technique of deleting one feature at a time to perform experiments on SVMs and neural networks to rank the importance of input features for the DARPA collected intrusion data and shows that SVM-based and neural network based IDSs using a reduced number of features can deliver enhanced or comparable performance.
IEEE Transactions on Pattern Analysis and Machine IntelligenceClass-dependent discretization for inductive learning from continuous and mixed-mode data
241 Citations1995J.Y. Ching, Andrew K. C. Wong +1 more
A new information theoretic discretization method optimized for supervised learning is proposed and described that seeks to maximize the mutual dependence as measured by the interdependence redundancy between the discrete intervals and the class labels, and can automatically determine the most preferred number of intervals for an inductive learning application.
Information FusionEnsemble feature selection with the simple Bayesian classification
158 Citations2003Alexey Tsymbal, Seppo Puuronen +1 more
IEEE Transactions on Electronics Packaging ManufacturingFeature transformation methods in data mining
117 Citations2001Andrew Kusiak
The proposed concept of feature bundling is applied to a data set from the semiconductor industry and shows that the classification accuracy of decision rules generated from data sets with feature bundles is enhanced.
Feature selection using multi-objective genetic algorithms for handwritten digit recognition
114 Citations2003Luiz S. Oliveira, Robert Sabourin +2 more
The use of genetic algorithms for feature selection for handwriting recognition where sensitivity analysis and neural networks are employed to allow the use of a representative database to evaluate fitness and a validation database to identify the subsets of selected features that provide a good generalization.
Lecture notes in computer scienceInduction of recursive Bayesian classifiers
93 Citations1993Pat Langley
A recursive algorithm is described that should outperform the simpler scheme in domains that involve disjunctive concepts, since they violate the independence assumption on which the latter relies.
Elsevier eBooksCompression-Based Discretization of Continuous Attributes
91 Citations1995Bernhard Pfahringer
A global evaluation measure for discretizations based on the so-called Minimum Description Length (MDL) principle from information theory is defined and the efficient algorithmic usage of this measure in the MDL-Disc algorithm is described.
Multivariate discretization of continuous variables for set mining
70 Citations2000Stephen D. Bay
It is argued that this is a sub-optimal approach for knowledge discovery as univariate discretization can destroy hidden patterns in data and presented a bottom up merging algorithm to discretize continuous variables based on this rule.
Fast and accurate feature selection using hybrid genetic strategies
49 Citations2003César Guerra-Salcedo, Stephen Chen +2 more
Results shows that CF/RSC and decision tables are a very good combination when dealing with large feature spaces and also suggests that CHC is better when used for problems with noise added to the features.
Lecture notes in computer scienceMulti-interval discretization methods for decision tree learning
45 Citations1998Petra Perner, Sascha Trautzsch
Four multi-interval discretization methods for induction of decision trees used in dynamic fashion are described based on a histogram based method and a neural net based method (LVQ), which are compared to accuracy of the resulting decision tree and to compactness of the tree.
An iterative improvement approach for the discretization of numeric attributes in Bayesian classifiers
39 Citations1995Michael J. Pazzani
It is shown that the discretization of numeric attributes is critical to successful application of the Bayesian classifier and a new method based on iterative improvement search is proposed and compared to previous approaches, which results in significant reductions in misclassification error and costs.
A Bayes Optimal Approach for Partitioning the Values of Categorical Attributes
37 Citations2005Marc Boullé
This paper proposes a new grouping method MODL founded on a Bayesian approach that results in an evaluation criterion of grouping, which is minimal for the most probable grouping given the data, i.e. the Bayes optimal grouping.
Novel class detection in concept-drifting data stream mining employing decision tree
36 Citations2012Dewan Md. Farid, Chowdhury Mofizur Rahman
The proposed approach for incremental learning of concept drift considers mining, where the streaming data distributions change over time, and build a decision tree model from training dataset, which continuously updates so that the tree represents the most recent concept in data stream.
Intelligent feature extraction for ensemble of classifiers
8 Citations2005Paulo V. W. Radtke, Robert Sabourin +1 more
This paper presents a two-level approach to create ensemble of classifiers based on intelligent feature extraction and multi-objective genetic optimization, which outperforms the single classifier approach.
Improve the quality of supervised discretization of continuous valued attributes in data mining
1 Citations2011Dewan Md. Farid
A new approach for dealing with continuous attributes that improve the quality of discretization as a preprocessing step for decision tree and naïve Bayesian classifier is presented.