Classifying skewed data streams based on reusing data
Published 1 October 2010
Peng Liu, Yong Wang, Lijun Cai, Longbo Zhang
Citations15
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
The method of discovering concept drifting on skewed data streams is introduced and an algorithm for classifying skewed data streams based on reusing data, RDFCSDS (Reuse Data for Classifying Skewed Data Streams) is proposed.
Abstract
Current research community on data streams mining focuses on mining balanced data streams. However, the skewed class distribution appears in many data streams applications. In this paper, we introduce the method of discovering concept drifting on skewed data streams and propose an algorithm for classifying skewed data streams based on reusing data, RDFCSDS (Reuse Data for Classifying Skewed Data Streams). We evaluate RDFCSDS algorithm on Moving Hyperplane data set. The experiment results show that the sampling method based on reusing data works better than the simple sampling method and cluster sampling method on skewed data streams with concept drifting.
Keywords
Computer Science
Pattern Recognition LettersAn introduction to ROC analysis
21,254 Citations2005Tom Fawcett
The purpose of this article is to serve as an introduction to ROC graphs and as a guide for using them in research.
Lecture notes in computer scienceEnsemble Methods in Machine Learning
7,862 Citations2000Thomas G. Dietterich
ACM SIGKDD Explorations NewsletterA study of the behavior of several methods for balancing machine learning training data
4,130 Citations2004Gustavo E. A. P. A. Batista, Ronaldo C. Prati +1 more
This work performs a broad experimental evaluation involving ten methods, three of them proposed by the authors, to deal with the class imbalance problem in thirteen UCI data sets, and shows that, in general, over-sampling methods provide more accurate results than under-sampled methods considering the area under the ROC curve (AUC).
ACM SIGKDD Explorations NewsletterEditorial
2,035 Citations2004Nitesh V. Chawla, Nathalie Japkowicz +1 more
ACM SIGKDD Explorations NewsletterExtreme re-balancing for SVMs
285 Citations2004Bhavani Raskutti, Adam Kowalczyk
There is a consistent pattern of performance differences between one and two-class learning for all SVMs investigated, and these patterns persist even with aggressive dimensionality reduction through automated feature selection.
Systematic data selection to mine concept-drifting data streams
244 Citations2004Wei Fan
A simple, efficient and accurate cross-validation decision tree ensemble method to mine data streams and points out that using old data blindly is not better than "gambling"; in other words, it helps increase the accuracy only if the authors are "lucky."
A General Framework for Mining Concept-Drifting Data Streams with Skewed Distributions
239 Citations2007Jing Gao, Wei Fan +2 more
This paper proposes a new approach to mine data streams by estimating reliable posterior probabilities using an ensemble of models to match the distribution over under-samples of negatives and repeated samples of positives and formally shows some interesting and important properties of the proposed framework.
IEEE Internet ComputingClassifying Data Streams with Skewed Class Distributions and Concept Drifts
124 Citations2008Jing Gao, Bolin Ding +3 more
The authors aim to develop a new approach to classify skewed data streams that uses an ensemble of models to match the distribution over under-samples of negatives and repeated samples of positives.
Positive Unlabeled Learning for Data Stream Classification
119 Citations2009Xiaoli Li, Philip S. Yu +2 more
Experimental results show that LELC is a PU learning method that can effectively address the issues in the data stream environment with significantly better speed and accuracy on capturing concept drift than the existing state-of-the-art PU learning techniques.
Mixture of Expert Agents for Handling Imbalanced Data Sets
82 Citations2003Sotiris Kotsiantis, Panagiotis Pintelas
A systematic study on the various methodologies that have tried to handle skewed class distributions and an experimental study of these methodologies with a proposed mixture of expert agents conclude that such a framework can be a more effective solution to the problem.
OcVFDT
45 Citations2009Chen Li, Shuicheng Yan +1 more
This paper model applications, such as credit fraud detection and intrusion detection, as a one-class data stream classification problem, and proposes the OcVFDT (One-class Very Fast Decision Tree) algorithm, which has excellent classification performance.
One-Class Classification of Text Streams with Concept Drift
33 Citations2008Shuicheng Yan, Xue Li +1 more
This paper considers the problem of one-class classification on text stream with respect to concept drift where a large volume of documents arrives at a high speed and with change of user interests and data distribution and proposes a stacking style ensemble-based approach that outperforms all other window-based approaches.
Studies in computational intelligenceMining Data Streams with Skewed Distribution by Static Classifier Ensemble
30 Citations2009Yi Wang, Yang Zhang +1 more
An clustering-sampling based ensemble algorithm with weighted majority voting for learning skewed data streams and the experiment results show that clusters outperforms under-sampled, and that compared with single window, the proposed ensemble based algorithm has better classification performance.