Symbiotic filtering for spam email detection
Expert Systems with ApplicationsPublished 20 February 2011Open access
C. M. C. Lopes, Paulo Cortez, Pedro Sousa, Miguel Rocha, Miguel Rio
Citations32
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
A novel spam filtering technique called Symbiotic Filtering (SF) that aggregates distinct local filters from several users to improve the overall performance of spam detection and is robust against both dictionary and focused contamination attacks is presented.
Abstract
This paper presents a novel spam filtering technique called Symbiotic Filtering (SF) that aggregates distinct local filters from several users to improve the overall perfor- mance of spam detection. SF is an hybrid approach combining some features from both Collaborative (CF) and Content-Based Filtering (CBF). It allows for the use of social networks to personalize and tailor the set of filters that serve as input to the filtering. A comparison is performed against the commonly used Naive Bayes CBF algorithm. Several experiments were held with the well-known Enron data, under both fixed and incremental symbiotic groups. We show that our system is competitive in performance and is robust against both dictionary and focused con- tamination attacks. Moreover, it can be implemented and deployed with few effort and low communication costs, while assuring privacy.
Keywords
Computer Science
Pattern Recognition LettersAn introduction to ROC analysis
21,254 Citations2005Tom Fawcett
The purpose of this article is to serve as an introduction to ROC graphs and as a guide for using them in research.
Journal of Statistical SoftwareText Mining Infrastructure in<i>R</i>
1,145 Citations2008Ingo Feinerer, Kurt Hornik +1 more
The tm package is presented which provides a framework for text mining applications within R and techniques for count-based analysis methods, text clustering, text classification and string kernels are presented.
Expert Systems with ApplicationsA review of machine learning approaches to Spam filtering
531 Citations2009Thiago Guzella, Walmir M. Caminhas
A comprehensive review of recent developments in the application of machine learning algorithms to Spam filtering, focusing on both textual- and image-based approaches concludes that while important advancements have been made in the last years, several aspects remain to be explored, especially under more realistic evaluation settings.
ACM SIGCOMM Computer Communication ReviewUnderstanding the network-level behavior of spammers
520 Citations2006Anirudh Ramachandran, Nick Feamster
Conference on Email and Anti-SpamSpam Filtering with Naive Bayes - Which Naive Bayes?
480 Citations2006Vangelis Metsis, Ion Androutsopoulos +1 more
An experimental procedure that emulates the incremental training of person- alized spam filters is adopted, and roc curves that allow us to compare the dierent versions of nb over the entire tradeo between true positives and true negatives are plotted.
Edinburgh Research Explorer (University of Edinburgh)Exploiting machine learning to subvert your spam filter
325 Citations2008Blaine Nelson, Marco Barreno +7 more
This paper shows how an adversary can exploit statistical machine learning, as used in the SpamBayes spam filter, to render it useless--even if the adversary's access is limited to only 1% of the training messages.
Understanding the network-level behavior of spammers
240 Citations2006Anirudh Ramachandran, Nick Feamster
It is found that most spam is being sent from a few regions of IP address space, and that spammers appear to be using transient "bots" that send only a few pieces of email over very short periods of time.
ScholarWorks@UMassAmherst (University of Massachusetts Amherst)Automatic Categorization of Email into Folders: Benchmark Experiments on Enron and SRI Corpora
154 Citations2005Ron Bekkerman, Andrew McCallum +1 more
An extensive benchmark study of email foldering using two large corpora of real-world email messages and foldering schemes: one from former Enron employees, another from participants in an SRI research project.
IEEE Transactions on Neural NetworksData strip mining for the virtual design of pharmaceuticals with neural networks
146 Citations2000Robert Kewley, Mark J. Embrechts +1 more
Elimination of variables through neural network sensitivity analysis and predicting performance through model cross-validation allows the analyst to reduce the number of inputs and improve the model's predictive ability at the same time.
Networked Systems Design and ImplementationRE: reliable email
132 Citations2006Scott Garriss, Michael Kaminsky +4 more
Reliable Email is presented, a new whitelisting system that incurs zero false positives among socially connected users and can prevent up to 88% of the false positives incurred by a widely deployed email rejection system, at modest computational cost.
Expert Systems with ApplicationsApplying lazy learning algorithms to tackle concept drift in spam filtering
120 Citations2006Florentino Fdez‐Riverola, Eva Iglesias +3 more
The results obtained are very promising and back up the idea that instance-based reasoning systems can offer a number of advantages tackling concept drift in dynamic problems, as in the case of the anti-spam filtering domain.
Communications of the ACMData mining standards initiatives
86 Citations2002Robert L. Grossman, Mark F. Hornick +1 more
Lacking standards for statistical and data mining models, applications cannot leverage the benefits of data mining and suffer from the consequences of poor quality models.
ComputerCollaborative Spam Filtering Using E-Mail Networks
73 Citations2006Joseph S. Kong, B. Rezaei +3 more
A distributed spam-filtering system that leverages e-mail networks' topological properties is more efficient and scalable than client-server-based solutions.
Personalised, Collaborative Spam Filtering
71 Citations2004Alan Gray, Mads Haahr
This paper presents an architecture for personalised, collaborative spam filtering and describes the design and implementation of proof-of-concept, peer-to-peer, signature-based system based on the architecture.
arXiv (Cornell University)Collaborative Ensemble Learning: Combining Collaborative and Content-Based Information Filtering via Hierarchical Bayes
69 Citations2012Kai Yu, Anton Schwaighofer +3 more
A novel probabilistic framework to unify CF and CBF, named collaborative ensemble learning, which combines a society of users' preferences to predict an active user's preferences and is applicable to problems typically solved via classical hierarchical Bayes, like multisensor fusion and multitask learning.
CERN Document Server (European Organization for Nuclear Research)Let Your CyberAlter Ego Share Information and Manage Spam
61 Citations2005Joseph S. Kong, P. Oscar Boykin +3 more
Communications of the ACMSpamalytics
52 Citations2009Chris Kanich, Christian Kreibich +5 more
The best way to measure spam is to be a spammer, and this study makes use of an existing spamming botnet, by infiltrating the botnet parasitically and convincing it to modify a subset of the spam it already sends, thereby directing any interested recipients to Web sites under the authors' control.
Combining email models for false positive reduction
50 Citations2005Shlomo Hershkop, Salvatore J. Stolfo
A new method to compare multiple and combined classifiers, and show how it differs from past work is introduced, which analyzes the relative gain and maximum possible accuracy that can be achieved for certain combinations of classifiers to automatically choose the best combination.
Partitioned logistic regression for spam filtering
45 Citations2008Ming‐Wei Chang, Wen-tau Yih +1 more
This paper proposes a novel hybrid model, partitioned logistic regression, which separates the original feature space into several disjoint feature groups and shows that this model is better both theoretically and empirically.
Personalized Spam Filtering with Semi-supervised Classifier Ensemble
42 Citations2006Victor Cheng, C.H. Li
Experimental results show that the proposed approaches can significantly increases the classification accuracy in spam filtering.
Distributed quota enforcement for spam control
37 Citations2006Michael Walfish, J. D. Zamfirescu +3 more
DQE's enforcer occupies a point in the design spectrum notable for simplicity: mutually untrusting nodes implement a storage abstraction but avoid neighbor maintenance, replica maintenance, and heavyweight cryptography.
An economic answer to unsolicited communication
30 Citations2004Thede Loder, Marshall Van Alstyne +1 more
An alternative approach to spam based on economic rather than technological or regulatory screening mechanisms is explored, and it is shown this approach leaves recipients better off than even an idealized or "perfect" filter that costs nothing and makes no mistakes.
ALPACAS: A Large-Scale Privacy-Aware Collaborative Anti-Spam System
27 Citations2008Zhenyu Zhong, Lakshmish Ramaswamy +1 more
The proposed ALPACAS framework is a privacy-aware framework for collaborative spam filtering that provides enhanced privacy guarantees to the participating entities and makes two unique contributions, including a feature-preserving message transformation technique that is highly resilient against the latest kinds of spam attacks.
Expert Systems with ApplicationsA collaborative anti-spam system
24 Citations2008Gu-Hsin Lai, Chia-Mei Chen +2 more
Rough set theory is used to generate spam rules and XML format for exchanging spam rules are used and the spam rule management is achieved by reinforcement learning approach to improve the accuracy and efficiency of spam filter.
Adaptive Spam Filtering Using Only Naive Bayes Text Classifiers
20 Citations2008Aris Kosmopoulos, Γεώργιος Παλιούρας +1 more
This paper puts Naive Bayes filters to the test, against potentially more elaborate spam filters that will participate in the ceas 2008 challenge, and proposes a simple active learning method for adapting the filter, under partial online supervision.
"May I borrow your filter?" Exchanging filters to combat spam in a community
18 Citations2006Anurag Garg, Roberto Battiti +1 more
This paper presents a mechanism through which members of a virtual community can exchange information to combat spam, and shows that exchanging spam filters provides an alternative method to improve spam filtering performance.
Lecture notes in computer scienceA Comparative Impact Study of Attribute Selection Techniques on Naïve Bayes Spam Filters
15 Citations2008José R. Méndez, I. Cid +3 more
A comparative study about the performance of five well-known feature selection techniques when they are applied in conjunction with four different types of Naive Bayes classifier shows the relevance of choosing an appropriate feature selection technique in order to obtain the most accurate results.