TREC 2005 Spam Track Overview

Published 1 January 2005

Gordon V. Cormack

Citations91

TL;DR

TREC’s Spam Track uses a standard testing framework that presents a set of chronologically ordered email messages a spam filter for classification and four different forms of user feedback are modeled, intended to model a user reading email from time to time and perhaps not diligently reporting the filter's errors.

Abstract

TREC’s Spam Track uses a standard testing framework that presents a set of chronologically ordered email messages a spam filter for classification. In the filtering task, the messages are presented one at at time to the filter, which yields a binary judgement (spam or ham [i.e. non-spam]) which is compared to a humanadjudicated gold standard. The filter also yields a spamminess score, intended to reflect the likelihood that

Keywords

Computer Science

Journal of Machine Learning ResearchSpam Filtering Using Statistical Data Compression Models

247 Citations2006Andrej Bratko, Bogdan Filipič +3 more

A novel approach to spam filtering based on adaptive statistical data compression models that outperform currently established spam filters, as well as a number of methods proposed in previous studies.

now publishers, Inc. eBooksEmail Spam Filtering: A Systematic Review

180 Citations2008Gordon V. Cormack

Statistical precision of information retrieval evaluation

107 Citations2006Gordon V. Cormack, Thomas R. Lynam

The model accurately predicts the degree of concordance between system results on randomly selected halves of the TREC-6 ad hoc corpus and advances a framework for statistical evaluation that uses other sources of chance variation as a source of input for meta-analysis techniques.

Online Active Learning Methods for Fast Label-Efficient Spam Filtering.

94 Citations2007D. Sculley

This work investigates an online active learning scenario where the filter is exposed to a stream of messages which must be classified one at a time, and shows that online activeLearning can dramatically reduce labeling and training costs with negligible additional overhead while maintaining high levels of classification performance.

Spam Corpus Creation for TREC.

84 Citations2005Gordon V. Cormack, Thomas R. Lynam

This work is concerned here with the creation of appropriate corpora for use with the TREC Spam Filter Evaluation Toolkit, which is free software that, given a corpus and a filter, automatically runs the filter on each message in the corpus, compares the result to the gold standard for the corpus and reports effectiveness measures with 95% confidence limits.

Text REtrieval ConferenceUniversity of Waterloo Participation in the TREC 2007 Spam Track

30 Citations2007Gordon V. Cormack

This is the first year that this team has submitted runs to TREC (Cormack is track coordinator), and in parallel with running the track, two new filtering methods are investigated, inspired by the excellent results that others have shown in the task.

Text REtrieval ConferenceRelaxed Online SVMs in the TREC Spam Filtering Track.

20 Citations2007D. Sculley, Gabriel Wachman

It is hypothesized that MR antagonism with eplerenone may be beneficial in patients with acute coronary syndromes, as the mechanisms operative in acute coronary Syndromes (ACS), including inflammation, altered hemostasis, and endothelial dysfunction, overlap significantly with those seen in the EPHESUS patient population.

Semi-supervised spam filtering

12 Citations2008Mona Mojdeh, Gordon V. Cormack

Differences between the ECML/PKDD and TREC data sets and methodologies are investigated, and methods like self-training and transductive support vector machines yield inferior classifiers to those constructed using supervised learning on the labeled data alone.

Three Non-Bayesian Methods of Spam Filtration: CRM114 at TREC 2007.

3 Citations2007Mamoru Kato, Joseph Langeway +2 more

The results show that the SVM has a spamfiltering accuracy of about a factor of two to three better accuracy than the OSB system, that substring compression is somewhat more accurate than OSB, and that bit entropy is somewhat less accurate for the TREC 2007 test sets.