Machine Learning Approach for IP-Flow Record Anomaly Detection
Lecture notes in computer sciencePublished 1 January 2011Open access
Cynthia Wagner, Jérôme François, Radu State, Thomas Engel
Citations90
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This paper presents an approach that leverages support vector machines in order to analyze large volumes of Netflow records using a special kernel function, that takes into account both the contextual and the quantitative information of Net flow records.
Abstract
Part 1: Anomaly Detection
Keywords
Computer Science
TechnometricsStatistical Learning Theory
26,913 Citations1999Yuhai Wu, Vladimir Vapnik
Presenting a method for determining the necessary and sufficient conditions for consistency of learning process, the author covers function estimates from small data pools, applying these estimations to real-life problems, and much more.
Data Mining and Knowledge DiscoveryA Tutorial on Support Vector Machines for Pattern Recognition
16,433 Citations1998Christopher J. C. Burges
There are several arguments which support the observed high accuracy of SVMs, which are reviewed and numerous examples and proofs of most of the key theorems are given.
The MIT Press eBooksLearning with Kernels
9,548 Citations2001Bernhard Schölkopf, Alexander J. Smola
Learning with Kernels provides an introduction to SVMs and related kernel methods that provide all of the concepts necessary to enable a reader equipped with some basic mathematical knowledge to enter the world of machine learning using theoretically well-founded yet easy-to-use kernel algorithms.
Neural ComputationEstimating the Support of a High-Dimensional Distribution
5,946 Citations2001Bernhard Schölkopf, John Platt +3 more
The algorithm is a natural extension of the support vector algorithm to the case of unlabeled data by carrying out sequential optimization over pairs of input patterns and providing a theoretical analysis of the statistical performance of the algorithm.
IEEE ExpertData mining and knowledge discovery: making sense out of data
4,643 Citations1996U.M. Feyyad
Find loads of the data mining and knowledge discovery making sense out of data book catalogues in this site as the choice of you visiting this page.
Studies in fuzziness and soft computingSupport Vector Machines: Theory and Applications
1,496 Citations2005Lipo Wang
This chapter discusses Kernel Discriminant Learning with Application to Face Recognition, Fast Color Texture-based Object Detection in Images: Application to License Plate Localization, and more.
Mining anomalies using traffic feature distributions
1,058 Citations2005Anukool Lakhina, Mark Crovella +1 more
It is argued that the distributions of packet features observed in flow traces reveals both the presence and the structure of a wide range of anomalies, and that using feature distributions, anomalies naturally fall into distinct and meaningful clusters that can be used to automatically classify anomalies and to uncover new anomaly types.
BLINC
998 Citations2005Thomas Karagiannis, Konstantina Papagiannaki +1 more
This work presents a fundamentally different approach to classifying traffic flows according to the applications that generate them, based on observing and identifying patterns of host behavior at the transport layer and demonstrates the effectiveness of this approach on three real traces.
The MIT Press eBooksConvolution Kernels for Natural Language
812 Citations2002Michael J. Collins, Nigel Duffy
It is shown how a kernel over trees can be applied to parsing using the voted perceptron algorithm, and experimental results on the ATIS corpus of parse trees are given.
The VLDB JournalA new intrusion detection system using support vector machines and hierarchical clustering
395 Citations2006Latifur Khan, Mamoun Awad +1 more
This paper presents a new approach of combination of SVM and DGSOT, which starts with an initial training set and expands it gradually using the clustering structure produced by the D GSOT algorithm, which has proved to overcome the drawbacks of traditional hierarchical clustering algorithms.
Towards highly reliable enterprise network services via inference of multi-level dependencies
373 Citations2007Paramvir Bahl, Ranveer Chandra +4 more
An Inference Graph model is introduced, which is well-adapted to user-perceptible problems rooted in conditions giving rise to both partial service degradation and hard faults, and takes into account multi-level structure, which leads to a 30% improvement in fault localization, as compared to two-level approaches.
Building a better NetFlow
355 Citations2004Cristian Estan, Ken Keys +2 more
Adaptive NetFlow is proposed, deployable through an update to router software, which addresses many shortcomings of NetFlow by dynamically adapting the sampling rate to achieve robustness without sacrificing accuracy.
Information Systems FrontiersAn SVM-based machine learning method for accurate internet traffic classification
228 Citations2008Ruixi Yuan, Li Zhu +2 more
A machine learning method based on SVM (supporting vector machine) is proposed in this paper for accurate Internet traffic classification that classifies the Internet traffic into broad application categories according to the network flow parameters obtained from the packet headers.
Mining in a data-flow environment
223 Citations1999Wenke Lee, Salvatore J. Stolfo +1 more
It is shown that in order to minimize the time required in using the classification models in a real-time environment, the “necessary conditions” associated with the lowcost features can be exploited to determine whether some high-cost features need to be computed and the corresponding classification rules need to been checked.
Lecture notes in computer scienceA Labeled Data Set for Flow-Based Intrusion Detection
146 Citations2009Anna Sperotto, Ramin Sadre +2 more
This paper proposes the first publicly available, labeled data set for flow-based intrusion detection, and aims to be realistic, i.e., representative of real traffic and complete from a labeling perspective.
Anomaly intrusion detection using one class SVM
144 Citations2005Yanxin Wang, Johnny Wong +1 more
This work extends kernel methods to intrusion detection domain by introducing a new family of kernels suitable for intrusion detection, combined with an unsupervised learning method - one-class support vector machine.
BioinformaticsA tree kernel to analyse phylogenetic profiles
132 Citations2002Jean‐Philippe Vert
It is shown how the phylogenetic profiles can be mapped to a high-dimensional vector space which incorporates evolutionarily relevant information, and an algorithm to compute efficiently the inner product in that space, which is called the tree kernel.
ACM SIGCOMM Computer Communication ReviewTowards highly reliable enterprise network services via inference of multi-level dependencies
121 Citations2007Paramvir Bahl, Ranveer Chandra +4 more
Lecture notes in computer scienceMICAI 2006: Advances in Artificial Intelligence
96 Citations2006MICAI 2006 Apizaco, Gelbukh, Alexander 1962-
NetFlow
85 Citations2002Robin Sommer, Anja Feldmann
This paper asks the question how and how accurately can one infer information from NetFlow, and is interested in TCP connection summaries and accurately aggregated packet and byte counts.
Lecture notes in computer scienceQuantifying the Extent of IPv6 Deployment
59 Citations2009Elliott Karpilovsky, Alexandre Gerber +3 more
The study suggests that from the vantage points, current IPv6 deployment appears somewhat experimental, and that the growth of IPv6 allocations, routing announcements, and traffic volume probably indicate more operators and users are preparing themselves for the transition to IPv6.
FLAME: a flow-level anomaly modeling engine
36 Citations2008Daniela Brauckhoff, Arno Wagner +1 more
This work presents flame, a tool for injection of hand-crafted anomalies into a given background traffic trace that combines the controllability offered by simulation with the realism provided by captured traffic traces, and believes that flame can contribute significantly to the development and evaluation of advanced anomaly detection mechanisms.
Lecture notes in computer sciencePortscan Detection with Sampled NetFlow
25 Citations2009Ignasi Paredes-Oliva, Pere Barlet‐Ros +1 more
It is found that flow sampling is not always better than packet sampling to continue detecting portscans reliably and should be considered as a supplement to packet sampling.
Lecture notes in computer scienceOptimizing Weighted Kernel Function for Support Vector Machine by Genetic Algorithm
9 Citations2006Ha-Nam Nguyen, Syng-Yup Ohn +3 more
The experiments on several clinical datasets such as colon cancer, leukemia cancer, and lung cancer datasets indicate that the proposed weighted kernel function results in higher and more stable classification performance than other kernel functions.
Lecture notes in computer scienceCharacteristics of Denial of Service Attacks on Internet Using AGURI
8 Citations2003Ryo Kaizaki, Osamu Nakamura +1 more
Using the traffic pattern aggregation method, AGURI can monitor the flooding attacks in real network traffic for a long term and is used for the monitoring tools.
PeekKernelFlows
7 Citations2010Cynthia Wagner, Gérard Wagener +3 more
This paper introduces a new method for getting insights into IP related data flows based on a simple visualization technique that leverages kernel functions defined over spatial and temporal aggregated IP flows that was implemented in a visualization tool called PeekKernelFlows.
Game theory driven monitoring of spatial-aggregated IP-Flow records
5 Citations2010Cynthia Wagner, Gérard Wagener +3 more
A new metric that leverages spatially and temporally aggregated IP-flow related information based on a new kernel function that captures both IP address space distribution as well as volume related traffic information is introduced.
NetFlow
4 Citations2002Robin Sommer, Anja Feldmann