A log mining approach to failure analysis of enterprise telephony systems

Published 1 June 2008

Chinghway Lim, Navjot Singh, Shalini Yajnik

DOI10.1109/dsn.2008.4630109

Citations94

TL;DR

The experiences with applying log mining techniques to characterize the behavior of large enterprise telephony systems are discussed and solutions are proposed that are general enough to be applicable to other systems logs and can be packaged into automated tools for log analysis.

Abstract

Log monitoring techniques to characterize system and user behavior have gained significant popularity. Some common applications of study of systems logs are syslog mining to detect and predict system failure behavior, web log mining to characterize web usage patterns, and error/debug log analysis for detecting anomalies. In this paper, we discuss our experiences with applying log mining techniques to characterize the behavior of large enterprise telephony systems. We aim to detect, and in some cases, predict system anomalies. We describe the problems encountered in the study of such logs and propose some solutions. The key differentiator of our solutions is the use of individual message frequencies to characterize system behavior and the ability to incorporate domain-specific knowledge through user feedback. The techniques that we propose are general enough to be applicable to other systems logs and can easily be packaged into automated tools for log analysis.

Keywords

Computer Science

What Supercomputers Say: A Study of Five System Logs

575 Citations2007Adam J. Oliner, Jon Stearley

This paper examines system logs from five supercomputers with the aim of providing useful insight and direction for future research into the use of such logs, and proposes a simpler and more effective filtering algorithm.

Data & Knowledge EngineeringMining interesting knowledge from weblogs: a survey

319 Citations2004Federico Michele Facca, Pier Luca Lanzi

A survey of the recent developments in Web Usage Mining, that area of Web Mining which deals with the extraction of interesting knowledge from logging information produced by Web servers, is presented.

Critical event prediction for proactive management in large-scale computer clusters

265 Citations2003R.K. Sahoo, Adam J. Oliner +6 more

An attempt to build a proactive prediction and control system for large clusters using time-series methods, rule-based classification algorithms and Bayesian network models to predict system performance parameters (SARs) with a high degree of accuracy.

Mining partially periodic event patterns with unknown periods

231 Citations2002Sheng Ma, Joseph L. Hellerstein

This work develops two algorithms for mining p-patterns based on the order in which the aforementioned sub-tasks are performed: the period-first algorithm and the association- first algorithm, and develops a novel approach based on a chi-squared test.

IEEE Transactions on ReliabilityError log analysis: statistical modeling and heuristic trend analysis

184 Citations1990Tse‐Chieh Lin, Daniel P. Siewiorek

The authors present the results of an analysis that demonstrates that the log is composed of at least two error processes: transient and intermittent, and it is shown that the DFT can extract intermittent errors from the error log and uses only one fifth of the error-log entry points required for failure prediction.

Lecture notes in computer scienceA Breadth-First Algorithm for Mining Frequent Patterns from Event Logs

84 Citations2004Risto Vaarandi

The properties of eventlog data are discussed, the suitability of popular mining algorithms for processing event log data is analyzed, and an efficient algorithm for mining frequent patterns from event logs is proposed.

IEEE Transactions on ComputersAnalysis and modeling of correlated failures in multicomputer systems

77 Citations1992Dong Tang, Ravishankar K. Iyer

Based on the measurements from two DEC VAX-cluster multicomputer systems, the issue of correlated failures is addressed and two validated models, the c- dependent model and the p-dependent model, are developed to evaluate the dependability of systems with correlated failures.

Communications of the ACMVisualizing online activity

51 Citations2001Stephen G. Eick

This paper describes how the dot-com revolution established the e-channel as a critical component of corpo-ration/customer interaction and how information visualization tools for displaying Web site structure, navigation, paths, flows, and activity are developed.

Error/failure analysis using event logs from fault tolerant systems

38 Citations2002I. Lee, Ravishankar K. Iyer +1 more

A methodology for the analysis of automatically generated event logs from fault tolerant systems is presented and it is found that the number of errors is small, even though the measurement period is relatively long, which reflects the high dependability of the measured systems.