HotMiner: Discovering Hot Topics from Dirty Text

This chapter presents a novel approach to discovering hot topics of customers’ problems by mining the logs of customer support centers using a toolbox HotMiner and includes a variety of techniques that either directly handle some of these anomalies or that are robust in spite of them.

Abstract

For companies with websites that contain millions of documents available to their customers, it is critical to identify their customers' hottest information needs along with their associated documents. This valuable information gives companies the potential of reducing costs and being more competitive and responsive to their customers' needs. In particular, technical support centers could drastically lower the number of support engineers by knowing the topics of their customers' hot problems (i.e., hot topics), and making them available on their websites along with links to the corresponding solutions documents so that customers could efficiently find the right documents to self-solve their problems. In this chapter we present a novel approach to discovering hot topics of customers' problems by mining the logs of customer support centers. Our technique for search log mining discovers hot topics that match the user's perspective, which often is different from the topics derived from document content categorization' methods. Our techniques to mine case logs include extracting relevant sentences from cases to conform case excerpts which are more suitable for hot topics mining. In contrast to most text mining work, our approach deals with dirty text containing typos, adhoc abbreviations, special symbols, incorrect use of English grammar, cryptic tables and ambiguous and missing punctuation. It includes a variety of techniques that either directly handle some of these anomalies or that are robust in spite of them. In particular, we have developed a postfiltering technique to deal with the effects of noisy clickstreams due to random clicking behavior, a Thesaurus Assistant to help in the generation of a thesaurus of "dirty" variations of words that is used to normalize the terminology, and a Sentence Identifier with the capability of eliminating code and tables. The techniques that compose our approach have been implemented as a toolbox HotMiner, which has been used in experiments on logs from Hewlett-Packard's Customer Support Center