Mining Text Using Keyword Distributions
Journal of Intelligent Information SystemsPublished 1 May 1998
Ronen Feldman, Ido Dagan, Haym Hirsh
Citations145
SJR quartileQ2
SJR score0.86
SNIP1.67
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This paper describes the KDT system for Knowledge Discovery in Text, in which documents are labeled by keywords, and knowledge discovery is performed by analyzing the co-occurrence frequencies of the various keywords labeling the documents.
Abstract
Knowledge Discovery in Databases (KDD) focuses on the computerized exploration of large amounts of data and on the discovery of interesting patterns within them. While most work on KDD has been concerned with structured databases, there has been little work on handling the huge amount of information that is available only in unstructured textual form. This paper describes the KDT system for Knowledge Discovery in Text, in which documents are labeled by keywords, and knowledge discovery is performed by analyzing the co-occurrence frequencies of the various keywords labeling the documents. We show how this keyword-frequency approach supports a range of KDD operations, providing a suitable foundation for knowledge discovery and exploration for collections of unstructured text.
Keywords
Computer Science
Elements of Information Theory
37,533 Citations2001Thomas M. Cover, Joy A. Thomas
Mining association rules between sets of items in large databases
14,720 Citations1993Rakesh Agrawal, Tomasz Imieliński +1 more
An efficient algorithm is presented that generates all significant association rules between items in the database of customer transactions and incorporates buffer management and novel estimation and pruning techniques.
ACM SIGMOD RecordMining association rules between sets of items in large databases
4,475 Citations1993Rakesh Agrawal, Tomasz Imieliński +1 more
MIT Press eBooksKnowledge Discovery in Databases
1,758 Citations1991Gregory Piateski, William Frawley
Future Generation Computer SystemsMining generalized association rules
1,617 Citations1997Ramakrishnan Srikant, Rakesh Agrawal
A new interest-measure for rules which uses the information in the taxonomy is presented, and given a user-specified “minimum-interest-level”, this measure prunes a large number of redundant rules.
Elsevier eBooksHeterogeneous Uncertainty Sampling for Supervised Learning
1,142 Citations1994David Lewis, Jason Catlett
This work test the use of one classifier (a highly efficient probabilistic one) to select examples for training another (the C4.5 rule induction program) and finds that the uncertainty samples yielded classifiers with lower error rates than random samples ten times larger.
Discovery of Multiple-Level Association Rules from Large Databases
920 Citations1995Jiawei Han, Yongjian Fu
A top-down progressive deepening method is developed for mining multiplelevel association rules from large transaction databases by extension of some existing association rule mining techniques.
Knowledge Discovery and Data MiningEfficient algorithms for discovering association rules
630 Citations1994Heikki Mannila, Hannu Toivonen +1 more
An improved algorithm for the problem of mining association rules from large collections of data based on careful combinatorial analysis of the information obtained in previous passes is given, which makes it possible to eliminate unnecessary candidate rules.
TileBars
554 Citations1995Marti A. Hearst
An evaluation of phrasal and clustered representations on a text categorization task
547 Citations1992David Lewis
It is shown that optimal effectiveness occurs when using only a small proportion of the indexing terms available, and that effectiveness peaks at a higher feature set size and lower effectiveness level for a syntactic phrase indexing than for word-based indexing.
Fraunhofer-Publica (Fraunhofer-Gesellschaft)Explora: a multipattern and multistrategy discovery assistant
440 Citations1996Willi Klösgen
An exercise assembly with parallel and spaced upright posts, a grip rod spanning the posts and having the opposite end portions movably connected to the posts by means of hollow joints each having a vertical lower pipe section movably receiving the associated end portion of the grip rod and a horizontal pipe section.
The dynamic HomeFinder
302 Citations1992Kyle Wm. Hall, Charles Périn
A new concept for visualizing and searching databases utilizing direct manipulation called dynamic queries, which allows users to formulate queries by adjusting graphical widgets, such as sliders, and see the results immediately, is designed and implemented.
Constant interaction-time scatter/gather browsing of very large document collections
206 Citations1993Douglass R. Cutting, David R. Karger +1 more
This work presents a scheme that supports constant interaction-time Scatter/Gather of arbitrarily large collections after near-linear time preprocessing, and involves the construction of a cluster hierarchy.
Similarity-based estimation of word cooccurrence probabilities
119 Citations1994Ido Dagan, Fernando Pereira +1 more
A probabilistic word association model based on distributional word similarity is described, and it is applied to improving probability estimates for unseen word bigrams in a variant of Katz's back-off model.
International Journal of Intelligent SystemsProblems for knowledge discovery in databases and their treatment in the statistics interpreter explora
72 Citations1992Willi Klösgen
This article introduces patterns to identify what is interesting in data and gives some examples of patterns for difference‐, change‐, and trend‐detection, and summarizes what must be specified to define a pattern.
Towards Language Independent Automated Learning of Text Categorization Models
62 Citations1994Chidanand Apté, Fred J. Damerau +1 more
The results of extensive machine learning experiments on large collections of Reuters’ English and German newswires are described, and the methodology, which seems to be insensitive to the language of the document collections, is presented.
Journal of Intelligent Information SystemsEfficient discovery of interesting statements in databases
48 Citations1995Willi Kl�sgen
This paper describes how Explora assesses interestingness and achieves computational efficiency, and describes strategies that reduce data accesses and speed up computation.
Joining statistics with NLP for text categorization
44 Citations1992Paul S. Jacobs
The approach has been to use statistics in the knowledge acquisition component of a linguistic pattern-based categorization system, using statistical methods, for example, to associate words with industries and identify phrases that information about businesses or products.
A probabilistic model for text categorization
38 Citations1994Makoto Iwayama, Takenobu Tokunaga
A new probabilistic model for text categorization, that is based on a Single random Variable with Multiple Values (SVMV) that has the following advantages; 1) it considers within-document term frequencies, 2) considers term weighting for target documents, and 3) is less affected by having insufficient training cases.
Keyword-Based Browsing and Analysis of Large Document Sets
31 Citations1996Ido Dagan, Ronen Feldman +1 more
The KDT system for KnowledgeDiscovery in Texts is described, built on top of atext-categorization paradigm where textarticles are annotated with keywordsorganized in a hierarchical structure, providing a generalframework for knowledge discovery and exploration in collections of unstructuredtext.
Journal of Intelligent Information SystemsOpportunity explorer: Navigating large databases using knowledge discovery templates
26 Citations1995Tej Anand
TheNielsen Opportunity Explorertmproduct can be used by sales and trade marketing personnel within consumer packaged goods manufacturers to understand how their products are performing in the market place and find opportunities to sell more product, more profitably to the retailers.
Knowledge Discovery and Data MiningKnowledge discovery in telecommunication services data using Bayesian network models
22 Citations1995Kazuo J. Ezawa, Steven W. Norton
This paper addresses the discovery of predictive knowledge bearing on fraud and uncollectible debt using a supervised machine learning method that constructs Bayesian network models and is able to predict rare event outcomes and cope with the quirks and copious amounts of input data.
Exploiting sophisticated representations for document retrieval
6 Citations1994Steven Finch
It is shown that statistical systems can exploit sophisticated representations of documents, and lends some support to the use of more linguistically sophisticated representations for document classification, on work done for the LRE project SISTA, which is creating a PC based tool to be used in the technical abstracting industry.