Text Classification for Intelligent Portfolio Management

Published 1 May 2002

Young‐Woo Seo, Joseph Giampapa, Katia Sycara

Citations28

TL;DR

A text classification algorithm is developed that classifies financial news article by using a combination of a reduced but highly informative word feature sets and a variant of weighted majority algorithm, which shows a better performance in terms of accuracy.

Abstract

In the application domain of stock portfolio management, software agents that evaluate the risks associated with the individual companies of a portfolio should be able to read electronic news articles that are written to give investors an indication of the financial outlook of a company. There is a positive correlation between news reports on a company&apos;s financial outlook and the company&apos;s attractiveness as an investment. However, because of the volume of such reports, it is impossible for financial analysts or investors to track and read each one. Therefore, it would be very helpful to have a system that automatically classifies news reports that reflect positively or negatively on a company&apos;s financial outlook. To accomplish this task, we treat the understanding of news articles as a text classification problem. In this paper, we propose a text classification method that we call, &quot;Domain Experts&quot; and &quot;Self-Confident&quot; sampling, and compare it with naive Bayes with expectation maximization (EM).

Keywords

Computer Science

Journal of the American Society for Information ScienceIndexing by latent semantic analysis

12,677 Citations1990Scott Deerwester, Susan Dumais +3 more

Lecture notes in computer scienceText categorization with Support Vector Machines: Learning with many relevant features

7,925 Citations1998Thorsten Joachims

SVMs achieve substantial improvements over the currently best performing methods and behave robustly over a variety of di-erent learning tasks, eliminating the need for manual parameter tuning.

A Comparative Study on Feature Selection in Text Categorization

4,766 Citations1997Yiming Yang, Jan Pedersen

DF thresholding, the simplest method with the lowest cost in computation, can be reliably used instead of IG or CHI when the computation of these measures are too expensive, and strong correlations between the DF, IG and CHI values of a term are found.

Machine LearningText Classification from Labeled and Unlabeled Documents using EM

2,749 Citations2000Kamal Nigam, Andrew Kachites McCallum +2 more

This paper shows that the accuracy of learned text classifiers can be improved by augmenting a small number of labeled training documents with a large pool of unlabeled documents, and presents two extensions to the algorithm that improve classification accuracy under these conditions.

View PDF

A re-examination of text categorization methods

2,660 Citations1999Yiming Yang, Xin Liu

The results show that SVM, kNN and LLSF signi cantly outperform NNet and NB when the number of positive training instances per category are small, and that all the methods perform comparably when the categories are over 300 instances.

View PDF

Machine LearningImproving Generalization with Active Learning

1,311 Citations1994David Cohn, Les Atlas +1 more

A formalism for active concept learning called selective sampling is described and it is shown how it may be approximately implemented by a neural network.

View PDF

Employing EM and Pool-Based Active Learning for Text Classification

681 Citations1998Andrew McCallum, Kamal Nigam

The Query-by-Committee method of active learning is modified to use the unlabeled pool for explicitly estimating document density when selecting examples for labeling, and the combination of EM and active learning requires only slightly more than half as many labeled training examples to achieve the same accuracy as either the improved active learning or EM alone.

IEEE ExpertDistributed intelligent agents

528 Citations1996Katia Sycara, Anandeep Pannu +3 more

In Retsina, the authors have developed a distributed collection of software agents that cooperate asynchronously to perform goal-directed information retrieval and integration for supporting a variety of decision-making tasks.

Elsevier eBooksCommittee-Based Sampling For Training Probabilistic Classifiers

497 Citations1995Ido Dagan, Sean P. Engelson

A general method for efficiently training probabilistic classifiers, by selecting for training only the more informative examples in a stream of unlabeled examples, which is particularly attractive because it evaluates the expected information gain from a training example implicitly.

Context-sensitive learning methods for text categorization

253 Citations1996William W. Cohen, Yoram Singer

View PDF