Automated news reading: Stock price prediction based on financial news using context-capturing features

It is shown that a robust feature selection allows lifting classification accuracies significantly above previous approaches when combined with complex feature types and reduces the problem of over-fitting when applying a machine learning approach.

Abstract

We examine whether stock price prediction based on textual information in financial news can be improved as previous approaches only yield prediction accuracies close to guessing probability. Accordingly, we enhance existing text mining methods by using more expressive features to represent text and by employing market feedback as part of our feature selection process. We show that a robust feature selection allows lifting classification accuracies significantly above previous approaches when combined with complex feature types. This is because our approach allows selecting semantically relevant features and thus, reduces the problem of over-fitting when applying a machine learning approach. We also demonstrate that our approach is highly profitable for trading in practice. The methodology can be transferred to any other application area providing textual information and corresponding effect data.