The Naive Bayes Classifier

Abstract

The Naive Bayes Classifier makes a so-called conditional independence assumption that is almost always wrong. This incorrect assumption earns the classifier the designation "naive." The assumption greatly simplifies calculations; the naive Bayes classifier is very fast. The assumption trades off increased bias with reduced variance making the classifier surprisingly successful. The Naive Bayes classifier often benefits from smoothing. We discuss Laplace smoothing and the m-estimator. Somewhat cheekily, we use the Naive Bayes classifier to determine whether the movie "Shakespeare in Love" would be classified as a history, tragedy, or comedy, had the movie been written by Shakespeare. Our case study is about an open-ended survey question where respondents give advice to "Patient Joe" in a hypothetical situation. We classify the text answers into one of four classes.