"What Parts of Your Apps are Loved by Users?" (T)

SUR-Miner classifies reviews into five categories and extracts aspects for sentences which include aspect evaluation using a pattern-based parser, and summarizes more accurate and clearer aspects than state-of-the-art techniques.

Abstract

Recently, Begel et al. found that one of the most important questions software developers ask is "what parts of software are used/loved by users." User reviews provide an effective channel to address this question. However, most existing review summarization tools treat reviews as bags-of-words (i.e., mixed review categories) and are limited to extract software aspects and user preferences. We present a novel review summarization framework, SUR-Miner. Instead of a bags-of-words assumption, it classifies reviews into five categories and extracts aspects for sentences which include aspect evaluation using a pattern-based parser. Then, SUR-Miner visualizes the summaries using two interactive diagrams. Our evaluation on seventeen popular apps shows that SUR-Miner summarizes more accurate and clearer aspects than state-of-the-art techniques, with an F1-score of 0.81, significantly greater than that of ReviewSpotlight (0.56) and Guzmans' method (0.55). Feedback from developers shows that 88% developers agreed with the usefulness of the summaries from SUR-Miner.