Double Standpoint Evaluation Method for Affect Analysis Systems

This work uses a double standpoint evaluation method (DSEM) for systems analyzing and recognizing affect in utterances to evaluate the ML-Ask system and shows the differences in results for two different standpoints – recognitive and commonsensical.

Abstract

We propose a double standpoint evaluation method (DSEM) for systems analyzing and recognizing affect in utterances. We use this method to evaluate our ML-Ask system and show the differences in results for two different standpoints – recognitive and commonsensical. Evaluation based on the former shows system’s accuracy in affect recognition needed for user-agent communication. The one based on the latter verifies system’s unanimity with the general commonsense interpretation of the affect conveyed in the utterance. Such evaluation is relevant to confirm the results of the recognition evaluation and can be further applied to sentiment analysis. Both standpoints are relevant and applying DSEM in research on affect analysis can verify performance of a system in much wider way than measures widely accepted.