PsychBERT: A Mental Health Language Model for Social Media Mental Health Behavioral Analysis
A novel mental health language model, PsychBERT, which is pretrained on a large corpus of biomedical literature on mental health and social media data and which outperforms state-of-the-art methods and is interpretable.
Abstract
Mental health behaviors are now recognized as primary factors contributing to suicide. This paper puts forth a novel mental health language model to address mental health and makes several contributions. First, it proposes a taxonomy and puts forth a comprehensive dataset of social media text. Second, it proposes a two-stage framework, first discriminating text relevant to mental health from non-relevant text and then carrying out multi-class classification for detection of mental health behaviors. Third, it proposes a novel mental health language model, PsychBERT, which is pretrained on a large corpus of biomedical literature on mental health and social media data. Fourth, the framework additionally incorporates components that enhance its explainability. Our evaluation shows that the proposed framework is outperforms state-of-the-art methods and is interpretable. Pre-trained PsychBERT is made publicly available for the community at https://huggingface.co/mnaylor/psychbert-cased.