Overview of the GermEval 2018 Shared Task on the Identification of Offensive Language
Publication Server of the Institute for German Language (Institute for German Language)Published 13 February 2019Open access
Michael Wiegand, Melanie Siegel, Josef Ruppenhofer
Citations237
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This pilot edition of the GermEval Shared Task on the Identification of Offensive Language deals with the classification of German tweets from Twitter and describes the process of extracting the raw-data for the data collection and the annotation schema.
Abstract
We present the pilot edition of the GermEval Shared Task on the Identification of Offensive Language. This shared task deals with the classification of German tweets from Twitter. It comprises two tasks, a coarse-grained binary classification task and a fine-grained multi-class classification task. The shared task had 20 participants submitting 51 runs for the coarse-grained task and 25 runs for the fine-grained task. Since this is a pilot task, we describe the process of extracting the raw-data for the data collection and the annotation schema. We evaluate the results of the systems submitted to the shared task. The shared task homepage can be found at https://projects.cai. fbi.h-da.de/iggsa/
Keywords
Computer ScienceSocial Sciences
BiometricsThe Measurement of Observer Agreement for Categorical Data
78,934 Citations1977J. Richard Landis, Gary G. Koch
A general statistical methodology for the analysis of multivariate categorical data arising from observer reliability studies is presented and tests for interobserver bias are presented in terms of first-order marginal homogeneity and measures of interob server agreement are developed as generalized kappa-type statistics.
Hateful Symbols or Hateful People? Predictive Features for Hate Speech Detection on Twitter
1,655 Citations2016Zeerak Waseem, Dirk Hovy
A list of criteria founded in critical race theory is provided, and these are used to annotate a publicly available corpus of more than 16k tweets and present a dictionary based the most indicative words in the data.
A Survey on Hate Speech Detection using Natural Language Processing
1,389 Citations2017Anna Grau Schmidt, Michael Wiegand
A survey on hate speech detection describes key areas that have been explored to automatically recognize these types of utterances using natural language processing and discusses limits of those approaches.
Abusive Language Detection in Online User Content
1,107 Citations2016Chikashi Nobata, Joel Tetreault +3 more
A machine learning based method to detect hate speech on online user comments from two domains which outperforms a state-of-the-art deep learning approach and a corpus of user comments annotated for abusive language, the first of its kind.
Text, speech and language technologyImprovements in Part-of-Speech Tagging with an Application to German
821 Citations1999Helmut Schmid
This paper presents a meta-modelling system that automates the very labor-intensive and therefore time-heavy and expensive process of manually tagging part-of-speech content in a variety of languages.
FigshareImproved Part-of-Speech Tagging for Online Conversational Text with Word Clusters
682 Citations2018Olutobi Owoputi, Brendan O’Connor +4 more
This work systematically evaluates the use of large-scale unsupervised word clustering and new lexical features to improve tagging accuracy on Twitter and achieves state-of-the-art tagging results on both Twitter and IRC POS tagging tasks.
DataStories at SemEval-2017 Task 4: Deep LSTM with Attention for Message-level and Topic-based Sentiment Analysis
503 Citations2017Christos Baziotis, Nikos Pelekis +1 more
Two deep-learning systems that competed at SemEval-2017 Task 4 “Sentiment Analysis in Twitter” are presented, which use Long Short-Term Memory networks augmented with two kinds of attention mechanisms, on top of word embeddings pre-trained on a big collection of Twitter messages.
Measuring the Reliability of Hate Speech Annotations: The Case of the European Refugee Crisis
176 Citations2016Björn Roß, Michael Rist +4 more
It is concluded that the presence of hate speech should perhaps not be considered a binary yes-or-no decision, and raters need more detailed instructions for the annotation, which was very low overall.
Inducing a Lexicon of Abusive Words – a Feature-Based Approach
150 Citations2018Michael Wiegand, Josef Ruppenhofer +2 more
The effectiveness of the (domain-independent) lexicon in the cross-domain detection of abusive microposts is demonstrated and it is shown that the word-level information the authors learn cannot be equally derived from a large dataset of annotated micropposts.
International Conference on Computational LinguisticsA High-Performance Syntactic and Semantic Dependency Parser
121 Citations2010Björkelund Anders, Bernd Bohnet +2 more
GermanPolarityClues: A Lexical Resource for German Sentiment Analysis
70 Citations2010Ulli Waltinger
Evaluating four different English and three different German sentiment resources in a comparative manner by combining a polarity-based feature selection with SVM-based machine learning classifier shows that the size of dictionaries clearly correlate to polarity, but this property does not correlate to classification accuracy.
Zurich Open Repository and Archive (University of Zurich)Exploiting Synergies Between Open Resources for German Dependency Parsing, POS-tagging, and Morphological Analysis
44 Citations2013Rico Sennrich, Martin Volk +1 more
The effect of POS tagging and morphological analysis on parsing performance is discussed, and novel ways of improving performance of the components are presented, including the use of morphological features for POS-tagging, and using syntactic information to select good POS sequences from an n-best list.
HilDok – Institutional Repository (Universität Hildesheim)GermEval 2014 Named Entity Recognition Shared Task: Companion Paper
34 Citations2014Darina Benikova, Chris Biemann +2 more
DSpace repository (University of Tartu)PolArt: a robust tool for sentiment analysis
34 Citations2009Manfred Klenner, Angela Fahrni +1 more
PolArt is a pattern-based approach designed to cope with polarity composition, based on the TreeTagger chunker output, and is customised for English and German.
GermEval 2015: LexSub -- A Shared Task for German-language Lexical Substitution
8 Citations2015Tristan Miller, Darina Benikova +1 more
GermEval 2015: LexSub, the first shared task for automated lexical substitution on German-language text, is described, the motivation for this task, the evaluation methods, and the manually annotated data set used to train and test the participating systems are described.