TyDi QA: A Benchmark for Information-Seeking Question Answering in\n Typologically Diverse Languages
No TL;DR found
Abstract
Confidently making progress on multilingual modeling requires challenging,\ntrustworthy evaluations. We present TyDi QA---a question answering dataset\ncovering 11 typologically diverse languages with 204K question-answer pairs.\nThe languages of TyDi QA are diverse with regard to their typology---the set of\nlinguistic features each language expresses---such that we expect models\nperforming well on this set to generalize across a large number of the world's\nlanguages. We present a quantitative analysis of the data quality and\nexample-level qualitative linguistic analyses of observed language phenomena\nthat would not be found in English-only corpora. To provide a realistic\ninformation-seeking task and avoid priming effects, questions are written by\npeople who want to know the answer, but don't know the answer yet, and the data\nis collected directly in each language without the use of translation.\n