Top Research Papers on NLP

This research presents a meta-modelling framework that automates the very labor-intensive and therefore time-heavy and expensive process of manually cataloging and cataloging individual words in a language.

An overview of NLP models is provided, with a focus on GPT-3, as well as discussion of applications specific to ophthalmology, and the limitations of G PT-3 and the challenges with its integration into routine ophthalmic care are outlined.

The purpose of this review is to enhance the performance of the text classifier based on effectiveness to improve accuracy and text processing speed by using a suitable methodology in order to produce the promising results in the future.

A unified model for attention architectures in natural language processing is defined, with a focus on those designed to work with vector representations of the textual data, providing the first extensive categorization of the vast body of literature in this exciting domain.

Key findings on language production disturbance in psychosis are reviewed and recent advances in the computational methods used to analyze language data are described, including methods for the automatic measurement of discourse coherence, syntactic complexity, poverty of content, referential coherence and metaphorical language.

This work discusses two specific medical issues, i.e., the coronavirus disease 2019 (COVID-19) pandemic and mental health, in which NLP-driven smart healthcare plays an important role, and discusses the limitations of current works.

This report presents a detailed analysis and review of NLP evaluation, in principle and in practice. Part 1 examines evaluation concepts and establishes a framework for NLP system evaluation. This makes use of experience in the related area of information retrieval and the analysis also refers to evaluation in speech processing. Part 2 surveys significant evaluation work done so far, for instance in machine translation, and discusses the particular problems of generic system evaluation. The conclusion is that evaluation strategies and techniques for NLP need much more development, in particula...

In the construction industry under "Industry 4.0", Natural Language Processing (NLP) has been widely used to process and analyze text data to achieve construction intelligence. However, there lacks a comprehensive review of NLP application in construction-related areas, raising bar of research entry and setting obstacles for the rapid development in this fields. Ninety one NLP-related research articles in construction-related fields were retrieved to conduct a scientometric analysis using CiteSpace and VOSViewer, and summarized from the perspectives of anchordatasets/data sources, technologies...

This paper presents a meta-modelling framework that automates the very labor-intensive and therefore time-heavy and expensive process of manually cataloging and cataloging language features in the input and output of a NLP system.

A quantum-enhanced Long Short-Term Memory network is successfully trained to perform the parts-of-speech tagging task via numerical simulations and a Quantum Transformer is proposed to performs the sentiment analysis based on the existing dataset.

A simple, actionable summary of recent work on bias in natural language processing (NLP) applications outlines five sources where bias can occur in NLP systems: the data, the annotation process, the input representations, the models, and finally the research design.

The \textit{Transformers} library is an open-source library that consists of carefully engineered state-of-the art Transformer architectures under a unified API and a curated collection of pretrained models made by and available for the community.

After a year of development, the library now includes more than 650 unique datasets, has more than 250 contributors, and has helped support a variety of novel cross-dataset research projects and shared tasks.

Text mining (TM) and natural language processing (NLP) have stirred interest within the construction field, as they offer enhanced capabilities for managing and analyzing text-based information. This highlights the need for a systematic review to identify the status quo, gaps, and future directions from the perspective of construction management. A review was conducted by aligning the objectives of 205 publications with the specific domains, areas, tasks, and processes outlined in construction management practices. This review reveals multiple facets of the construction sector empowered by TM/...

v4.10.0: LayoutLM-v2, LayoutXLM, BEiT LayoutLM-v2 and LayoutXLM Four new models are released as part of the LatourLM-v2 implementation: LayoutLMv2ForSequenceClassification , LayoutLMv2Model , LayoutLMv2ForTokenClassification and LayoutLMv2ForQuestionAnswering , in PyTorch. The LayoutLMV2 model was proposed in LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding by Yang Xu, Yiheng Xu, Tengchao Lv, Lei Cui, Furu Wei, Guoxin Wang, Yijuan Lu, Dinei Florencio, Cha Zhang, Wanxiang Che, Min Zhang, Lidong Zhou. LayoutLMV2 improves LayoutLM to obtain state-of-the-art results ac...

This work introduces Stanza, an open-source Python natural language processing toolkit supporting 66 human languages that features a language-agnostic fully neural pipeline for text analysis, including tokenization, multi-word token expansion, lemmatization, part-of-speech and morphological feature tagging, dependency parsing, and named entity recognition.

It is shown that for domains with abundant unlabeled text, such as biomedicine, pretraining language models from scratch results in substantial gains over continual pretraining of general-domain language models.

We introduce Stanza, an open-source Python natural language processing toolkit supporting 66 human languages. Compared to existing widely used toolkits, Stanza features a language-agnostic fully neural pipeline for text analysis, including tokenization, multi-word token expansion, lemmatization, part-of-speech and morphological feature tagging, dependency parsing, and named entity recognition. We have trained Stanza on a total of 112 datasets, including the Universal Dependencies treebanks and other multilingual corpora, and show that the same neural architecture generalizes well and achieves ...

This work looks at the problem of CL through the lens of various NLP tasks, and discusses major challenges in CL and current methods applied in neural network models.

This study shows that modern language algorithms partially converge towards brain-like solutions, and thus delineates a promising path to unravel the foundations of natural language processing.

In this review, we describe the application of one of the most popular deep learning-based language models - BERT. The paper describes the mechanism of operation of this model, the main areas of its application to the tasks of text analytics, comparisons with similar models in each task, as well as a description of some proprietary models. In preparing this review, the data of several dozen original scientific articles published over the past few years, which attracted the most attention in the scientific community, were systematized. This survey will be useful to all students and researchers ...

The operations and explainability techniques currently available for generating explanations for NLP model predictions are detailed to serve as a resource for model developers in the community and to point out the current gaps.

This survey categorizes and addresses the different aspects and applications of NLP that have benefited from deep learning and describes how deep learning methods and models advance these areas.

An introduction to the field and a quick overview of deep learning architectures and methods is provided and a discussion of the current state of the art is provided along with recommendations for future research in the field.

This survey is purposed to be a hands-on guide for understanding, using, and developing PTMs for various NLP tasks.

A new taxonomy of GNNs for NLP is proposed, which systematically organizes existing research of Gnns forNLP along three axes: graph construction, graph representation learning, and graph based encoder-decoder models.

Recent years have seen important advances in the quality of state-of-the-art models, but this has come at the expense of models becoming less interpretable. This survey presents an overview of the current state of Explainable AI (XAI), considered within the domain of Natural Language Processing (NLP). We discuss the main categorization of explanations, as well as the various ways explanations can be arrived at and visualized. We detail the operations and explainability techniques currently available for generating explanations for NLP model predictions, to serve as a resource for model develop...

This paper frames DA methods into three categories based on the diversity of augmented data, including paraphrasing, noising, and sampling, and introduces their applications in NLP tasks as well as the challenges.

Overall, domain-specific vocabulary and pretraining facilitate robust models for fine-tuning and establish a new state of the art on a wide range of biomedical NLP applications.

This paper proposes a multi-modal neural network that applies BI-GRU (Bidirectional Gated Recurrent Unit) to English word segmentation, and uses the CRF (Conditional Random Field) model to annotate sentences in sequence, effectively solving the long-distance dependency of text semantics, shortening network training and predicted time.

The success, promise and pitfalls of applying NLP algorithms to the study of proteins, and methods for encoding the information of proteins as text and analyzing it with NLP methods, reviewing classic concepts such as bag-of-words, k-mers/n-grams and text search.

Abstract Background Natural language processing (NLP) has a significant role in advancing healthcare and has been found to be key in extracting structured information from radiology reports. Understanding recent developments in NLP application to radiology is of significance but recent reviews on this are limited. This study systematically assesses and quantifies recent literature in NLP applied to radiology reports. Methods We conduct an automated literature search yielding 4836 results using automated filtering, metadata enriching steps and citation search combined with manual review. Our an...

A conceptual overview of common techniques used to analyse large volumes of text, and reproducible code that can be readily applied to other research studies using open-source software is presented.

A systematic survey on preliminary knowledge of NLP and related seminal works in computer vision is presented, which collects all related academic works since the first appearance in 2017 and analyzes 40 representative works in a comprehensive way.

A systematic literature review to capture the current state-of-the-art of NLP research on user stories identified 38 primary studies that discuss NLP techniques in user stories and found NLP can help system analysts manage user stories.

Jose Camacho-collados, Kiamehr Rezaee, Talayeh Riahi, Asahi Ushio, Daniel Loureiro, Dimosthenis Antypas, Joanne Boisson, Luis Espinosa Anke, Fangyu Liu, Eugenio Martínez Cámara. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing: System Demonstrations. 2022.

This paper discusses in detail the state of the art presenting the various applications of NLP, current trends, and challenges, and presents a discussion on some available datasets, models, and evaluation metrics in NLP.

Experimental results show that MacBERT could achieve state-of-the-art performances on many NLP tasks, and it is proposed that this model improves upon RoBERTa in several ways, especially the masking strategy that adopts MLM as correction (Mac).

It is found that ChatGPT performs well on many tasks favoring reasoning capabilities while it still faces challenges when solving specific tasks such as sequence tagging, and with extensive empirical studies, both the effectiveness and limitations of the current version of ChatG PT are demonstrated.

This work designs Hardware-Aware Transformers with neural architecture search, and trains a SuperTransformer that covers all candidates in the design space, and efficiently produces many SubTransformers with weight sharing, and performs an evolutionary search with a hardware latency constraint.