Top Research Papers on Sign Language Recognition

A taxonomy to categorize the proposed models for isolated and continuous sign language recognition is presented, discussing applications, datasets, hybrid models, complexity, and future lines of research in the field.

Assessment of ML-based SLR model was conducted with the help of 4 candidates under a controlled environment, and the model yielded 65% accuracy.

Robust modeling of static signs in the context of sign language recognition using deep learning-based convolutional neural networks (CNN) and demonstrates its effectiveness over the earlier works in which only a few hand signs are considered for recognition.

This paper introduces a robust pose normalization scheme which takes the signing space in consideration and processes the hand poses in a separate local coordinate system, independent on the body pose, and demonstrates the significant impact of this normalization on the accuracy of the proposed system.

A novel skeleton Aware Multimodal SLR framework (SAM-SLR) is proposed and a Sign Language Graph Convolution Network (SL-GCN) to model the embedded dynamics and a novel Separable SpatialTemporal Convolution network (SSTCN) to exploit skeleton features to improve the recognition rate.

This work revisits the overfitting problem in recent CTC-based CSLR works and proposes a Visual Alignment Constraint (VAC) to enhance the feature extractor with more alignment supervision and proposes two metrics to evaluate the contributions of the feature Extractor and the alignment model, which provide evidence for the over fitting problem.

A detailed review of all the works in vision-based CSLR is presented, based on the methods they have followed, followed by a brief on sensor-based systems and benchmark databases.

This study proposed an efficient deep convolutional neural networks approach for hand gesture recognition that employed transfer learning to beat the scarcity of a large labeled hand gesture dataset.

A novel method is proposed that learns domain-invariant visual concepts and fertilizes WSLR models by transferring knowledge of subtitled news sign to them, and outperforms previous state-of-the-art methods significantly.

Analysis of studies that use wearable sensor-based systems to classify sign language gestures finds trends, best practices, and common challenges could aid in the development of user-centred and robust wearable sensors for sign language recognition.

A Self-Mutual Knowledge Distillation (SMKD) method is proposed, which enforces the visual and contextual modules to focus on short-term and long-term information and enhances the discriminative power of both modules simultaneously.

Hand signs are an effective form of human-to-human communication that has a number of possible applications. Being a natural means of interaction, they are commonly used for communication purposes by speech impaired people worldwide. In fact, about one percent of the Indian population belongs to this category. This is the key reason why it would have a huge beneficial effect on these individuals to incorporate a framework that would understand Indian Sign Language. In this paper, we present a technique that uses the Bag of Visual Words model (BOVW) to recognize Indian sign language alphabets (...

Evaluation results of the proposed model indicate that the model outperforms state-of-the-art models in hand sign language recognition, hand pose estimation, and hand action recognition.

Long-Short Term Memory Recurrent Neural Network with k-Nearest-Neighbour method is adopted as the classification method is based on handling of sequences of input, and revealed that the recognition rate for 26 ASL alphabets yields an average of 99.44% accuracy rate.

The aim of the present study is to provide insights on sign language recognition, focusing on mapping non-segmented video streams to glosses, by implementing the most recent deep neural network methods in this field.

This work proposes a spatial-temporal multi-cue (STMC) network, which aims to solve the vision-based sequence learning problem in continuous sign language recognition and designs a joint optimization strategy to achieve the end-to-end sequence learning of the STMC network.

The literature review presented in this paper shows the importance of incorporating intelligent solutions into the sign language recognition systems and reveals that perfect intelligent systems for sign language recognition are still an open problem.

Abstract Sign language recognition is challenged by problems, such as accurate tracking of hand gestures, occlusion of hands, and high computational cost. Recently, it has benefited from advancements in deep learning techniques. However, these larger complex approaches cannot manage long-term sequential data and they are characterized by poor information processing and learning efficiency in capturing useful information. To overcome these challenges, we propose an integrated MediaPipe-optimized gated recurrent unit (MOPGRU) model for Indian sign language recognition. Specifically, we improved ...

A spatial-temporal multi-cue (STMC) network is proposed to solve the vision-based sequence learning problem in video-based sign language understanding and achieves new state-of-the-art performance on all three benchmarks.

This work proposes a novel architecture with cross modality augmentation for continuous sign language recognition that can be easily extended to other existing CTC based continuous SLR architectures and validates the effectiveness of the proposed method on two continuousSLR benchmarks.

People lacking the sense of hearing and the ability to speak have undeniable communication problems in their life. People with hearing and speech problems communicate using sign language with themselves and others. Sign language is not essentially known to a more significant portion of the human population who uses spoken and written language for communication. Therefore, it is a necessity to develop technological tools for interpretation of sign language. Much research have been carried out to acknowledge sign language using technology for most global languages. But there are still scopes of ...

It appears that recognition based on a combination of data sources, including vision-based and sensor-based channels, is superior to a unimodal analysis in sign language recognition, and a general framework for researchers is proposed.

A deep learning based convolutional neural network model is specifically designed for the recognition of gesture-based sign language that achieves better classification accuracy with a fewer number of model parameters over the other existing architectures of CNN.

The proposed design for automatic American sign language recognition is cost-effective, computationally inexpensive, does not require any special sensors or devices, and has outperformed previous studies.

A vision-based system by applying CNN for the recognition of Arabic hand sign-based letters and translating them into Arabic speech is proposed in this paper and gives 90% accuracy to recognize the Arabic hand signs and gestures which assures it as a highly dependable system.

Natural sign languages of deaf communities are acquired on the same time scale as that of spoken languages if children have access to fluent signers providing input from birth. Infants are sensitive to linguistic information provided visually, and early milestones show many parallels. The modality may affect various areas of language acquisition; such effects include the form of signs (sign phonology), the potential advantage presented by visual iconicity, and the use of spatial locations to represent referents, locations, and movement events. Unfortunately, the vast majority of deaf children ...

Novel approaches for developing the complete framework for handling SL recognition, translation, and production tasks in real-time cases are introduced and the evaluation metrics show noticeable improvements in the model.

A novel system is proposed for dynamic hand gesture recognition using multiple deep learning architectures for hand segmentation, local and global feature representations, and sequence feature globalization and recognition, which outperforms state-of-the-art approaches.

Language barrier is the main cause of disagreement. Sign language, which is a common language in all the worldwide language families, is difficult to be entirely popularized due to the high cost of learning as well as the technical barrier in real-time translation. To solve these problems, here, we constructed a wearable organohydrogel-based electronic skin (e-skin) with fast self-healing, strong adhesion, extraordinary anti-freezing and moisturizing properties for sign language recognition under complex environments. The e-skin was obtained by using an acrylic network as the main body, alumin...

A more advanced successor of the Convolutional Neural Networks (CNNs) called 3-D CNNs is employed, which can recognize the patterns in volumetric data like videos like videos, which outperforms the existing state-of-art models in terms of precision, recall, and f-measure.

Experimental results show that the performance of proposed framework outperforms with large margin the state-of-the-art methods for signer-independent testing strategy.

A skeleton-based CSL recognition and generation framework based on a recurrent neural network (RNN), to support bidirectional CSL communication and the proposed algorithm achieved high recognition accuracy for both real and synthetic data, with a reduced runtime.

An artificial intelligence enabled sign language recognition and communication system comprising sensing gloves, deep learning block, and virtual reality interface helps barrier-free communication between signers and non-signers.

This paper introduces a new large-scale Word-Level American Sign Language (WLASL) video dataset, containing more than 2000 words performed by over 100 signers, and proposes a novel pose-based temporal graph convolution networks (Pose-TGCN) that model spatial and temporal dependencies in human pose trajectories simultaneously, which has further boosted the performance of the pose- based method.

Sign language is the comprehensive medium of mass communication for hearing and speaking impaired individuals. As they cannot speak or hear, they are not able to use sound or vocal signals as an information medium for their communication. Rather, they are bound to exchange visual signals to express their feeling in their day-to-day life. For this, they use various body language mainly hand gestures as sign language. Sign language fundamentals can be largely divided into two parts namely digits (numerals) and characters (alphabetical). In this paper, we proposed a hybrid model consisting of a d...

The proposed sign back-translation (SignBT) approach, which incorporates massive spoken language texts into SLT training, and obtains a substantial improvement over previous state-of-the-art SLT methods.

This survey aims to provide a comprehensive review of state-of-the-art methods in sign language capturing, recognition, translation and representation, pinpointing their advantages and limitations.

An experiment is implemented to evaluate the performance of the latest version of YOLOv5 based on the dataset for Traffic Sign Recognition (TSR), which unfolds how the model for visual object recognition in deep learning is suitable for TSR through a comprehensive comparison with SSD.

The video-to-text translation of the STMC-Transformer outperforms translation of GT glosses and contradicts previous claims that GT gloss translation acts as an upper bound for SLT performance and reveals that glosses are an inefficient representation of sign language.

This paper applies deep learning-based convolutional neural networks (CNNs) for robust modeling of static signs in the context of sign language recognition and highlights the recognition accuracy of each character, and their similarities with identical gestures.