Sign language recognition using MediaPipe
MediaPipe is used as a method of feature extraction to extract coordinates of the joint points of the human body and is able to recognize 64 Argentine sign language words or phrases.
Abstract
Sign language recognition is a popular and important problem in computer vision. This paper uses MediaPipe as a method of feature extraction to extract coordinates of the joint points of the human body. The sign language dataset used in this paper is 64 Argentine sign languages (LSA64). Then the coordinate data is input to Long-Short Term Memory (LSTM) and Gated Recurrent Unit (GRU) neural networks. After testing two models in 640 videos, the result has shown that the accuracy of the trained LSTM and GRU model reaches 94.0625% and 94.5312%, respectively. It is able to recognize 64 Argentinian sign language words or phrases. As a convenient method of sign language recognition, it is feasible in reality.