Virtual Assistant for Deaf and Dumb
This work proposes a system that will bridge the gap between all the potential users whether normal or disabled (hearing or speaking impaired), and take in input in the form of sign language video, which is fed to Google Assistant/Amazon Alexa as input.
Abstract
: Voice is the future of computing interfaces, such as Robotics or the Internet of things. It is very difficult for those who have hearing or speaking disabilities. Almost all products or applications which are being developed today have voice-controlled features in them. It creates an indifference among the normal people and those with hearing or speaking disabilities. These applications should be able to be used by all sections of the potential users equally, so we propose a system that will bridge the gap between all the potential users whether normal or disabled (hearing or speaking impaired). Our systems take in input1 in the form of sign language video, it first breaks the gesture video into frames and then apply Convolutional Neural Network (CNN) on these gesture-frames so as to extract meaningful text, which is then spoken aloud by the device (on which the system is running) this is fed to Google Assistant/Amazon Alexa as input2, the response is converted to text and displayed on the screen and also spoken aloud by the device. This way people with hearing or speaking disabilities can communicate with a virtual assistant.