This work designed and implemented a deep learning-based language learning Android app that provides both speech recognition and handwriting recognition and created Convolutional Neural Networks models that support dictation/handwriting training in Chinese and Japanese respectively.
With the amazing advancement of mobile technology, people can easily learn a new language from their mobile devices. Exploring popular language learning apps, we discovered that speech and handwriting recognition functionalities are lacking in these apps. To address this issue, we designed and implemented a deep learning-based language learning Android app that provides both speech recognition and handwriting recognition. We created the speech recognition module based on the Google Speech API, which supports speech recognition in 110 languages. For handwriting recognition, we created Convolutional Neural Networks (CNN) models that support dictation/handwriting training in Chinese and Japanese respectively.