A study of English word category prediction based on neutral networks

Training results show that the NETgrams are comparable to the statistical model and compress information, and a method for speeding up the back-propagation algorithm, which can automatically determine better parameters and achieve a shorter training time is proposed.

Abstract

Using traditional statistical approaches, it is difficult to develop an N-gram word prediction model for constructing an accurate word recognition system because of the increased demand for sample data and parameters to memorize probabilities. To solve this problem, NETgrams, which are neural networks for N-gram word category prediction in text are proposed. NETgrams can easily be expanded from bigram to N-gram networks without exponentially increasing the number of free parameters. Training results show that the NETgrams are comparable to the statistical model and compress information. Results of analyzing the hidden layer (microfeatures) show that the word categories are classified into some linguistically significant groups. It is confirmed that NETgrams perform effectively for unknown data, i.e NETgrams interpolate sparse training data naturally just like the deleted interpolation. A method for speeding up the back-propagation algorithm, which can automatically determine better parameters and achieve a shorter training time is proposed.>