Implementation of Training Convolutional Neural Networks

arXiv (Cornell University)Published 3 June 2015Open access

Tianyi Liu, Shuangsang Fang, Yuehui Zhao, Peng Wang, Jun Zhang

DOI10.48550/arxiv.1506.01195

Citations125

TL;DR

This article gave a detailed analysis of the process of CNN algorithm both the forward process and back propagation and applied the particular convolutional neural network to implement the typical face recognition problem by java.

Abstract

Deep learning refers to the shining branch of machine learning that is based on learning levels of representations. Convolutional Neural Networks (CNN) is one kind of deep neural network. It can study concurrently. In this article, we gave a detailed analysis of the process of CNN algorithm both the forward process and back propagation. Then we applied the particular convolutional neural network to implement the typical face recognition problem by java. Then, a parallel strategy was proposed in section4. In addition, by measuring the actual time of forward and backward computing, we analysed the maximal speed up and parallel efficiency theoretically.

Keywords

Computer Science

Communications of the ACMImageNet classification with deep convolutional neural networks

75,709 Citations2017Alex Krizhevsky, Ilya Sutskever +1 more

A large, deep convolutional neural network was trained to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes and employed a recently developed regularization method called "dropout" that proved to be very effective.

View PDF

Proceedings of the IEEEGradient-based learning applied to document recognition

58,219 Citations1998Yann LeCun, Léon Bottou +2 more

This paper reviews various methods applied to handwritten character recognition and compares them on a standard handwritten digit recognition task, and Convolutional neural networks are shown to outperform all other techniques.

View PDF

Convolutional networks and applications in vision

2,138 Citations2010Yann LeCun, Koray Kavukcuoglu +1 more

New unsupervised learning algorithms, and new non-linear stages that allow ConvNets to be trained with very few labeled samples are described, including one for visual object recognition and vision navigation for off-road mobile robots.

View PDF

Deep Convolutional Network Cascade for Facial Point Detection

1,436 Citations2013Yi Sun, Xiaogang Wang +1 more

The proposed approach outperforms state-of-the-art methods in both detection accuracy and reliability and can avoid local minimum caused by ambiguity and data corruption in difficult image samples due to occlusions, large pose variations, and extreme lightings.

International Joint Conference on Artificial IntelligenceFlexible, high performance convolutional neural networks for image classification

1,203 Citations2011Dan Cireşan, Ueli Meier +3 more

Applying Convolutional Neural Networks concepts to hybrid NN-HMM model for speech recognition

888 Citations2012Ossama Abdel‐Hamid, Abdelrahman Mohamed +2 more

The proposed CNN architecture is applied to speech recognition within the framework of hybrid NN-HMM model to use local filtering and max-pooling in frequency domain to normalize speaker variance to achieve higher multi-speaker speech recognition performance.

arXiv (Cornell University)Multi-digit Number Recognition from Street View Imagery using Deep Convolutional Neural Networks

284 Citations2013Ian Goodfellow, Yaroslav Bulatov +3 more

This paper employs the DistBelief implementation of deep neural networks in order to train large, distributed neural networks on high quality images and finds that the performance of this approach increases with the depth of the convolutional network.

View PDF