Deep Thoughts on Deep Learning

Abstract

Deep learning is often credited with having nearly metaphysical powers to solve challenging problems. Yet, the techniques behind deep learning are often treated as mysterious black boxes. In this tutorial, we attempt to provide a solid foundation for a deeper understanding of deep learning. Our primary emphasis is on backpropagation and automatic differentiation, but we also discuss a variety of related topics, including gradient descent, and various parameters that arise. In addition, we point out some of the many connections between deep learning and other not-so-deep techniques—primarily, hidden Markov models (HMM) and support vector machines (SVM). But first, we discuss artificial neural networks, which are the basic building blocks of deep learning.