Research on learning in deep networks has led to an impressive performance by machines on tasks such as object recognition, but a deep understanding of the behaviour of these networks and why they perform so well remains a mystery. In this unit, you will first learn about a model of rapid object recognition in the visual cortex that resembles the structure of deep networks. You will then learn some of the theory behind how the structural connectivity, complexity, and dynamics of deep networks govern their learning behaviour.
Tomaso Poggio describes a theory of processing in the ventral pathway of the brain that solves the challenging problem of recognizing objects despite variations in their visual appearance due to geometric transformations such as translation and rotation.
Tomaso Poggio and his colleagues have developed a model of the early processing stages in the ventral visual pathway of the brain, which may underlie our ability to recognize object categories from visual input in a brief flash of less than 100 milliseconds. (Courtesy of Tomaso Poggio and Thomas Serre. From “Models of visual cortex.” Scholarpedia 8 no. 4 (2013): 3516. License CC BY-NC-SA.)
The guest lecture by Surya Ganguli shows how insights from statistical mechanics applied to the analysis of high-dimensional data can contribute to our understanding of how functions such as object categorization emerge in multi-layer neural networks.
Haim Sompolinsky explores the theoretical role of common properties of neural architectures in biological systems, in learning tasks such as classification. These properties include the number of stages of the neural network, compression or expansion of the dimensionality of the information, the role of noise, and presence of recurrent or feedback connections.
- Introductions to statistics and machine learning, including deep learning networks