Location: MIC-Arena, Maria-Goeppert-Str. 3, 23562 Lübeck
Title: "An Optimal Control Approach to Deep Learning"
Deep neural networks (DNNs) have become invaluable tools for supervised machine learning problems, e.g., classification of text or images. While offering superior flexibility to find and express complicated patterns in data, deep architectures are known to be challenging to design and train so that they generalize well to new data.
In this talk, we establish the relation of deep learning and optimal control. We show that, given training data and labels, DNNs can be trained by optimizing parameters of a system of nonlinear Ordinary Differential Equation (ODE). Using this interpretation we give intuitive explanations for commonly observed problems, for example, exploding or vanishing gradients by analyzing the stability of the underlying ODE. The new framework also outlines strategies for designing new, stable learning algorithms and training strategies. We present several new network architectures inspired by Hamiltonian systems and learning approaches inspired by multilevel and multiscale approaches commonly used in imaging. The talk features several small-scale academic test problems as well as some larger image classification problems.
The talk is based in part on this manuscript: Stable Architectures for Deep Neural Networks
