Fall 2023 Data Science Seminar Series
Robert Nowak–Deeper Understanding of Deep Learning: Functional Analysis for Neural Networks

Speaker: Robert Nowak, Ph.D., Grace Wahba Professor of Data Science, Keith and Jane Nosbusch Professor in Electrical, Computer Engineering at the University of Wisconsin-Madison.

Faculty Host: Jonathan Siegel, MATH

Abstract: This talk explores tools from functional analysis that explain the exceptional performance of deep neural networks, which form the backbone of most state-of-the-art artificial intelligence systems. Taking center stage in this discussion is a relatively new mathematical framework that precisely details the functional properties of trained neural networks. The methodology for this framework relies heavily on transform-domain sparse regularization, the Radon transform of computed tomography, and approximation theory.

Furthermore, the framework clarifies several key aspects of neural network training and architecture. It provides insights into the effect of weight decay regularization in training, the relevance of skip connections and low-rank weight matrices in network design, the significance of sparsity in neural networks, and the proficiency of neural networks in handling high-dimensional problems. Moreover, the characterization of neural functions within this framework reveals new insights into the complexity and structure of deep neural networks. This fresh perspective opens a novel pathway to effectively study the behavioral traits of deep networks.

This talk is based on joint work with Rahul Parhi, Joe Shenouda, and Liu Yang.

Biography: Robert Nowak holds the Nosbusch Professorship in Electrical and Computer Engineering at the University of Wisconsin-Madison, where he directs the AFOSR/AFRL University Center of Excellence on Data Efficient Machine Learning. His research focuses on signal processing, machine learning, optimization, and statistics. He serves on the editorial boards of the SIAM Journal on the Mathematics of Data Science and the IEEE Journal on Selected Areas in Information Theory.

Link to PDF Version

You can also click this link to join the seminar

For more information about TAMIDS tutorial series, please contact Ms. Jennifer South at jsouth@tamu.edu