Seminar: Learning Over-Parameterized Models: Regularization, Geometry, and Generalization
Arindam Banerjee: Professor, Department of Computer & Engineering, and Resident Fellow at the Institute on the Environment, University of Minnesota
Event Details
Abstract: The past decade has seen unprecedented empirical success of machine
learning based on over-parameterized models, where the number of
parameters is considerably larger than the number of training
samples. In terms of representation, such models range from linear
predictors to non-convex and non-smooth deep networks. In this talk,
we will discuss high-dimensional geometry of over-parameterized models
and the implications of such geometry on the behavior of such
models. We will start with the geometry of explicit regularization for
linear over-parameterized models and illustrate how the geometry helps
in establishing finite sample statistical and computational
guarantees. We will then discuss the somewhat surprising
high-dimensional geometry from implicit regularization based on
running SGD (stochastic gradient descent) on over-parameterized deep
networks. We will briefly review the mysteries behind generalization
in deep networks and present a geometric characterization of such
generalization. We will end with a discussion on promising advances
on real world scientific problems, including ecology and climate
science, and an outlook for future work.
Bio: Arindam Banerjee is a Professor at the Department of Computer &
Engineering and a Resident Fellow at the Institute on the Environment
at the University of Minnesota, Twin Cities. His research interests
are in machine learning, data mining, and applications in complex
real-world problems in different areas including climate science,
ecology, recommendation systems, finance, medicine, and aviation
safety. He has won several awards, including the NSF CAREER award
(2010), the IBM Faculty Award (2013), and six best paper awards in
top-tier conferences.