Talk: Towards a Statistical Foundation for Reinforcement Learning

LIVE STREAM: https://uwmadison.zoom.us/j/95503137028?pwd=Ry8zZExBeW42aXN2dlkzWmNyL25OZz09#success

Abstract: In recent years, reinforcement learning algorithms have achieved a number of headline- grabbing empirical successes on various complex tasks. However, applying the reinforcement learning paradigm to new problems remains highly challenging. In many cases, the existing algorithms need to be modified, and new ones may have to be developed to solve the problem at hand. In order to do so effectively, we must gain some understanding about the foundations of reinforcement learning. In this talk I will present some recent results of my research towards this goal. I will first present an algorithm that can exploit the domain structure to learn much faster on easier problems, while retaining state-of-the art worst-case guarantees on pathologically hard ones. Then I will discuss a fundamental information-theoretic lower-bound, which establishes that reinforcement learning can be exponentially harder than supervised learning even when simple linear predictors are implemented. Finally, I will discuss a statistically optimal algorithm to learn from historical data.

Bio: Andrea Zanette is a postdoctoral scholar in the Department of Electrical Engineering and Computer Sciences at the University of California, Berkeley, supported by a fellowship from the Foundation of Data Science Institute. He completed his PhD (2017-2021) in the Institute for Computational and Mathematical Engineering at Stanford University, advised by Prof Emma Brunskill and Mykel J. Kochenderfer. His PhD dissertation investigated modern Reinforcement Learning challenges such as exploration, function approximation, adaptivity, and learning from offline data. His work was supported by a Total Innovation Fellowship and his PhD thesis was awarded the Gene Golub Outstanding Dissertation Award from his department. Andrea’s background is in mechanical engineering. Before Stanford, he worked as a software developer in high-performance computing, as well as at the von Karman Institute for Fluid Dynamics, a NATO-affiliated international research establishment.