Talk: Sample Efficient Reinforcement Learning

LIVE STREAM: https://uwmadison.zoom.us/j/91702238837?pwd=VDRiYk4vbHlQNTQzTW5MVldoR1F1QT09

Abstract: Despite the empirical success of reinforcement learning (RL) in gaming, this level of success has not been observed in many real-world domains. This is often attributed to the high cost and risk associated with running experiments in domains such as healthcare and robotics. Therefore, it is essential to employ and collect samples in a sample-efficient manner to address this limitation.

In this presentation, I will demonstrate my research on sample-efficient RL. Firstly, I will address a major challenge in offline RL, which is the insufficient coverage of offline data. To overcome this challenge, I will propose a novel model-based algorithm that can employ rich function approximation. Our algorithm, CPPO, demonstrates superior performance of the output policy compared to any policy covered by offline data, even when the offline data coverage is inadequate. For the remainder of the presentation, I will discuss another crucial challenge, which is the integration of representation learning into online RL. I will introduce a new sample and computationally efficient algorithm, Rep-UCB, that enables a sophisticated interplay between representation learning, exploration, and exploitation. Our algorithm has the tightest sample complexity compared to existing computationally efficient algorithms.

Bio: Masatoshi Uehara is a third-year Ph.D. student at Cornell CS, advised by Nathan Kallus. He previously received a bachelor’s degree in applied mathematics and computer science from the University of Tokyo and a master’s of science in statistics at Harvard University. His research is at the intersection of reinforcement learning and causal machine learning. He has won two scholarships, awarded to the most outstanding students from Japan, during his Ph.D. program. His works have been selected as spotlight/oral papers (2–5%) in top machine learning conferences such as ICML, NeurIPS, and ICLR.