Seminar: Modern Perspectives on Classical Learning Problems: Role of Memory and Data Amplification

Abstract: This talk will discuss statistical and computation requirements---and how they interact---for three learning setups. In the first part, we inspect the role of memory in learning. We study how the total memory available to a learning algorithm affects the amount of data needed for learning (or optimization), beginning by considering the fundamental problem of linear regression. Next, we examine the role of long-term memory vs. short-term memory for the task of predicting the next observation in a sequence given the past observations. Finally, we explore the statistical requirements for the task of manufacturing more data---namely how to generate a larger set of samples from an unknown distribution. Can "amplifying" a dataset be easier than learning?