Bridging Lifelong and Multi-Task Representation Learning via Algorithm and Complexity Measure

In lifelong learning, a learner faces a sequence of tasks with shared structure and aims to identify and leverage it to accelerate learning along the way. We study the setting where such structure is captured by a common representation of data. Such shared representation structure across tasks is well studied for multi-task learning and learning-to-learn settings, where data from various tasks are available upfront to learn the shared representation. In contrast, lifelong learning does not have the luxury of having data available a priori. Therefore, it requires the learner to make use of its existing knowledge while continually gathering partial information in an online fashion. In this talk, I will present our recent work where we propose a generalized framework of lifelong representation learning. We also propose a simple and natural algorithm that uses multi-task empirical risk minimization as a subroutine and establishes a sample complexity bound based on a new complexity notion we introduce—the task-eluder dimension. Our result applies to a wide range of learning problems involving general function classes. As concrete examples, we instantiate our result on classification and regression tasks under noise deriving novel sample complexity bounds.

(This talk is part of the weekly Machine Learning Lunch Meetings (MLLM), held every Tuesday from 12:15 to 1:15 p.m.  Professors from Computer Sciences, Statistics, ECE, the iSchool, and other departments will discuss their latest research in machine learning, covering both theory and applications. This is a great opportunity to network with faculty and fellow researchers, learn about cutting-edge research at our university, and foster new collaborations. For the talk schedule, please visit https://sites.google.com/view/wiscmllm/home. To receive future weekly talk announcements, please subscribe to our UW Google Group at https://groups.google.com/u/1/a/g-groups.wisc.edu/g/mllm.)