Machine Learning Lunch Meeting

Everyone is invited to the weekly machine learning lunch meetings, where our faculty members from Computer Science, Statistics, ECE, and other departments will discuss their latest groundbreaking research in machine learning. This is an opportunity to network with faculty and fellow researchers while learning about the cutting-edge research being conducted at our university. See https://sites.google.com/view/wiscmllm/home for more information.

Speaker: Grigoris Chrysos (ECE)

Abstract: Activation functions are an indispensable component of deep networks for learning challenging tasks, such as image recognition. However, activation functions raise major hurdles for deep learning theory, studying the dynamics of networks, or even properties such as interpretability. In this talk, we will show that activation functions might not be necessary in all cases. In particular, we will show that if our outputs express high-degree interactions of the input elements, then we can design networks that do not rely on activation functions. In our upcoming work in ICLR'24, we design precisely such networks using multilinear operations alone and we show that we can obtain strong-performing networks even in challenging tasks such as image recognition on ImageNet.