Machine Learning Lunch Meeting

General: MLLM is a cross-discipline weekly seminar open to all where UW-Madison professors present their research in machine learning, both theory and applications.  The goal is to promote the great work at UW-Madison and stimulate collaborations. Please see our website for more information.

Speaker: Manolis Vlatakis (CS)

Abstract: A wide range of modern machine learning applications—from adversarial models to multi-agent reinforcement learning—can be formulated as non-cooperative games, where Nash equilibria represent the system’s desired operational states. Despite the inherently non-convex loss landscapes of these settings, many exhibit latent convex structures that, if properly leveraged, could facilitate convergence to equilibrium. Motivated by this observation, we introduce a flexible first-order method that successfully exploits such hidden structures, ensuring convergence under minimal assumptions regarding the transformation that maps players’ control variables to the game’s underlying convex-structured layer.

As a primary application, we explore Convex Markov Games (cMGs)—a recently emerging framework that generalizes traditional Markov games by incorporating convex utility functions over the state-action occupancy measure. This formulation enables a richer spectrum of player preferences, better reflecting real-world applications. In this setting, each player’s occupancy measure captures the long-term frequency of visiting states in their respective Markov Decision Process (MDP), potentially over an infinite time horizon. cMGs have broad applicability across multi-agent learning, including: Fostering creativity in machine gameplay, such as discovering novel strategies in chess, Enhancing exploration in robotic systems, Ensuring safety in autonomous driving, Facilitating imitation learning from expert demonstrations, and Promoting robustness and fairness in multi-agent decision-making.

Given the versatility and importance of cMGs, a fundamental question arises:  Are there algorithmic solutions for provably computing equilibria in these games?

Surprisingly, despite their practical appeal, existing methods fall short of providing general-purpose equilibrium computation for cMGs. In this talk, we will outline key algorithmic challenges that arise in these settings and present new techniques that address them, shedding light on how equilibrium computation can be made tractable in convex Markov games.