Machine Learning Lunch Meeting

Everyone is invited to the weekly machine learning lunch meetings, where our faculty members from Computer Science, Statistics, ECE, and other departments will discuss their latest groundbreaking research in machine learning. This is an opportunity to network with faculty and fellow researchers while learning about the cutting-edge research being conducted at our university. See https://sites.google.com/view/wiscmllm/home for more information.

Speaker: Yongyi Guo (STAT)

Abstract: In contrast to traditional clinical trials, digital health interventions facilitate adaptive personalized treatments delivered in near real-time to manage health risks and promote healthy behaviors. Integrating Reinforcement Learning (RL) algorithms into mHealth (mobile health) studies presents numerous challenges, with a critical one being the constrained time horizon leading to data scarcity, affecting decision quality, as well as the autonomy and stability of RL algorithms in practical applications.

To address this challenge, we propose a solution for online decision-making and post-study statistical inference. Leveraging the mixed-effects reward model in Thompson sampling, we efficiently utilize user data to expedite informed decision-making. The online algorithm makes traditional statistical analysis for the treatment effect invalid: The user history are not independent even if we assume the potential outcomes are i.i.d. This is because the RL algorithm makes decisions using pooled user information in addition to the user state variables. We provide valid asymptotic confidence intervals for the average causal excursion effect using the idea of decomposing the policy into ’population statistics’ and decisions based on ‘(expanded) user states’. As an example use case, I will also present the MiWaves clinical trial, which is an AI-based mobile health intervention to reduce cannabis use amongst emerging adults.