Machine Learning Lunch Meeting

Everyone is invited to the weekly machine learning lunch meetings, where our faculty members from Computer Science, Statistics, ECE, and other departments will discuss their latest groundbreaking research in machine learning. This is an opportunity to network with faculty and fellow researchers while learning about the cutting-edge research being conducted at our university. See https://sites.google.com/view/wiscmllm/home for more information.

Speaker: Junjie Hu (BMI)

Abstract: Large foundation models, such as GPT-4, have acquired extensive knowledge through large-scale pre-training and have subsequently been refined through instruction fine-tuning and alignment learning methods such as RLHF/DPO. Despite their potential, ensuring the alignment of these models with complex real-world knowledge and diverse human values poses a challenging yet vital task for their safe and effective deployment at scale. In this talk, I will introduce our work on aligning the “interlingual” knowledge of vision-language foundation models using an efficient visual tokenization technique. Our approach enhances the understanding of detailed visual concepts and enables the generalization of novel visual concepts seamlessly through multimodal in-context prompting. Furthermore, beyond merely aligning these models’ internal world knowledge, I will delve into our research on simulating opinion dynamics using foundation models to analyze their alignment with human behavior. Additionally, we will discuss our ongoing work on developing a multi-community alignment framework aimed at aligning foundation models with diverse human values from various communities, thereby promoting AI inclusivity and safety.