Talk: Crafting Human-AI Collaborative Analysis for Usability Evaluations

Abstract:  In Human-Computer Interaction (HCI), delivering an excellent user experience (UX) is crucial to the success and adoption of digital products. A key component of UX is usability, which measures how easily users can effectively, efficiently, and satisfactorily achieve their goals. Usability testing, where users use an interface to uncover pain points, plays a vital role in improving user interface design. However, traditional methods of analyzing usability tests—where UX evaluators watch usability test videos, take notes, and manually identify problems—are time-consuming and prone to errors, bias, and inconsistencies.

To overcome these limitations, my research proposes a human-AI collaborative approach that enhances both the efficiency and accuracy of usability analysis. In this talk, I discuss how AI can work alongside human evaluators through four key areas: (1) representations of AI, such as visualizations or conversational assistants, (2) interaction modalities, comparing voice versus text-based interactions, (3) the timing of AI suggestions, whether they occur before, during, or after usability problems appear in the video, and (4) varying levels of AI expertise, ranging from novice to expert suggestions. In highlighting these factors, my work empowers UX evaluators to detect usability problems effectively, leading to improved UX design outcomes. I conclude by discussing potential future directions for human-AI collaboration, highlighting how AI-powered systems can evolve to address diverse user needs and support more adaptive, inclusive usability testing.

Bio: Emily Kuang is a PhD candidate in the Golisano College of Computing and Information Sciences at Rochester Institute of Technology (RIT), where she is a member of the Center for Accessibility and Inclusion Research (CAIR). She earned her BASc in Biomedical Engineering from the University of Waterloo. Emily’s research lies at the intersection of Human-Computer Interaction (HCI), Artificial Intelligence (AI), and User Experience (UX), focusing on designing and evaluating human-AI collaborative tools to improve usability analysis. She also leads accessibility initiatives aimed at fostering inclusive computing education and developing assistive technologies for diverse user communities.

Emily’s work has been published in top HCI venues such as CHI, CSCW, TVCG, VIS, DIS, and ASSETS. She has received many prestigious awards, including the Google PhD Fellowship in HCI (one of only eight awarded globally in 2023), the inaugural Outstanding Graduate Student Award at RIT (one of two PhD students schoolwide), a Best Paper Honorable Mention (Top 5%) at CHI 2024, a Best Paper at VAHC 2021 (top one in the workshop), and the AWARE-AI NSF Research Traineeship Seed Funding Award (only one trainee in her cohort). Emily has also gained industry experience through her work at Autodesk Research and Meta Reality Labs, where she contributed to products like Quest 3 and Ray-Ban Stories.