Talk: Enhancing the Interaction Capabilities of Robots

LIVE STREAM: TBD

Abstract: Robots are poised to become integral in our everyday lives, helping us with activities like driving and chores, but their seamless integration necessitates interacting with people effectively. In this talk, I will describe three threads of my research aimed at enhancing the interaction capabilities of robots, which I argue must match their technical capabilities such as manipulating objects. First, robots must be able to coordinate with people. To enable coordination during ad-hoc human-robot collaboration on tabletops, I introduce new explicit and implicit techniques inspired by human teaming to allow users to allocate actions to robots. Second, robots must be able to communicate with people to navigate complex social situations. To that end, I propose Generative Expressive Motion, a new method that leverages the rich social context and code-writing capabilities of large language models to generate expressive robot behaviors based on natural language instructions from humans. Lastly, users must be able to instruct robots to complete manipulation tasks based on their requirements. I present ImageInThat, which allows users to directly manipulate images representing a robot’s observations of its environment—such as repositioning objects—to create instructions the robot can later execute.