Talk: Bridging the Semantic Gap between Autonomous System Requirements and Complex Sensor Data

Live stream: TBD

Full title: Bridging the Semantic Gap between Autonomous System Requirements and Complex Sensor Data or How does your self-driving car know it is safe?

Abstract: Autonomous systems perform safety-critical tasks by interpreting their complex environments through advanced, semantically rich sensor inputs such as LiDAR point clouds and camera images. To safely complete their missions, these systems must translate their sensor inputs into an accurate and comprehensive internal representation of the world. Simultaneously, developers must be able to utilize this internal representation to assess the system’s adherence to safety requirements. For instance, when processing a camera image of the road ahead, an autonomous vehicle must construct a representation that includes details about stop signs, the lanes they govern, and the vehicle’s own position within the lanes, enabling it to reason effectively about its obligation to stop at the sign. Without aligning sensor data with safety requirements, reasoning about system safety becomes impossible. The key challenge lies in bridging this semantic gap by effectively leveraging the proper internal world representation.

In this talk I first introduce this novel problem formulation and how it enables a richer understanding of autonomous system safety in practice. Next, I propose the use of scene graphs as a suitable world representation to connect complex sensor data and formalized autonomous system safety requirements; in particular as a means for autonomous vehicles to reason about compliance with traffic laws. I introduce methods for using scene graphs to reason about test coverage as measured by how well a test suite exercises the requirements. Then, I demonstrate a novel approach for encoding traffic laws into a runtime verification framework that leverages scene graphs to enable autonomous vehicles to reason about their compliance in the field. Finally, I conclude by discussing future directions that will enable not only validating and verifying autonomous system safety but driving the development process to produce safer systems.

Bio: Trey Woodlief is a final-year Ph.D. candidate in Computer Science at the University of Virginia advised by Sebastian Elbaum and Kevin Sullivan. His research interests lie at the intersection of robotics and software engineering, with a focus on understanding and improving autonomous system safety by developing novel verification and validation techniques tailored to the unique constraints of the robotics domain. Results of his work have appeared in top venues in robotics and software engineering including ICRA, IROS, and ICSE. He is the recipient of multiple prestigious honors, including the UVA School of Engineering and Applied Science Copenhaver Charitable Trust Bicentennial Fellowship, the UVA SEAS Dean’s Scholar Fellowship, and the UVA Computer Science Outstanding Graduate Research Award. His teaching and mentorship have also been recognized with the UVA All-University Graduate Teaching Award and his contributions in supporting STEM education through FIRST robotics were recognized with the global Volunteer of the Year award.