Assuring Cognitive Systems with Dr. Karen Haigh

Abstract: Cognitive systems operate in environments where they must autonomously interpret sensor data, choose actions, and learn from their own experiences, often from sparse or novel information. Because these systems continue to evolve after deployment with limited human supervision, traditional verification, validation, and test methods designed for deterministic or offline-trained models are no longer sufficient. Assuring cognitive systems requires shifting the focus from validating a fixed model to evaluating the learning process itself. Effective assessment must consider the nature and quality of data, operational constraints, and goals such as safety, explainability, robustness, and computational feasibility. Static test vectors and prescripted scenarios cannot capture the closed-loop, interactive behavior of systems that adapt in real time. This presentation outlines the paradigm shifts required to test truly cognitive systems and identifies essential requirements for effective test processes. Rather than asking whether a system performs correctly under fixed conditions, we must determine whether it can learn effectively, adapt safely, and remain reliable under uncertainty, adversity, and adversarial pressure. 

Speaker Bio: Dr. Karen Haigh is an internationally recognized expert in cognitive radio frequency systems and embedded AI. Her focus is on physical systems with limited communications and limited computation resources that must perform under fast hard-real-time requirements. She was a pioneer in three fields now common across the globe: (1) closed-loop planning and machine learning for autonomous robots, (2) smart homes for elder care, and (3) cognitive radio frequency systems. She wrote the book "Cognitive Electronic Warfare: An Artificial Intelligence Approach" with Julia Andrusenko, now in its second edition.  Dr. Haigh has created a variety of online content discussing embedded AI for mission-critical systems, supporting rapid real-time in-mission learning, and assuring AI in the field. She received her PhD in from Carnegie Mellon University in Computer Science with a focus on AI and Robotics, and her undergraduate from the University of Ottawa in Computer Science. She is a Fellow of IEEE and AAIA.