Talk: Randomness-Aware Testing of Machine Learning-based Systems

LIVE STREAM LINK: https://uwmadison.zoom.us/j/93065314047?pwd=OUdvNXFIVUxkTHNHekNwY3kxTWxmQT09

Abstract: The goal of my research is to develop novel testing techniques and tools to make Machine Learning-based systems more reliable. Machine Learning is rapidly revolutionizing the way modern-day systems are developed. However, testing Machine Learning-based systems is challenging due to 1) the presence of non-determinism, both internal (e.g., stochastic algorithms) and external (e.g., execution environment) and 2) the absence of well-defined accuracy specifications. Most traditional software testing techniques that are widely used to improve software reliability cannot tackle these challenges because they often assume determinism and require a test oracle.

In this talk, I will present my work on automated testing of Machine Learning-based systems and on improving developer-written tests in such systems. To achieve these goals, I develop principled techniques that build on solid mathematical foundations provided by probability theory and statistics to reason about the underlying non-determinism and accuracy. I implement my techniques into practical and scalable tools that help developers detect more bugs and efficiently navigate the trade-offs between the quality and efficiency of their tests. My research has exposed more than 50 bugs and improved the quality of more than 200 tests in over 60 popular open-source ML libraries, many of which are widely used at companies like Microsoft, Google, Meta, Uber, and DeepMind as well as in many academic and scientific communities.

Bio: Saikat Dutta is a PhD Candidate in Computer Science at University of Illinois Urbana-Champaign, where he is advised by Prof. Sasa Misailovic. Saikat earned his bachelor's degree in Computer Science from Jadavapur University, India. Saikat’s research interests lie at the intersection of Software Engineering and Machine Learning. Saikat’s current research focuses on improving the reliability of Machine-learning based systems by developing novel testing techniques and tools. Saikat has also received the Facebook PhD Fellowship, 3M Foundation Fellowship, and the Mavis Future Faculty Fellowship for his contributions.