Virtualizing Data Infrastructures

Abstract
Organizations (e.g., corporations, educational institutions, nonprofits) usually manage their data using multiple specialized database engines (e.g., Aurora, PostgreSQL, Snowflake) to take advantage of each engine’s individual strengths on different parts of their workload. However, designing and managing such multi-engine infrastructures is difficult; there can be many designs, each offering different performance and a different cost. Changing the design later (e.g., due to business growth) is also challenging since application code can end up tightly coupled to the engines, making migrations risky.

In this talk, I will present data infrastructure virtualization: a new unifying framework that addresses these challenges while also enabling new optimization opportunities. The key idea is to declare a set of virtual database engines (VDBEs), which specify an engine’s application-facing properties (e.g., query interface and performance) and its tables, but do not prescribe a concrete engine. Crucially, each VDBE represents a distinct logical snapshot of its tables, which allows the same table to appear in multiple VDBEs serving different data use cases (e.g., a ticket sales VDBE and a sales reporting VDBE). An automated planner (or a data architect) can then map these VDBEs to one or more physical engines based on the workload as well as cost and performance constraints. Clients connect to VDBE endpoints and remain oblivious to the underlying physical engines, allowing for transparent infrastructure changes. Our experimental results show that automated infrastructure design with VDBEs achieves 1.6x–13x lower cost than using existing cloud serverless engines, and enables non-invasively using workload-specific query execution techniques, achieving geomean speedups of up to 1.76x on three distinct workloads.

Biography
Geoffrey is a postdoctoral associate at MIT in the Data Systems Group and completed his PhD there in August 2026, advised by Tim Kraska. Geoffrey’s research interests are in databases, systems, and more recently, in LLM-driven systems synthesis. His PhD research was on “virtualizing databases”, which introduced new methods to automatically design multi-database infrastructures. He has also done research on key-value stores and GPU performance modeling for neural network training. Geoffrey is a recipient of the NSERC CGS D, CGS M, and Snap Research Scholarship.