Advanced Topics in Computer Science: Systems for Serving Generative AI

Generative machine learning models, from large language models for chatbots to diffusion models for text-to-image generation, have become key players in important decisions and tasks across societal sectors. Unfortunately, their abilities to engage in conversation and spark creativity is coupled with high costs (both monetary and compute resource) and sometimes sluggish performance, especially when contextualized relative to the millions of requests that such models face for serving in production services. This research-centric course examines a wide range of systems optimizations that enable large-scale serving of generative models.