M8: AI Safety and Monitoring
As autonomous systems grow more capable, monitoring and oversight become critical. This module covers interpretability, alignment, and the submodular optimization methods that deploy monitoring resources efficiently. Building on M3 trigger strategies and M6 evaluation methods, we study how to detect deviations from expected behavior and design oversight that scales. PA5 makes you the AI controller: you will build a monitoring system for a multi-agent environment.
Module preview
As autonomous systems grow more capable, monitoring and oversight become critical. This module covers interpretability, alignment, and the submodular optimization methods that deploy monitoring resources efficiently. Building on M3 trigger strategies and M6 evaluation methods, we study how to detect deviations from expected behavior and design oversight that scales. PA5 makes you the AI controller: you will build a monitoring system for a multi-agent environment.
Lectures and materials
L18: Monitoring Autonomous Agents
Interpretability, alignment, and detecting unexpected behavior.
L19: Robust Submodularity, the Interventionist Greedy Game, and Poison Pills
Adversarial monitoring: robust submodular optimization, the Interventionist Greedy Game, the off-limits model, and why non-monotone objectives are harder to monitor.
Programming assignment
PA5: AI Control · Due December 5 at 11:59 PM CDT