M5: CFR and Self-Play
Counterfactual Regret Minimization is the algorithm that made AI superhuman at poker, Texas Hold'em, and Hearthstone. We build from external and internal regret to the full CFR algorithm: decomposition, convergence guarantees, and implementation techniques. For LLM agents, CFR supplies a computational backbone for self-play training—the pattern by which agents improve by playing against themselves that recurs throughout LLM research. PA4 extends this thread into multi-agent debate and mechanism design.
Module preview
Counterfactual Regret Minimization is the algorithm that made AI superhuman at poker, Texas Hold'em, and Hearthstone. We build from external and internal regret to the full CFR algorithm: decomposition, convergence guarantees, and implementation techniques. For LLM agents, CFR supplies a computational backbone for self-play training—the pattern by which agents improve by playing against themselves that recurs throughout LLM research. PA4 extends this thread into multi-agent debate and mechanism design.
Lectures and materials
L13: CFR Algorithm
Counterfactual decomposition and self-play in imperfect-information games.
L14: Monte Carlo CFR and Sampling
Outcome and external sampling for Monte Carlo CFR.
Programming assignment
PA4: Multi-Agent Debate · Due November 21 at 11:59 PM CDT