{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "# Demo 2 — Empirical payoff estimation with confidence intervals\n\nLLM policies are stochastic: the same prompt can produce a different action on the next call. Therefore an LLM-agent payoff matrix is **estimated from repeated play**, not copied from a single transcript.\n\nThis notebook implements the Week 1 / PA1 pipeline for a 2×2 Prisoner's Dilemma:\n\n1. Treat `cooperate` and `defect` system prompts as two agent strategies.\n2. Run 30 game episodes for each of the four prompt profiles.\n3. Query both players in every episode (240 Sonnet calls total).\n4. Report empirical mean payoff and an approximate 95% confidence interval for every cell.\n\nCore calls are pinned to `protected.Claude Sonnet 4.5` at `https://chat.tamu.ai/api`. Credentials are read from `TAMU_API_KEY` and `CF_COOKIE` (`CF_Authorization=eyJ...`) or entered through hidden prompts. The notebook does not store credentials.\n\n> Cost warning: the full cell makes 240 calls against the **$5/day** student allowance. Keep `N_REPS = 30` for the recorded demonstration; reduce it only for a smoke test and disclose the actual sample count.\n"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": "import getpass\nimport os\nimport re\nfrom openai import OpenAI\n\nTAMU_BASE_URL = \"https://chat.tamu.ai/api\"\nSONNET_MODEL = \"protected.Claude Sonnet 4.5\"\nN_REPS = 30\nSEED = 42\n\nSTRATEGY_PROMPTS = {\n    \"cooperate\": (\n        \"You are playing a one-shot Prisoner's Dilemma. You should cooperate \"\n        \"with the other player. Respond with exactly one word: COOPERATE or DEFECT.\"\n    ),\n    \"defect\": (\n        \"You are playing a one-shot Prisoner's Dilemma. You should defect \"\n        \"against the other player. Respond with exactly one word: COOPERATE or DEFECT.\"\n    ),\n}\nPD_PAYOFFS = {\n    (\"COOPERATE\", \"COOPERATE\"): (3, 3),\n    (\"COOPERATE\", \"DEFECT\"): (0, 5),\n    (\"DEFECT\", \"COOPERATE\"): (5, 0),\n    (\"DEFECT\", \"DEFECT\"): (1, 1),\n}\n\ndef normalize_cf_cookie(raw):\n    raw = raw.strip().strip('\"').strip(\"'\")\n    match = re.search(r\"CF_Authorization=([^;\\s]+)\", raw)\n    return f\"CF_Authorization={match.group(1)}\" if match else (\n        f\"CF_Authorization={raw}\" if raw.startswith(\"eyJ\") else raw\n    )\n\ndef make_client():\n    key = os.environ.get(\"TAMU_API_KEY\") or getpass.getpass(\"TAMU_API_KEY: \")\n    raw = os.environ.get(\"CF_COOKIE\") or getpass.getpass(\"CF_COOKIE: \")\n    return OpenAI(api_key=key, base_url=TAMU_BASE_URL,\n                  default_headers={\"Cookie\": normalize_cf_cookie(raw)})\n\nclient = make_client()\nprint(f\"Ready: {TAMU_BASE_URL}\")\nprint(f\"Pinned model: {SONNET_MODEL}\")\nprint(f\"Experiment: 4 cells × {N_REPS} episodes × 2 agents = {8 * N_REPS} API calls\")\n"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "## Estimation procedure\n\nFor prompt profile `(i, j)`, repeat the following independently `N = 30` times:\n\n1. Query Player 1 using strategy prompt `i`.\n2. Query Player 2 using strategy prompt `j`.\n3. Parse each output as `COOPERATE` or `DEFECT`.\n4. Score the realized action pair with the theoretical Prisoner's Dilemma payoff table.\n\nFor each player and prompt profile, report\n\n\\[\n\\widehat{u}_{ij} = \\frac{1}{N}\\sum_{r=1}^{N} u_{ij}^{(r)},\n\\qquad\n\\widehat{u}_{ij} \\pm 1.96\\frac{s_{ij}}{\\sqrt{N}}.\n\\]\n\nThis is a normal-approximation 95% confidence interval for the mean. It describes sampling uncertainty under this exact model, prompt, and gateway configuration; it does not make the estimate transferable to another model version.\n"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": "# Self-contained live estimator. Rerunning resets the data instead of appending duplicates.\nimport getpass\nimport itertools\nimport math\nimport os\nimport re\nimport statistics\nimport time\nimport pandas as pd\nfrom openai import OpenAI, APIConnectionError, APIError, RateLimitError\n\nTAMU_BASE_URL = \"https://chat.tamu.ai/api\"\nSONNET_MODEL = \"protected.Claude Sonnet 4.5\"\nN_REPS = 30\nSEED = 42  # Records experiment metadata; API sampling itself is not seed-controlled here.\nSTRATEGY_PROMPTS = {\n    \"cooperate\": \"You are playing a one-shot Prisoner's Dilemma. You should cooperate. Respond with exactly one word: COOPERATE or DEFECT.\",\n    \"defect\": \"You are playing a one-shot Prisoner's Dilemma. You should defect. Respond with exactly one word: COOPERATE or DEFECT.\",\n}\nPD_PAYOFFS = {\n    (\"COOPERATE\", \"COOPERATE\"): (3, 3),\n    (\"COOPERATE\", \"DEFECT\"): (0, 5),\n    (\"DEFECT\", \"COOPERATE\"): (5, 0),\n    (\"DEFECT\", \"DEFECT\"): (1, 1),\n}\n\ndef _cookie(raw):\n    raw = raw.strip().strip('\"').strip(\"'\")\n    match = re.search(r\"CF_Authorization=([^;\\s]+)\", raw)\n    return f\"CF_Authorization={match.group(1)}\" if match else (\n        f\"CF_Authorization={raw}\" if raw.startswith(\"eyJ\") else raw\n    )\n\nif \"client\" not in globals():\n    key = os.environ.get(\"TAMU_API_KEY\") or getpass.getpass(\"TAMU_API_KEY: \")\n    raw = os.environ.get(\"CF_COOKIE\") or getpass.getpass(\"CF_COOKIE: \")\n    client = OpenAI(api_key=key, base_url=TAMU_BASE_URL,\n                    default_headers={\"Cookie\": _cookie(raw)})\n\ndef choose_action(strategy, player_label):\n    messages = [\n        {\"role\": \"system\", \"content\": STRATEGY_PROMPTS[strategy]},\n        {\"role\": \"user\", \"content\": f\"You are {player_label}. Choose your action now.\"},\n    ]\n    for attempt in range(4):\n        try:\n            response = client.chat.completions.create(\n                model=SONNET_MODEL,\n                messages=messages,\n                temperature=1,\n                max_tokens=16_384,\n            )\n            visible = re.sub(\n                r\"<think>.*?</think>\", \"\", response.choices[0].message.content,\n                flags=re.DOTALL,\n            ).strip()\n            actions = re.findall(r\"\\b(COOPERATE|DEFECT)\\b\", visible.upper())\n            if not actions:\n                raise ValueError(f\"Could not parse action from {visible!r}\")\n            usage = response.usage\n            return actions[-1], visible, (\n                getattr(usage, \"prompt_tokens\", 0) or 0,\n                getattr(usage, \"completion_tokens\", 0) or 0,\n            )\n        except (RateLimitError, APIConnectionError, APIError, ValueError) as exc:\n            if attempt == 3:\n                raise\n            delay = 2 ** attempt\n            print(f\"Transient {type(exc).__name__}; retrying in {delay}s...\")\n            time.sleep(delay)\n\nrecords = []\nraw_example = None\nprompt_tokens = completion_tokens = 0\n\nfor p1_strategy, p2_strategy in itertools.product(STRATEGY_PROMPTS, repeat=2):\n    for rep in range(1, N_REPS + 1):\n        a1, text1, usage1 = choose_action(p1_strategy, \"Player 1\")\n        a2, text2, usage2 = choose_action(p2_strategy, \"Player 2\")\n        u1, u2 = PD_PAYOFFS[(a1, a2)]\n        prompt_tokens += usage1[0] + usage2[0]\n        completion_tokens += usage1[1] + usage2[1]\n        record = {\n            \"p1_strategy\": p1_strategy,\n            \"p2_strategy\": p2_strategy,\n            \"rep\": rep,\n            \"p1_action\": a1,\n            \"p2_action\": a2,\n            \"u1\": u1,\n            \"u2\": u2,\n        }\n        records.append(record)\n        if raw_example is None:\n            raw_example = {**record, \"p1_visible_response\": text1, \"p2_visible_response\": text2}\n    print(f\"[{p1_strategy}, {p2_strategy}]: {N_REPS}/{N_REPS} complete\")\n\nepisodes_df = pd.DataFrame(records)\n\ndef mean_ci(values):\n    values = list(values)\n    mean = statistics.fmean(values)\n    half_width = 0.0 if len(values) < 2 else 1.96 * statistics.stdev(values) / math.sqrt(len(values))\n    return mean, half_width\n\nsummary_rows = []\nfor (s1, s2), group in episodes_df.groupby([\"p1_strategy\", \"p2_strategy\"], sort=False):\n    u1_mean, u1_ci = mean_ci(group[\"u1\"])\n    u2_mean, u2_ci = mean_ci(group[\"u2\"])\n    summary_rows.append({\n        \"p1_strategy\": s1, \"p2_strategy\": s2, \"n\": len(group),\n        \"u1_mean\": u1_mean, \"u1_ci95\": u1_ci,\n        \"u2_mean\": u2_mean, \"u2_ci95\": u2_ci,\n    })\n\nestimates_df = pd.DataFrame(summary_rows)\nestimated_cost = prompt_tokens / 1_000_000 * 3 + completion_tokens / 1_000_000 * 15\n\nprint(\"\\n=== Raw transcript: first episode ===\")\nprint(raw_example)\nprint(\"\\n=== Per-cell estimates ===\")\nprint(estimates_df.to_string(index=False, float_format=lambda x: f\"{x:.3f}\"))\nprint(f\"\\nEpisodes: {len(episodes_df)}; API calls: {2 * len(episodes_df)}\")\nprint(f\"Tokens: prompt={prompt_tokens}, completion={completion_tokens}\")\nprint(f\"Approximate Sonnet cost: ${estimated_cost:.4f} of $5.00/day\")\n\n# Illustrative output shape (not a claim about an unrun experiment):\n# p1_strategy p2_strategy  n  u1_mean  u1_ci95  u2_mean  u2_ci95\n# cooperate    cooperate   30    ...       ...       ...       ...\n"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": "# This display cell uses live estimates when available; otherwise it creates clearly\n# labeled illustrative data so the formatting can be taught before spending API credit.\nimport pandas as pd\n\nif \"estimates_df\" not in globals():\n    print(\"No live results found: displaying ILLUSTRATIVE values from the lecture notes.\")\n    estimates_df = pd.DataFrame([\n        {\"p1_strategy\": \"cooperate\", \"p2_strategy\": \"cooperate\", \"n\": 30, \"u1_mean\": 2.83, \"u1_ci95\": 0.24, \"u2_mean\": 2.83, \"u2_ci95\": 0.24},\n        {\"p1_strategy\": \"cooperate\", \"p2_strategy\": \"defect\",    \"n\": 30, \"u1_mean\": 0.47, \"u1_ci95\": 0.30, \"u2_mean\": 4.60, \"u2_ci95\": 0.31},\n        {\"p1_strategy\": \"defect\",    \"p2_strategy\": \"cooperate\", \"n\": 30, \"u1_mean\": 4.60, \"u1_ci95\": 0.31, \"u2_mean\": 0.47, \"u2_ci95\": 0.30},\n        {\"p1_strategy\": \"defect\",    \"p2_strategy\": \"defect\",    \"n\": 30, \"u1_mean\": 1.07, \"u1_ci95\": 0.12, \"u2_mean\": 1.07, \"u2_ci95\": 0.12},\n    ])\n\ndef matrix_with_ci(player):\n    mean_col, ci_col = f\"u{player}_mean\", f\"u{player}_ci95\"\n    labeled = estimates_df.assign(\n        estimate=estimates_df.apply(\n            lambda row: f\"{row[mean_col]:.2f} ± {row[ci_col]:.2f}\", axis=1\n        )\n    )\n    return labeled.pivot(index=\"p1_strategy\", columns=\"p2_strategy\", values=\"estimate\").reindex(\n        index=[\"cooperate\", \"defect\"], columns=[\"cooperate\", \"defect\"]\n    )\n\nprint(\"=== Empirical payoff matrix: Player 1 (mean ± 95% CI) ===\")\ndisplay(matrix_with_ci(1))\nprint(\"=== Empirical payoff matrix: Player 2 (mean ± 95% CI) ===\")\ndisplay(matrix_with_ci(2))\n\nprint(\"Theoretical Player 1 matrix: [[3, 0], [5, 1]]\")\nprint(\"Theoretical Player 2 matrix: [[3, 5], [0, 1]]\")\nprint(\"Compare prompt-profile estimates to theory; disagreement measures policy compliance, not an error in the payoff table.\")\n"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "## Interpretation\n\n- Tight intervals mean the realized payoff under that prompt profile was stable across the sampled episodes. Wide intervals mean action compliance or outcomes varied more.\n- A confidence interval quantifies uncertainty in the **mean payoff**, not uncertainty about the theoretical scoring rule.\n- Large overlap between strategically relevant cells can make a computed equilibrium fragile. PA1 therefore requires sensitivity analysis that perturbs entries within their intervals.\n- Record failures and parsing decisions. Silently dropping malformed outputs can bias the empirical game.\n\nThe illustrative values printed when no live results exist are copied from the lecture's **expected transcript** and are not evidence of an API run.\n"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": "# Bonus: deliberately compare Sonnet 4.5 with one other gateway model.\n# This is the only cell that is not Sonnet-only; a comparison would otherwise be impossible.\nimport getpass\nimport os\nimport re\nimport statistics\nimport pandas as pd\nfrom openai import OpenAI\n\nTAMU_BASE_URL = \"https://chat.tamu.ai/api\"\nSONNET_MODEL = \"protected.Claude Sonnet 4.5\"\nCOMPARISON_MODEL = os.environ.get(\"TAMU_COMPARISON_MODEL\", \"protected.gpt-5-mini\")\nBONUS_REPS = 5\n\ndef _cookie(raw):\n    raw = raw.strip().strip('\"').strip(\"'\")\n    match = re.search(r\"CF_Authorization=([^;\\s]+)\", raw)\n    return f\"CF_Authorization={match.group(1)}\" if match else (\n        f\"CF_Authorization={raw}\" if raw.startswith(\"eyJ\") else raw\n    )\n\nif \"client\" not in globals():\n    key = os.environ.get(\"TAMU_API_KEY\") or getpass.getpass(\"TAMU_API_KEY: \")\n    raw = os.environ.get(\"CF_COOKIE\") or getpass.getpass(\"CF_COOKIE: \")\n    client = OpenAI(api_key=key, base_url=TAMU_BASE_URL,\n                    default_headers={\"Cookie\": _cookie(raw)})\n\ndef model_action(model):\n    kwargs = {\n        \"model\": model,\n        \"messages\": [\n            {\"role\": \"system\", \"content\": \"In a one-shot Prisoner's Dilemma, follow a COOPERATE policy. Reply with exactly COOPERATE or DEFECT.\"},\n            {\"role\": \"user\", \"content\": \"Choose your action.\"},\n        ],\n    }\n    if model == SONNET_MODEL:\n        kwargs.update(temperature=1, max_tokens=16_384)\n    else:\n        kwargs.update(temperature=0.7, max_tokens=64)\n    response = client.chat.completions.create(**kwargs)\n    visible = response.choices[0].message.content.upper()\n    actions = re.findall(r\"\\b(COOPERATE|DEFECT)\\b\", visible)\n    if not actions:\n        raise ValueError(f\"Could not parse {model} response: {visible!r}\")\n    return actions[-1], getattr(response.usage, \"total_tokens\", 0) or 0\n\ncomparison_rows = []\nfor model in [SONNET_MODEL, COMPARISON_MODEL]:\n    outputs = [model_action(model) for _ in range(BONUS_REPS)]\n    cooperate_rate = sum(action == \"COOPERATE\" for action, _ in outputs) / BONUS_REPS\n    comparison_rows.append({\n        \"model\": model,\n        \"n\": BONUS_REPS,\n        \"cooperate_prompt_compliance\": cooperate_rate,\n        \"mean_total_tokens\": statistics.fmean(tokens for _, tokens in outputs),\n    })\n\ndisplay(pd.DataFrame(comparison_rows))\nprint(\"This small bonus is descriptive only; increase N and add CIs before making model-level claims.\")\n"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "## PA1 handoff\n\nExtend this starter in four directions:\n\n1. Add a coordination game and one negotiation game.\n2. Save raw responses and a machine-readable experiment manifest without saving credentials.\n3. Compute Nash equilibria of the estimated games.\n4. Perturb each payoff within its confidence interval and report whether the equilibrium support changes.\n\nEvery result should name the exact model, prompts, gateway endpoint, sample counts, dates, parsing rules, failures, and approximate token use. The $5/day cap rewards careful pilot runs and caching rather than repeated blind execution.\n"
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}