{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "# Demo 1 — A ReAct trace for game theory\n\nThis Week 1 demonstration makes the **observe → reason → act** loop concrete. We first ask Claude Sonnet 4.5 to analyze a one-shot game, then let the model choose actions over three rounds of an iterated Prisoner's Dilemma while the notebook supplies observations.\n\nAll course calls use the exact TAMU gateway settings documented in the Summer 2026 API guide:\n\n- endpoint: `https://chat.tamu.ai/api`\n- model: `protected.Claude Sonnet 4.5`\n- Claude thinking-mode parameters: `temperature=1`, `max_tokens=16384`\n- credentials: `TAMU_API_KEY` plus `CF_COOKIE`, whose value has the form `CF_Authorization=eyJ...`\n\nThe shared API key comes from Canvas. The personal `CF_Authorization` cookie comes from an authenticated `https://chat.tamu.ai` browser session and expires after about 24 hours. For safety, this notebook reads credentials from environment variables or prompts without echoing; it never stores them in the notebook.\n\n> Budget reminder: each student has a gateway-enforced **$5/day** allowance. This notebook prints token use and an approximate cost. The TAMU gateway is authoritative for billing.\n"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": "import getpass\nimport os\nimport re\nfrom openai import OpenAI\n\nTAMU_BASE_URL = \"https://chat.tamu.ai/api\"\nSONNET_MODEL = \"protected.Claude Sonnet 4.5\"\nSONNET_TEMPERATURE = 1\nSONNET_MAX_TOKENS = 16_384\n\ndef normalize_cf_cookie(raw: str) -> str:\n    \"\"\"Accept either eyJ... or CF_Authorization=eyJ... and return a Cookie header.\"\"\"\n    raw = raw.strip().strip('\"').strip(\"'\")\n    if raw.lower().startswith(\"cookie:\"):\n        raw = raw.split(\":\", 1)[1].strip()\n    match = re.search(r\"CF_Authorization=([^;\\s]+)\", raw)\n    if match:\n        return f\"CF_Authorization={match.group(1)}\"\n    if raw.startswith(\"eyJ\"):\n        return f\"CF_Authorization={raw}\"\n    raise ValueError(\"Expected eyJ... or CF_Authorization=eyJ...\")\n\ndef make_tamu_client() -> OpenAI:\n    api_key = os.environ.get(\"TAMU_API_KEY\") or getpass.getpass(\n        \"Paste the course TAMU_API_KEY from Canvas (hidden): \"\n    )\n    raw_cookie = (\n        os.environ.get(\"CF_COOKIE\")\n        or os.environ.get(\"CF_AUTHORIZATION\")\n        or getpass.getpass(\"Paste your CF_Authorization cookie (hidden): \")\n    )\n    return OpenAI(\n        api_key=api_key,\n        base_url=TAMU_BASE_URL,\n        default_headers={\"Cookie\": normalize_cf_cookie(raw_cookie)},\n    )\n\nclient = make_tamu_client()\n\n# A real, minimal completion verifies both authentication and model routing.\nsmoke = client.chat.completions.create(\n    model=SONNET_MODEL,\n    messages=[{\"role\": \"user\", \"content\": \"Reply with exactly: READY\"}],\n    temperature=SONNET_TEMPERATURE,\n    max_tokens=SONNET_MAX_TOKENS,\n)\nprint(f\"Connected to {TAMU_BASE_URL}\")\nprint(f\"Pinned model requested: {SONNET_MODEL}\")\nprint(f\"Gateway model returned: {getattr(smoke, 'model', 'not reported')}\")\nprint(f\"Visible response: {smoke.choices[0].message.content.strip()}\")\nprint(f\"Tokens: {getattr(smoke.usage, 'total_tokens', 'not reported')}\")\n"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "## ReAct: Reason → Act → Observe → loop\n\nReAct (Yao et al., ICLR 2023) interleaves language-model reasoning with actions and environment feedback:\n\n1. **Reason:** use the current observation to form a short, task-relevant rationale.\n2. **Act:** choose a permitted action, such as `COOPERATE` or `DEFECT`.\n3. **Observe:** receive the opponent's action and the resulting payoff.\n4. **Loop:** add that observation to the history before choosing again.\n\nThe model's private chain-of-thought is not needed and is not printed. We request a concise **visible reason summary** so the class can audit the decision without treating hidden reasoning as an observable game action.\n"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": "# This cell is self-contained: it can be rerun without relying on Cell 2.\nimport getpass\nimport os\nimport re\nfrom openai import OpenAI\n\nTAMU_BASE_URL = \"https://chat.tamu.ai/api\"\nSONNET_MODEL = \"protected.Claude Sonnet 4.5\"\n\ndef _cookie(raw):\n    raw = raw.strip().strip('\"').strip(\"'\")\n    match = re.search(r\"CF_Authorization=([^;\\s]+)\", raw)\n    return f\"CF_Authorization={match.group(1)}\" if match else (\n        f\"CF_Authorization={raw}\" if raw.startswith(\"eyJ\") else raw\n    )\n\nif \"client\" not in globals():\n    key = os.environ.get(\"TAMU_API_KEY\") or getpass.getpass(\"TAMU_API_KEY: \")\n    raw = os.environ.get(\"CF_COOKIE\") or getpass.getpass(\"CF_COOKIE: \")\n    client = OpenAI(api_key=key, base_url=TAMU_BASE_URL,\n                    default_headers={\"Cookie\": _cookie(raw)})\n\npd_prompt = \"\"\"\nAnalyze this one-shot Prisoner's Dilemma. Row and Column each choose C or D.\n\n             Column C    Column D\nRow C          (3, 3)       (0, 5)\nRow D          (5, 0)       (1, 1)\n\nIdentify every Nash equilibrium. Give a concise, checkable derivation based on\nunilateral deviations; do not reveal private chain-of-thought.\n\"\"\".strip()\n\nresponse = client.chat.completions.create(\n    model=SONNET_MODEL,\n    messages=[{\"role\": \"user\", \"content\": pd_prompt}],\n    temperature=1,\n    max_tokens=16_384,\n)\n\n# The OpenAI-compatible response separates hidden reasoning from visible content.\n# We print only the visible answer, never message.reasoning_content.\nraw_visible_response = response.choices[0].message.content\nprint(\"PROMPT\\n------\")\nprint(pd_prompt)\nprint(\"\\nRAW VISIBLE RESPONSE\\n--------------------\")\nprint(raw_visible_response)\nprint(f\"\\nModel requested: {SONNET_MODEL}\")\nprint(f\"Total tokens: {getattr(response.usage, 'total_tokens', 'not reported')}\")\n\n# Expected mathematical checkpoint (independent of the stochastic wording):\n# Defect strictly dominates Cooperate for both players, so (D, D) is the unique NE.\n"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "## What happened in the one-shot call?\n\n- **Reason:** the model compared each player's payoff after a unilateral switch.\n- **Act:** here the \"action\" was an answer naming the equilibrium, rather than a game move.\n- **Observe:** the notebook displayed the gateway's visible response, model route, and token count.\n\nThe mathematical checkpoint is deterministic even when wording is not: `D` strictly dominates `C` for both players, so `(D, D)` is the unique Nash equilibrium. If the response disagrees, the trace has revealed an agent error rather than changing the game.\n\nNext we use genuine repeated interaction. The opponent follows the fixed sequence `C, D, D`; the agent sees only past observations, not future moves.\n"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": "# Self-contained three-loop ReAct demonstration.\nimport getpass\nimport json\nimport os\nimport re\nfrom openai import OpenAI\n\nTAMU_BASE_URL = \"https://chat.tamu.ai/api\"\nSONNET_MODEL = \"protected.Claude Sonnet 4.5\"\nPAYOFFS = {\n    (\"COOPERATE\", \"COOPERATE\"): -1,\n    (\"COOPERATE\", \"DEFECT\"): -3,\n    (\"DEFECT\", \"COOPERATE\"): 0,\n    (\"DEFECT\", \"DEFECT\"): -2,\n}\n\ndef _cookie(raw):\n    raw = raw.strip().strip('\"').strip(\"'\")\n    match = re.search(r\"CF_Authorization=([^;\\s]+)\", raw)\n    return f\"CF_Authorization={match.group(1)}\" if match else (\n        f\"CF_Authorization={raw}\" if raw.startswith(\"eyJ\") else raw\n    )\n\ndef _visible_json(text):\n    \"\"\"Parse JSON even if the model surrounds it with a Markdown code fence.\"\"\"\n    text = re.sub(r\"<think>.*?</think>\", \"\", text, flags=re.DOTALL).strip()\n    fenced = re.search(r\"```(?:json)?\\s*(\\{.*?\\})\\s*```\", text, re.DOTALL)\n    candidate = fenced.group(1) if fenced else text[text.find(\"{\"):text.rfind(\"}\") + 1]\n    data = json.loads(candidate)\n    action = str(data[\"action\"]).upper()\n    if action not in {\"COOPERATE\", \"DEFECT\"}:\n        raise ValueError(f\"Invalid action: {action}\")\n    return str(data[\"reason_summary\"]), action\n\nif \"client\" not in globals():\n    key = os.environ.get(\"TAMU_API_KEY\") or getpass.getpass(\"TAMU_API_KEY: \")\n    raw = os.environ.get(\"CF_COOKIE\") or getpass.getpass(\"CF_COOKIE: \")\n    client = OpenAI(api_key=key, base_url=TAMU_BASE_URL,\n                    default_headers={\"Cookie\": _cookie(raw)})\n\nsystem_prompt = \"\"\"\nYou are playing a three-round iterated Prisoner's Dilemma.\nYour payoffs are (C,C)=-1, (C,D)=-3, (D,C)=0, and (D,D)=-2.\nMaximize cumulative payoff. After each observation, return exactly one JSON object:\n{\"reason_summary\": \"one short visible decision rationale\", \"action\": \"COOPERATE or DEFECT\"}\nDo not provide private chain-of-thought or any text outside the JSON object.\n\"\"\".strip()\n\nmessages = [{\"role\": \"system\", \"content\": system_prompt}]\nopponent_schedule = [\"COOPERATE\", \"DEFECT\", \"DEFECT\"]\nhistory = []\ncumulative_payoff = 0\nprompt_tokens = completion_tokens = 0\n\nprint(\"=== ReAct Agent: Iterated Prisoner's Dilemma ===\")\nprint(f\"Model: {SONNET_MODEL}\\nAPI: {TAMU_BASE_URL}\\nBudget: $5.00/day\")\n\nfor round_number, opponent_action in enumerate(opponent_schedule, start=1):\n    observation = (\n        \"No prior history.\" if not history else\n        \"History: \" + \"; \".join(\n            f\"round {h['round']}: you={h['agent']}, opponent={h['opponent']}, payoff={h['payoff']}\"\n            for h in history\n        )\n    )\n    user_message = f\"Round {round_number}. {observation} Choose your action.\"\n    messages.append({\"role\": \"user\", \"content\": user_message})\n\n    response = client.chat.completions.create(\n        model=SONNET_MODEL,\n        messages=messages,\n        temperature=1,\n        max_tokens=16_384,\n    )\n    visible = response.choices[0].message.content\n    reason_summary, agent_action = _visible_json(visible)\n    messages.append({\"role\": \"assistant\", \"content\": visible})\n\n    payoff = PAYOFFS[(agent_action, opponent_action)]\n    cumulative_payoff += payoff\n    result = {\n        \"round\": round_number,\n        \"agent\": agent_action,\n        \"opponent\": opponent_action,\n        \"payoff\": payoff,\n    }\n    history.append(result)\n    messages.append({\n        \"role\": \"user\",\n        \"content\": (\n            f\"Observation: you played {agent_action}; opponent played {opponent_action}; \"\n            f\"round payoff {payoff}; cumulative payoff {cumulative_payoff}.\"\n        ),\n    })\n\n    usage = response.usage\n    prompt_tokens += getattr(usage, \"prompt_tokens\", 0) or 0\n    completion_tokens += getattr(usage, \"completion_tokens\", 0) or 0\n    print(f\"\\n--- Loop {round_number} ---\")\n    print(f\"Reason summary: {reason_summary}\")\n    print(f\"Action: play({agent_action.lower()})\")\n    print(f\"Observation: opponent={opponent_action}; payoff={payoff}; cumulative={cumulative_payoff}\")\n\nestimated_cost = prompt_tokens / 1_000_000 * 3 + completion_tokens / 1_000_000 * 15\nprint(\"\\n=== Demo complete ===\")\nprint(f\"Final cumulative payoff: {cumulative_payoff}\")\nprint(f\"Prompt tokens: {prompt_tokens}; completion tokens: {completion_tokens}\")\nprint(f\"Approximate Sonnet cost: ${estimated_cost:.4f} of the $5.00 daily budget\")\n\n# Illustrative shape only (the live actions and counts may differ):\n# Loop 1: reason summary -> Action -> environment Observation\n# Loop 2: updated reason summary -> Action -> new Observation\n# Loop 3: updated reason summary -> Action -> final Observation\n"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "## Wrap-up and PA1 bridge\n\nNotice three separations:\n\n1. The **game** fixes actions and payoffs; the LLM is only a stochastic policy.\n2. A visible reason summary can help diagnose behavior, but the submitted action and observed payoff are the empirical data.\n3. One trace is a trajectory, not a game tree and not a payoff estimate.\n\nPA1 turns this qualitative trace into data: fix prompt strategies, repeat every strategy profile, estimate mean payoffs with confidence intervals, and then compute equilibria. Save model IDs, prompts, seeds, sample counts, token use, and failures so another researcher can reproduce the empirical game.\n\n**Reference:** Shunyu Yao et al. (2023, ICLR), *ReAct: Synergizing Reasoning and Acting in Language Models*.\n"
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}