{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Homework\n",
    "\n",
    "- Use this assignment to practice the chapter's concepts and coding skills in runnable Python code.\n",
    "- Sign in from the account menu before you start working on the assignment.\n",
    "- Write and run your code directly in the coding cells.\n",
    "- Multiple submissions are allowed. The highest score is kept.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "::::{div} assignment-question-card\n",
    "\n",
    "**1. Opt-In Comparisons**\n",
    "\n",
    "A store shows a new product page only to customers who click \"Try the new design.\" Those customers buy more often than everyone else. This proves that the new page causes more purchases.\n",
    "\n",
    "<div class=\"tf-options\" role=\"radiogroup\" aria-label=\"Question 1 answer\">\n",
    "  <label><input type=\"radio\" name=\"ch10-hw-q1\" value=\"true\"> True</label>\n",
    "  <label><input type=\"radio\" name=\"ch10-hw-q1\" value=\"false\"> False</label>\n",
    "</div>\n",
    "\n",
    "::::\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [],
   "source": [
    "### Solution\n",
    "# False\n",
    "# Customers chose whether to see the new page, so they may differ from others in ways that also affect buying. Without random assignment, the difference shows association, not causation.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "::::{div} assignment-question-card\n",
    "\n",
    "**2. What Shuffling Simulates**\n",
    "\n",
    "In a permutation test, shuffling the group labels simulates data under the null hypothesis that the label has nothing to do with the outcome.\n",
    "\n",
    "<div class=\"tf-options\" role=\"radiogroup\" aria-label=\"Question 2 answer\">\n",
    "  <label><input type=\"radio\" name=\"ch10-hw-q2\" value=\"true\"> True</label>\n",
    "  <label><input type=\"radio\" name=\"ch10-hw-q2\" value=\"false\"> False</label>\n",
    "</div>\n",
    "\n",
    "::::\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [],
   "source": [
    "### Solution\n",
    "# True\n",
    "# If the label doesn't matter, any assignment of labels to the observed values is equally likely, so shuffled labels show how big a difference chance alone tends to produce.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "::::{div} assignment-question-card\n",
    "\n",
    "**3. Choosing a One-Sided Statistic**\n",
    "\n",
    "The alternative hypothesis says that version B's order values tend to be larger than version A's. \"Mean of B minus mean of A\" is a sensible test statistic, with large values favoring the alternative.\n",
    "\n",
    "<div class=\"tf-options\" role=\"radiogroup\" aria-label=\"Question 3 answer\">\n",
    "  <label><input type=\"radio\" name=\"ch10-hw-q3\" value=\"true\"> True</label>\n",
    "  <label><input type=\"radio\" name=\"ch10-hw-q3\" value=\"false\"> False</label>\n",
    "</div>\n",
    "\n",
    "::::\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [],
   "source": [
    "### Solution\n",
    "# True\n",
    "# Under the alternative, B's mean tends to be larger, so the difference tends to be large and positive.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "::::{div} assignment-question-card\n",
    "\n",
    "**4. A Randomized Trial**\n",
    "\n",
    "Employees were randomly assigned to a new training program or to the usual one. Follow-up and sales measurement were comparable, and participants did not affect one another's outcomes. The new-program group had higher average sales, and a permutation test gives a p-value of 0.002. This is evidence that the new program caused an improvement for these participants.\n",
    "\n",
    "<div class=\"tf-options\" role=\"radiogroup\" aria-label=\"Question 4 answer\">\n",
    "  <label><input type=\"radio\" name=\"ch10-hw-q4\" value=\"true\"> True</label>\n",
    "  <label><input type=\"radio\" name=\"ch10-hw-q4\" value=\"false\"> False</label>\n",
    "</div>\n",
    "\n",
    "::::\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [],
   "source": [
    "### Solution\n",
    "# True\n",
    "# Under the stated design conditions, random assignment supports a causal\n",
    "# comparison. The observed direction is positive, and the small p-value is\n",
    "# evidence against the no-effect model. Randomization does not make the\n",
    "# groups identical or establish that the same effect holds in every setting.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "::::{div} assignment-question-card\n",
    "\n",
    "**5. Small Randomized Studies**\n",
    "\n",
    "A randomized experiment with only 30 customers in each group cannot support a causal conclusion.\n",
    "\n",
    "<div class=\"tf-options\" role=\"radiogroup\" aria-label=\"Question 5 answer\">\n",
    "  <label><input type=\"radio\" name=\"ch10-hw-q5\" value=\"true\"> True</label>\n",
    "  <label><input type=\"radio\" name=\"ch10-hw-q5\" value=\"false\"> False</label>\n",
    "</div>\n",
    "\n",
    "::::\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [],
   "source": [
    "### Solution\n",
    "# False\n",
    "# Random assignment, not sample size, is what supports a causal claim. A small sample makes the estimate less precise and makes a real effect harder to detect.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "tags": [
     "thebe-interactive"
    ]
   },
   "outputs": [],
   "source": [
    "### 6. Basket Size by Segment\n",
    "# Compare the average basket size (in dollars) of new and returning customers.\n",
    "#   1. Use groupby to compute the mean basket for each segment. Keep full\n",
    "#      precision for calculations and round to 2 decimals only when printing.\n",
    "#   2. Compute the distance between the two means, | returning - new |, rounded to 2 decimals.\n",
    "# Print exactly:\n",
    "# New: 30.67\n",
    "# Returning: 33.17\n",
    "# Distance: 2.5\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "baskets = pd.DataFrame({\n",
    "    \"segment\": [\"new\"] * 6 + [\"returning\"] * 6,\n",
    "    \"basket\": [31, 28, 35, 30, 27, 33, 33, 31, 36, 29, 38, 32],\n",
    "})\n",
    "\n",
    "### Your code starts here.\n",
    "\n",
    "\n",
    "\n",
    "### Your code ends here.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "tags": [
     "hide-input",
     "homework-answer"
    ]
   },
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": "New: 30.67\nReturning: 33.17\nDistance: 2.5\n"
    }
   ],
   "source": [
    "### Solution\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "baskets = pd.DataFrame({\n",
    "    \"segment\": [\"new\"] * 6 + [\"returning\"] * 6,\n",
    "    \"basket\": [31, 28, 35, 30, 27, 33, 33, 31, 36, 29, 38, 32],\n",
    "})\n",
    "\n",
    "means = baskets.groupby('segment')['basket'].mean()\n",
    "\n",
    "print(\"New:\", round(means['new'], 2))\n",
    "print(\"Returning:\", round(means['returning'], 2))\n",
    "print(\"Distance:\", round(abs(means['returning'] - means['new']), 2))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "tags": [
     "thebe-interactive"
    ]
   },
   "outputs": [],
   "source": [
    "### 7. Churn by Plan\n",
    "# Each row is a customer, with their plan and whether they cancelled (1) or stayed (0).\n",
    "#   1. Use groupby to compute the churn rate for each plan.\n",
    "#   2. Compute the difference, monthly minus annual, rounded to 2 decimals.\n",
    "# Print exactly:\n",
    "# Annual: 0.1\n",
    "# Monthly: 0.25\n",
    "# Difference: 0.15\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "customers = pd.DataFrame({\n",
    "    \"plan\": [\"annual\"] * 10 + [\"monthly\"] * 8,\n",
    "    \"churned\": [0, 0, 1, 0, 0, 0, 0, 0, 0, 0,\n",
    "                1, 0, 0, 0, 1, 0, 0, 0],\n",
    "})\n",
    "\n",
    "### Your code starts here.\n",
    "\n",
    "\n",
    "\n",
    "### Your code ends here.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "tags": [
     "hide-input",
     "homework-answer"
    ]
   },
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": "Annual: 0.1\nMonthly: 0.25\nDifference: 0.15\n"
    }
   ],
   "source": [
    "### Solution\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "customers = pd.DataFrame({\n",
    "    \"plan\": [\"annual\"] * 10 + [\"monthly\"] * 8,\n",
    "    \"churned\": [0, 0, 1, 0, 0, 0, 0, 0, 0, 0,\n",
    "                1, 0, 0, 0, 1, 0, 0, 0],\n",
    "})\n",
    "\n",
    "rates = customers.groupby('plan')['churned'].mean()\n",
    "\n",
    "print(\"Annual:\", rates['annual'])\n",
    "print(\"Monthly:\", rates['monthly'])\n",
    "print(\"Difference:\", round(rates['monthly'] - rates['annual'], 2))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "tags": [
     "thebe-interactive"
    ]
   },
   "outputs": [],
   "source": [
    "### 8. A Two-Sided Permutation Test\n",
    "# For the basket sizes, the alternative says the segments differ in either direction.\n",
    "# Generate the null distribution rather than using a supplied list.\n",
    "#   1. Repeat repetitions times: permute values, split into two groups of six,\n",
    "#      and compute the absolute difference between their means. Store each distance.\n",
    "#   2. Convert the list to a pandas Series. Compute the proportion >= observed.\n",
    "#   3. Apply the preselected 5% cutoff to print the decision.\n",
    "# Twenty-five repetitions make the calculation easy to inspect, but give a coarse\n",
    "# estimate. Use many more for a real analysis, especially near a cutoff.\n",
    "# Existing customer segments were not randomly assigned, so this comparison\n",
    "# assesses association under an exchangeable-label model, not a causal segment effect.\n",
    "# Print exactly:\n",
    "# P-value: 0.16\n",
    "# Decision: do not reject the null\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "rng = np.random.default_rng(4)\n",
    "values = np.array([31, 28, 35, 30, 27, 33, 33, 31, 36, 29, 38, 32])\n",
    "group_size = 6\n",
    "observed = 2.5\n",
    "repetitions = 25\n",
    "### Your code starts here.\n",
    "\n",
    "\n",
    "\n",
    "### Your code ends here.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "tags": [
     "hide-input",
     "homework-answer"
    ]
   },
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": "P-value: 0.16\nDecision: do not reject the null\n"
    }
   ],
   "source": [
    "### Solution\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "rng = np.random.default_rng(4)\n",
    "values = np.array([31, 28, 35, 30, 27, 33, 33, 31, 36, 29, 38, 32])\n",
    "group_size = 6\n",
    "observed = 2.5\n",
    "repetitions = 25\n",
    "\n",
    "simulated = []\n",
    "for _ in range(repetitions):\n",
    "    shuffled = rng.permutation(values)\n",
    "    difference = (shuffled[:group_size].sum() - shuffled[group_size:].sum()) / group_size\n",
    "    simulated.append(abs(difference))\n",
    "simulated = pd.Series(simulated)\n",
    "p_value = (simulated >= observed).mean()\n",
    "if p_value < 0.05:\n",
    "    decision = 'reject the null'\n",
    "else:\n",
    "    decision = 'do not reject the null'\n",
    "print('P-value:', p_value)\n",
    "print('Decision:', decision)\n",
    "# Four of 25 distances are at least 2.5, including one exactly equal to 2.5.\n",
    "# A failure to reject is not proof that the population distributions match.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "tags": [
     "thebe-interactive"
    ]
   },
   "outputs": [],
   "source": [
    "### 9. Relative Lift\n",
    "# As in the chapter's checkout example, distinguish percentage points from\n",
    "# relative lift. Here, 0.10 - 0.08 is 2 percentage points, while relative lift\n",
    "# divides that difference by the 0.08 baseline to give 25%.\n",
    "# Neither number by itself establishes statistical significance or profitability.\n",
    "#   1. Use groupby to compute the conversion rate for each version.\n",
    "#   2. Compute the relative lift in percent: (rate B - rate A) / rate A * 100,\n",
    "#      rounded to 1 decimal.\n",
    "# Print exactly:\n",
    "# Rate A: 0.08\n",
    "# Rate B: 0.1\n",
    "# Relative lift: 25.0\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "summary = pd.DataFrame({\n",
    "    \"version\": [\"A\", \"B\"],\n",
    "    \"visitors\": [500, 500],\n",
    "    \"buyers\": [40, 50],\n",
    "})\n",
    "\n",
    "### Your code starts here.\n",
    "\n",
    "\n",
    "\n",
    "### Your code ends here.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "tags": [
     "hide-input",
     "homework-answer"
    ]
   },
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": "Rate A: 0.08\nRate B: 0.1\nRelative lift: 25.0\n"
    }
   ],
   "source": [
    "### Solution\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "summary = pd.DataFrame({\n",
    "    \"version\": [\"A\", \"B\"],\n",
    "    \"visitors\": [500, 500],\n",
    "    \"buyers\": [40, 50],\n",
    "})\n",
    "\n",
    "summary['rate'] = summary['buyers'] / summary['visitors']\n",
    "rates = summary.groupby('version')['rate'].mean()\n",
    "\n",
    "print(\"Rate A:\", rates['A'])\n",
    "print(\"Rate B:\", rates['B'])\n",
    "print(\"Relative lift:\", round((rates['B'] - rates['A']) / rates['A'] * 100, 1))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "tags": [
     "thebe-interactive"
    ]
   },
   "outputs": [],
   "source": [
    "### 10. A Randomized Controlled Trial\n",
    "# A company randomly assigned 60 employees with back pain to an ergonomic-chair\n",
    "# program (treatment) or to no program (control), then recorded who reported relief.\n",
    "#   1. Compute the proportion with relief in each group.\n",
    "#   2. Compute the difference, treatment minus control, rounded to 2 decimals.\n",
    "# Print exactly:\n",
    "# Treatment relief: 0.4\n",
    "# Control relief: 0.1\n",
    "# Difference: 0.3\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "trial = pd.DataFrame({\n",
    "    \"group\": [\"treatment\", \"control\"],\n",
    "    \"relief\": [12, 3],\n",
    "    \"no_relief\": [18, 27],\n",
    "})\n",
    "\n",
    "### Your code starts here.\n",
    "\n",
    "\n",
    "\n",
    "### Your code ends here.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "tags": [
     "hide-input",
     "homework-answer"
    ]
   },
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": "Treatment relief: 0.4\nControl relief: 0.1\nDifference: 0.3\n"
    }
   ],
   "source": [
    "### Solution\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "trial = pd.DataFrame({\n",
    "    \"group\": [\"treatment\", \"control\"],\n",
    "    \"relief\": [12, 3],\n",
    "    \"no_relief\": [18, 27],\n",
    "})\n",
    "\n",
    "trial['proportion'] = trial['relief'] / (trial['relief'] + trial['no_relief'])\n",
    "proportions = trial.groupby('group')['proportion'].mean()\n",
    "\n",
    "print(\"Treatment relief:\", proportions['treatment'])\n",
    "print(\"Control relief:\", proportions['control'])\n",
    "print(\"Difference:\", round(proportions['treatment'] - proportions['control'], 2))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<style>\n",
    ".ch10-hw-grader { max-width: 900px; margin-top: 1rem; }\n",
    ".ch10-hw-feedback-list { display: grid; gap: 0.45rem; margin: 0 0 1rem; }\n",
    ".ch10-hw-feedback-list[hidden] { display: none; }\n",
    ".ch10-hw-feedback-row { border: 1px solid #d8dee4; border-radius: 6px; padding: 0.55rem 0.65rem; }\n",
    ".ch10-hw-feedback-row strong { margin-right: 0.35rem; }\n",
    ".ch10-hw-feedback-row.correct { border-color: #aceebb; background: #dafbe1; color: #116329; }\n",
    ".ch10-hw-feedback-row.incorrect { border-color: #ffcecb; background: #ffebe9; color: #cf222e; }\n",
    ".ch10-hw-actions { display: flex; gap: 0.75rem; align-items: center; margin: 1rem 0 0; flex-wrap: wrap; }\n",
    ".ch10-hw-actions button { border: 1px solid #176de8; border-radius: 6px; background: #176de8; color: white; padding: 0.55rem 0.9rem; cursor: pointer; font-weight: 600; }\n",
    ".ch10-hw-actions button.secondary { background: white; color: #176de8; }\n",
    ".ch10-hw-actions.is-locked { opacity: 0.55; }\n",
    ".ch10-hw-actions button:disabled { cursor: not-allowed; opacity: 0.65; }\n",
    ".ch10-hw-result { font-weight: 700; }\n",
    "</style>\n",
    "\n",
    "<div class='ch10-hw-grader' id='ch10-hw-grader'>\n",
    "  <div class='ch10-hw-feedback-list' aria-live='polite' hidden>\n",
    "    <div class='ch10-hw-feedback-row' data-question='q1'><strong>Question 1</strong><span></span></div>\n",
    "    <div class='ch10-hw-feedback-row' data-question='q2'><strong>Question 2</strong><span></span></div>\n",
    "    <div class='ch10-hw-feedback-row' data-question='q3'><strong>Question 3</strong><span></span></div>\n",
    "    <div class='ch10-hw-feedback-row' data-question='q4'><strong>Question 4</strong><span></span></div>\n",
    "    <div class='ch10-hw-feedback-row' data-question='q5'><strong>Question 5</strong><span></span></div>\n",
    "    <div class='ch10-hw-feedback-row' data-question='q6'><strong>Question 6</strong><span></span></div>\n",
    "    <div class='ch10-hw-feedback-row' data-question='q7'><strong>Question 7</strong><span></span></div>\n",
    "    <div class='ch10-hw-feedback-row' data-question='q8'><strong>Question 8</strong><span></span></div>\n",
    "    <div class='ch10-hw-feedback-row' data-question='q9'><strong>Question 9</strong><span></span></div>\n",
    "    <div class='ch10-hw-feedback-row' data-question='q10'><strong>Question 10</strong><span></span></div>\n",
    "  </div>\n",
    "  <div class='ch10-hw-actions'>\n",
    "    <button type='button' id='ch10-hw-submit' disabled>Submit</button>\n",
    "    <button type='button' class='secondary' id='ch10-hw-reset'>Clear Results</button>\n",
    "    <span class='ch10-hw-result' id='ch10-hw-result' aria-live='polite'></span>\n",
    "  </div>\n",
    "</div>\n",
    "\n",
    "<script>\n",
    "(function () {\n",
    "  var homeworkId = 'ch10-homework';\n",
    "\n",
    "  function getParams() {\n",
    "    var params = new URLSearchParams(window.location.search);\n",
    "    return {\n",
    "      canvas_course_id: params.get('canvas_course_id') || params.get('course_id') || '',\n",
    "      canvas_assignment_id: params.get('canvas_assignment_id') || params.get('assignment_id') || '',\n",
    "      canvas_user_id: params.get('canvas_user_id') || params.get('user_id') || ''\n",
    "    };\n",
    "  }\n",
    "\n",
    "  function loadSessionIdentity(identity, actions, submit, result) {\n",
    "    submit.disabled = true;\n",
    "    actions.classList.add('is-locked');\n",
    "    result.textContent = 'Sign in from the account menu to submit this assignment.';\n",
    "\n",
    "    return fetch('/api/v1/session.php', { credentials: 'same-origin' })\n",
    "      .then(function (response) { return response.ok ? response.json() : null; })\n",
    "      .then(function (payload) {\n",
    "        if (!payload || !payload.authenticated || !payload.identity) return identity;\n",
    "        identity.canvas_user_id = payload.identity.canvas_user_id || identity.canvas_user_id || '';\n",
    "        identity.student_identifier = payload.identity.student_identifier || '';\n",
    "        submit.disabled = false;\n",
    "        actions.classList.remove('is-locked');\n",
    "        result.textContent = payload.identity.display_name ? 'Signed in as ' + payload.identity.display_name + '.' : 'Signed in.';\n",
    "        return identity;\n",
    "      })\n",
    "      .catch(function () { return identity; });\n",
    "  }\n",
    "\n",
    "  function codeFromCell(cell) {\n",
    "    var cm = cell.querySelector('.CodeMirror');\n",
    "    if (cm && cm.CodeMirror && typeof cm.CodeMirror.getValue === 'function') return cm.CodeMirror.getValue();\n",
    "    var cmContent = cell.querySelector('.cm-content');\n",
    "    if (cmContent && cmContent.cmView && cmContent.cmView.view) return cmContent.cmView.view.state.doc.toString();\n",
    "    var textarea = cell.querySelector('textarea');\n",
    "    if (textarea) return textarea.value;\n",
    "    var pre = cell.querySelector('.cell_input pre');\n",
    "    return pre ? pre.textContent : '';\n",
    "  }\n",
    "\n",
    "  function collectCodeCells() {\n",
    "    var cells = Array.prototype.slice.call(document.querySelectorAll('div.cell.tag_thebe-interactive')).slice(0, 5);\n",
    "    var code = {};\n",
    "    cells.forEach(function (cell, index) {\n",
    "      code['q' + (index + 6)] = codeFromCell(cell);\n",
    "    });\n",
    "    return code;\n",
    "  }\n",
    "\n",
    "  function collectAnswers() {\n",
    "    var answers = {};\n",
    "    ['q1', 'q2', 'q3', 'q4', 'q5'].forEach(function (question) {\n",
    "      var checked = document.querySelector('input[name=\"ch10-hw-' + question + '\"]:checked');\n",
    "      answers[question] = checked ? checked.value : '';\n",
    "    });\n",
    "    return answers;\n",
    "  }\n",
    "\n",
    "  function applyFeedback(grader, feedback) {\n",
    "    var list = grader.querySelector('.ch10-hw-feedback-list');\n",
    "    list.hidden = false;\n",
    "    ['q1', 'q2', 'q3', 'q4', 'q5', 'q6', 'q7', 'q8', 'q9', 'q10'].forEach(function (question) {\n",
    "      var row = grader.querySelector('[data-question=' + question + ']');\n",
    "      var message = row.querySelector('span');\n",
    "      var item = feedback[question];\n",
    "      row.className = 'ch10-hw-feedback-row';\n",
    "      if (!item) return;\n",
    "      message.textContent = item.message + ' (' + item.score + ' / ' + item.max_score + ')';\n",
    "      row.classList.add(item.correct ? 'correct' : 'incorrect');\n",
    "    });\n",
    "  }\n",
    "\n",
    "  function clearFeedback(grader) {\n",
    "    var list = grader.querySelector('.ch10-hw-feedback-list');\n",
    "    list.hidden = true;\n",
    "    grader.querySelectorAll('.ch10-hw-feedback-row').forEach(function (row) {\n",
    "      row.className = 'ch10-hw-feedback-row';\n",
    "      var message = row.querySelector('span');\n",
    "      if (message) message.textContent = '';\n",
    "    });\n",
    "  }\n",
    "\n",
    "  function saveAttempt(payload) {\n",
    "    return fetch('/api/v1/homework-attempts.php', {\n",
    "      method: 'POST',\n",
    "      headers: { 'Content-Type': 'application/json' },\n",
    "      credentials: 'same-origin',\n",
    "      body: JSON.stringify(payload)\n",
    "    }).then(function (response) {\n",
    "      return response.text().then(function (text) {\n",
    "        var payload = text ? JSON.parse(text) : null;\n",
    "        if (!payload || !response.ok || !payload.ok) {\n",
    "          throw new Error((payload && (payload.message || payload.error)) || 'Submission API is unavailable.');\n",
    "        }\n",
    "        return payload;\n",
    "      });\n",
    "    });\n",
    "  }\n",
    "\n",
    "  function initHomeworkGrader() {\n",
    "    var grader = document.getElementById('ch10-hw-grader');\n",
    "    if (!grader || grader.dataset.ready === 'true') return;\n",
    "    grader.dataset.ready = 'true';\n",
    "\n",
    "    var identity = getParams();\n",
    "    var actions = grader.querySelector('.ch10-hw-actions');\n",
    "    var result = document.getElementById('ch10-hw-result');\n",
    "    var submit = document.getElementById('ch10-hw-submit');\n",
    "    var reset = document.getElementById('ch10-hw-reset');\n",
    "\n",
    "    loadSessionIdentity(identity, actions, submit, result);\n",
    "\n",
    "    submit.addEventListener('click', function () {\n",
    "      clearFeedback(grader);\n",
    "      submit.disabled = true;\n",
    "      result.textContent = 'Submitting homework.';\n",
    "\n",
    "      saveAttempt({\n",
    "        homework_id: homeworkId,\n",
    "        answers: collectAnswers(),\n",
    "        code: collectCodeCells(),\n",
    "        canvas_course_id: identity.canvas_course_id,\n",
    "        canvas_assignment_id: identity.canvas_assignment_id,\n",
    "        canvas_user_id: identity.canvas_user_id\n",
    "      })\n",
    "        .then(function (payload) {\n",
    "          var best = (payload.best_score === null || typeof payload.best_score === 'undefined') ? payload.score : payload.best_score;\n",
    "          var attempt = (payload.attempt_count === null || typeof payload.attempt_count === 'undefined') ? '' : ' Attempt ' + payload.attempt_count + '.';\n",
    "          applyFeedback(grader, payload.feedback || {});\n",
    "          result.textContent = 'Submission saved.' + attempt + ' Score: ' + payload.score + ' / ' + payload.max_score + '. Best score: ' + best + ' / ' + payload.max_score + '. Canvas sync: ' + payload.canvas_sync_status + '.';\n",
    "        })\n",
    "        .catch(function (error) {\n",
    "          result.textContent = 'Could not save this attempt: ' + error.message;\n",
    "        })\n",
    "        .finally(function () {\n",
    "          submit.disabled = false;\n",
    "        });\n",
    "    });\n",
    "\n",
    "    reset.addEventListener('click', function () {\n",
    "      clearFeedback(grader);\n",
    "      result.textContent = '';\n",
    "    });\n",
    "  }\n",
    "\n",
    "  if (document.readyState === 'loading') {\n",
    "    document.addEventListener('DOMContentLoaded', initHomeworkGrader);\n",
    "  } else {\n",
    "    initHomeworkGrader();\n",
    "  }\n",
    "})();\n",
    "</script>\n"
   ]
  }
 ],
 "metadata": {
  "jupytext": {
   "cell_metadata_filter": "-all",
   "main_language": "python",
   "notebook_metadata_filter": "-all"
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.13.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}