Homework

10.4.3. Homework#

  • Use this assignment to practice the chapter’s concepts and coding skills in runnable Python code.

  • Sign in from the account menu before you start working on the assignment.

  • Write and run your code directly in the coding cells.

  • Multiple submissions are allowed. The highest score is kept.

1. Opt-In Comparisons

A store shows a new product page only to customers who click “Try the new design.” Those customers buy more often than everyone else. This proves that the new page causes more purchases.

Hide code cell source

 
### Solution
# False
# Customers chose whether to see the new page, so they may differ from others in ways that also affect buying. Without random assignment, the difference shows association, not causation.

2. What Shuffling Simulates

In a permutation test, shuffling the group labels simulates data under the null hypothesis that the label has nothing to do with the outcome.

Hide code cell source

 
### Solution
# True
# If the label doesn't matter, any assignment of labels to the observed values is equally likely, so shuffled labels show how big a difference chance alone tends to produce.

3. Choosing a One-Sided Statistic

The alternative hypothesis says that version B’s order values tend to be larger than version A’s. “Mean of B minus mean of A” is a sensible test statistic, with large values favoring the alternative.

Hide code cell source

 
### Solution
# True
# Under the alternative, B's mean tends to be larger, so the difference tends to be large and positive.

4. A Randomized Trial

Employees were randomly assigned to a new training program or to the usual one. Follow-up and sales measurement were comparable, and participants did not affect one another’s outcomes. The new-program group had higher average sales, and a permutation test gives a p-value of 0.002. This is evidence that the new program caused an improvement for these participants.

Hide code cell source

 
### Solution
# True
# Under the stated design conditions, random assignment supports a causal
# comparison. The observed direction is positive, and the small p-value is
# evidence against the no-effect model. Randomization does not make the
# groups identical or establish that the same effect holds in every setting.

5. Small Randomized Studies

A randomized experiment with only 30 customers in each group cannot support a causal conclusion.

Hide code cell source

 
### Solution
# False
# Random assignment, not sample size, is what supports a causal claim. A small sample makes the estimate less precise and makes a real effect harder to detect.

### 6. Basket Size by Segment
# Compare the average basket size (in dollars) of new and returning customers.
#   1. Use groupby to compute the mean basket for each segment. Keep full
#      precision for calculations and round to 2 decimals only when printing.
#   2. Compute the distance between the two means, | returning - new |, rounded to 2 decimals.
# Print exactly:
# New: 30.67
# Returning: 33.17
# Distance: 2.5
import numpy as np
import pandas as pd

baskets = pd.DataFrame({
    "segment": ["new"] * 6 + ["returning"] * 6,
    "basket": [31, 28, 35, 30, 27, 33, 33, 31, 36, 29, 38, 32],
})

### Your code starts here.



### Your code ends here.

Hide code cell source

 
### Solution
import numpy as np
import pandas as pd

baskets = pd.DataFrame({
    "segment": ["new"] * 6 + ["returning"] * 6,
    "basket": [31, 28, 35, 30, 27, 33, 33, 31, 36, 29, 38, 32],
})

means = baskets.groupby('segment')['basket'].mean()

print("New:", round(means['new'], 2))
print("Returning:", round(means['returning'], 2))
print("Distance:", round(abs(means['returning'] - means['new']), 2))

New: 30.67
Returning: 33.17
Distance: 2.5

### 7. Churn by Plan
# Each row is a customer, with their plan and whether they cancelled (1) or stayed (0).
#   1. Use groupby to compute the churn rate for each plan.
#   2. Compute the difference, monthly minus annual, rounded to 2 decimals.
# Print exactly:
# Annual: 0.1
# Monthly: 0.25
# Difference: 0.15
import numpy as np
import pandas as pd

customers = pd.DataFrame({
    "plan": ["annual"] * 10 + ["monthly"] * 8,
    "churned": [0, 0, 1, 0, 0, 0, 0, 0, 0, 0,
                1, 0, 0, 0, 1, 0, 0, 0],
})

### Your code starts here.



### Your code ends here.

Hide code cell source

 
### Solution
import numpy as np
import pandas as pd

customers = pd.DataFrame({
    "plan": ["annual"] * 10 + ["monthly"] * 8,
    "churned": [0, 0, 1, 0, 0, 0, 0, 0, 0, 0,
                1, 0, 0, 0, 1, 0, 0, 0],
})

rates = customers.groupby('plan')['churned'].mean()

print("Annual:", rates['annual'])
print("Monthly:", rates['monthly'])
print("Difference:", round(rates['monthly'] - rates['annual'], 2))

Annual: 0.1
Monthly: 0.25
Difference: 0.15

### 8. A Two-Sided Permutation Test
# For the basket sizes, the alternative says the segments differ in either direction.
# Generate the null distribution rather than using a supplied list.
#   1. Repeat repetitions times: permute values, split into two groups of six,
#      and compute the absolute difference between their means. Store each distance.
#   2. Convert the list to a pandas Series. Compute the proportion >= observed.
#   3. Apply the preselected 5% cutoff to print the decision.
# Twenty-five repetitions make the calculation easy to inspect, but give a coarse
# estimate. Use many more for a real analysis, especially near a cutoff.
# Existing customer segments were not randomly assigned, so this comparison
# assesses association under an exchangeable-label model, not a causal segment effect.
# Print exactly:
# P-value: 0.16
# Decision: do not reject the null
import numpy as np
import pandas as pd

rng = np.random.default_rng(4)
values = np.array([31, 28, 35, 30, 27, 33, 33, 31, 36, 29, 38, 32])
group_size = 6
observed = 2.5
repetitions = 25
### Your code starts here.



### Your code ends here.

Hide code cell source

 
### Solution
import numpy as np
import pandas as pd

rng = np.random.default_rng(4)
values = np.array([31, 28, 35, 30, 27, 33, 33, 31, 36, 29, 38, 32])
group_size = 6
observed = 2.5
repetitions = 25

simulated = []
for _ in range(repetitions):
    shuffled = rng.permutation(values)
    difference = (shuffled[:group_size].sum() - shuffled[group_size:].sum()) / group_size
    simulated.append(abs(difference))
simulated = pd.Series(simulated)
p_value = (simulated >= observed).mean()
if p_value < 0.05:
    decision = 'reject the null'
else:
    decision = 'do not reject the null'
print('P-value:', p_value)
print('Decision:', decision)
# Four of 25 distances are at least 2.5, including one exactly equal to 2.5.
# A failure to reject is not proof that the population distributions match.

P-value: 0.16
Decision: do not reject the null

### 9. Relative Lift
# As in the chapter's checkout example, distinguish percentage points from
# relative lift. Here, 0.10 - 0.08 is 2 percentage points, while relative lift
# divides that difference by the 0.08 baseline to give 25%.
# Neither number by itself establishes statistical significance or profitability.
#   1. Use groupby to compute the conversion rate for each version.
#   2. Compute the relative lift in percent: (rate B - rate A) / rate A * 100,
#      rounded to 1 decimal.
# Print exactly:
# Rate A: 0.08
# Rate B: 0.1
# Relative lift: 25.0
import numpy as np
import pandas as pd

summary = pd.DataFrame({
    "version": ["A", "B"],
    "visitors": [500, 500],
    "buyers": [40, 50],
})

### Your code starts here.



### Your code ends here.

Hide code cell source

 
### Solution
import numpy as np
import pandas as pd

summary = pd.DataFrame({
    "version": ["A", "B"],
    "visitors": [500, 500],
    "buyers": [40, 50],
})

summary['rate'] = summary['buyers'] / summary['visitors']
rates = summary.groupby('version')['rate'].mean()

print("Rate A:", rates['A'])
print("Rate B:", rates['B'])
print("Relative lift:", round((rates['B'] - rates['A']) / rates['A'] * 100, 1))

Rate A: 0.08
Rate B: 0.1
Relative lift: 25.0

### 10. A Randomized Controlled Trial
# A company randomly assigned 60 employees with back pain to an ergonomic-chair
# program (treatment) or to no program (control), then recorded who reported relief.
#   1. Compute the proportion with relief in each group.
#   2. Compute the difference, treatment minus control, rounded to 2 decimals.
# Print exactly:
# Treatment relief: 0.4
# Control relief: 0.1
# Difference: 0.3
import numpy as np
import pandas as pd

trial = pd.DataFrame({
    "group": ["treatment", "control"],
    "relief": [12, 3],
    "no_relief": [18, 27],
})

### Your code starts here.



### Your code ends here.

Hide code cell source

 
### Solution
import numpy as np
import pandas as pd

trial = pd.DataFrame({
    "group": ["treatment", "control"],
    "relief": [12, 3],
    "no_relief": [18, 27],
})

trial['proportion'] = trial['relief'] / (trial['relief'] + trial['no_relief'])
proportions = trial.groupby('group')['proportion'].mean()

print("Treatment relief:", proportions['treatment'])
print("Control relief:", proportions['control'])
print("Difference:", round(proportions['treatment'] - proportions['control'], 2))

Treatment relief: 0.4
Control relief: 0.1
Difference: 0.3