Homework

15.4.3. Homework#

  • Use this assignment to practice the chapter’s concepts and coding skills in runnable Python code.

  • Sign in from the account menu before you start working on the assignment.

  • Write and run your code directly in the coding cells.

  • Multiple submissions are allowed. The highest score is kept.

1. Choosing k Honestly

An analyst tries k = 1 to 51, picks the k with the highest accuracy on the test set, and reports that same test accuracy to the manager as the expected accuracy on new customers.

Hide code cell source

### Solution
# False
# Choosing k by test accuracy uses the test set to build the model, so the reported number is optimistic. Choose k on a validation set and keep the test set for the final report.

2. Accuracy and Cost

Classifier A is more accurate than classifier B on the same test set, so A must also have the lower total error cost.

Hide code cell source

### Solution
# False
# Accuracy counts every error the same. When a missed case costs much more than a false alarm, a less accurate classifier that misses fewer costly cases can be cheaper.

3. Lowering the Threshold

A missed fraud case costs a bank far more than reviewing a legitimate payment. Lowering the logistic regression threshold from 0.5 to 0.2 can reduce the bank’s total error cost.

Hide code cell source

### Solution
# True
# A lower threshold flags more payments, catching more fraud (higher recall) at the price of more false alarms. When misses cost much more, that trade can lower the total cost.

4. Precision or Recall

A retailer asks: of the orders the model flagged as fraud, what share really were fraud? That question asks for the model’s recall.

Hide code cell source

### Solution
# False
# That is precision: TP / (TP + FP), the share of flagged cases that are real. Recall asks what share of the real fraud cases were flagged.

5. Reading a Coefficient

In a logistic regression that predicts purchase, page value has a positive coefficient. Holding the other features fixed, sessions with higher page value get higher predicted purchase probabilities.

Hide code cell source

### Solution
# True
# A positive coefficient raises the weighted sum, and the logistic curve turns a larger sum into a larger probability.
### 6. From Score to Probability
#   A model's score for a session is z = -4 + 0.25 * page_value.
#   1. Turn the score into a probability with the logistic function 1 / (1 + e^(-z))
#      for page values 8, 16, and 24, each rounded to 3 decimals.
# Print exactly:
# Page value 8 probability: 0.119
# Page value 16 probability: 0.5
# Page value 24 probability: 0.881
import numpy as np
import pandas as pd

page_values = [8, 16, 24]

### Your code starts here.



### Your code ends here.

Hide code cell source

### Solution
import numpy as np
import pandas as pd

page_values = [8, 16, 24]

for page_value in page_values:
    z = -4 + 0.25 * page_value
    probability = 1 / (1 + np.exp(-z))
    print("Page value", page_value, "probability:", round(probability, 3))
Page value 8 probability: 0.119
Page value 16 probability: 0.5
Page value 24 probability: 0.881
### 7. Fitting LogisticRegression
#   1. Fit a LogisticRegression on all 16 sessions (X and y).
#   2. Print the predicted purchase probability (rounded to 2 decimals) and the
#      predicted label for each new session.
# Print exactly:
# Probability: 0.26 Prediction: 0
# Probability: 0.74 Prediction: 1
import numpy as np
import pandas as pd
from sklearn.linear_model import LogisticRegression

# Shopping sessions (made-up numbers for practice): page value in dollars,
# seconds spent on product pages, and whether the session ended in a purchase.
sessions = pd.DataFrame({
    "page_value": [0, 12, 0, 35, 0, 4, 28, 0, 0, 19, 0, 41, 0, 7, 0, 22],
    "duration": [120, 1500, 300, 2400, 60, 900, 1300, 200, 2600, 1700, 150, 3100, 700, 450, 30, 1400],
    "purchase": [0, 1, 0, 1, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 0, 1],
})
X = sessions[["page_value", "duration"]]
y = sessions["purchase"]
new_sessions = pd.DataFrame({"page_value": [9, 12], "duration": [900, 900]})

### Your code starts here.



### Your code ends here.

Hide code cell source

### Solution
import numpy as np
import pandas as pd
from sklearn.linear_model import LogisticRegression

# Shopping sessions (made-up numbers for practice): page value in dollars,
# seconds spent on product pages, and whether the session ended in a purchase.
sessions = pd.DataFrame({
    "page_value": [0, 12, 0, 35, 0, 4, 28, 0, 0, 19, 0, 41, 0, 7, 0, 22],
    "duration": [120, 1500, 300, 2400, 60, 900, 1300, 200, 2600, 1700, 150, 3100, 700, 450, 30, 1400],
    "purchase": [0, 1, 0, 1, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 0, 1],
})
X = sessions[["page_value", "duration"]]
y = sessions["purchase"]
new_sessions = pd.DataFrame({"page_value": [9, 12], "duration": [900, 900]})

model = LogisticRegression().fit(X, y)
probabilities = model.predict_proba(new_sessions)[:, 1]
labels = model.predict(new_sessions)
for probability, label in zip(probabilities, labels):
    print("Probability:", round(probability, 2), "Prediction:", label)
Probability: 0.26 Prediction: 0
Probability: 0.74 Prediction: 1
### 8. Moving the Threshold
#   1. For thresholds 0.5 and 0.3, predict purchase when prob >= threshold.
#   2. Print how many sessions are flagged and the recall, rounded to 2 decimals.
# Print exactly:
# Threshold 0.5 flagged: 4 recall: 0.6
# Threshold 0.3 flagged: 7 recall: 1.0
import numpy as np
import pandas as pd

# Predicted purchase probabilities for 10 test sessions and what actually happened
# (made-up data for practice; 1 = purchase).
prob = [0.92, 0.35, 0.61, 0.08, 0.44, 0.27, 0.75, 0.12, 0.55, 0.31]
actual = [1, 1, 1, 0, 0, 0, 1, 0, 0, 1]

### Your code starts here.



### Your code ends here.

Hide code cell source

### Solution
import numpy as np
import pandas as pd

# Predicted purchase probabilities for 10 test sessions and what actually happened
# (made-up data for practice; 1 = purchase).
prob = [0.92, 0.35, 0.61, 0.08, 0.44, 0.27, 0.75, 0.12, 0.55, 0.31]
actual = [1, 1, 1, 0, 0, 0, 1, 0, 0, 1]

for threshold in [0.5, 0.3]:
    flagged = [int(p >= threshold) for p in prob]
    caught = sum([a == 1 and f == 1 for a, f in zip(actual, flagged)])
    print("Threshold", threshold, "flagged:", sum(flagged), "recall:", round(caught / sum(actual), 2))
Threshold 0.5 flagged: 4 recall: 0.6
Threshold 0.3 flagged: 7 recall: 1.0
### 9. The Cheapest Threshold
#   A missed buyer costs $30 and a wasted offer to a non-buyer costs $3.
#   1. For thresholds 0.2, 0.4, and 0.6, compute the total error cost.
#   2. Print each cost and the cheapest threshold.
# Print exactly:
# Threshold 0.2 cost: $9
# Threshold 0.4 cost: $66
# Threshold 0.6 cost: $60
# Cheapest threshold: 0.2
import numpy as np
import pandas as pd

# Predicted purchase probabilities for 10 test sessions and what actually happened
# (made-up data for practice; 1 = purchase).
prob = [0.92, 0.35, 0.61, 0.08, 0.44, 0.27, 0.75, 0.12, 0.55, 0.31]
actual = [1, 1, 1, 0, 0, 0, 1, 0, 0, 1]
COST_FN = 30
COST_FP = 3

### Your code starts here.



### Your code ends here.

Hide code cell source

### Solution
import numpy as np
import pandas as pd

# Predicted purchase probabilities for 10 test sessions and what actually happened
# (made-up data for practice; 1 = purchase).
prob = [0.92, 0.35, 0.61, 0.08, 0.44, 0.27, 0.75, 0.12, 0.55, 0.31]
actual = [1, 1, 1, 0, 0, 0, 1, 0, 0, 1]
COST_FN = 30
COST_FP = 3

costs = {}
for threshold in [0.2, 0.4, 0.6]:
    flagged = [int(p >= threshold) for p in prob]
    fn = sum([a == 1 and f == 0 for a, f in zip(actual, flagged)])
    fp = sum([a == 0 and f == 1 for a, f in zip(actual, flagged)])
    costs[threshold] = COST_FN * fn + COST_FP * fp
    print("Threshold", threshold, "cost: $" + str(costs[threshold]))
print("Cheapest threshold:", min(costs, key=costs.get))
Threshold 0.2 cost: $9
Threshold 0.4 cost: $66
Threshold 0.6 cost: $60
Cheapest threshold: 0.2
### 10. The Cost Rule
#   The cost-minimizing threshold is C_FP / (C_FP + C_FN).
#   1. Compute it for each pair of costs below, rounded to 3 decimals.
# Print exactly:
# FN $30, FP $3: threshold 0.091
# FN $100, FP $25: threshold 0.2
import numpy as np
import pandas as pd

cost_pairs = [(30, 3), (100, 25)]   # (cost of a false negative, cost of a false positive)

### Your code starts here.



### Your code ends here.

Hide code cell source

### Solution
import numpy as np
import pandas as pd

cost_pairs = [(30, 3), (100, 25)]   # (cost of a false negative, cost of a false positive)

for cost_fn, cost_fp in cost_pairs:
    threshold = cost_fp / (cost_fp + cost_fn)
    print("FN $" + str(cost_fn) + ", FP $" + str(cost_fp) + ": threshold", round(threshold, 3))
FN $30, FP $3: threshold 0.091
FN $100, FP $25: threshold 0.2