Homework

14.6.3. Homework#

  • Use this assignment to practice the chapter’s concepts and coding skills in runnable Python code.

  • Sign in from the account menu before you start working on the assignment.

  • Write and run your code directly in the coding cells.

  • Multiple submissions are allowed. The highest score is kept.

1. Testing on Training Data

A 1-nearest-neighbor classifier is tested on the same customers it was trained on. Its accuracy there is an honest estimate of how well it will classify new customers.

Hide code cell source

 
### Solution
# False
# Each training customer is its own nearest neighbor, so a 1-NN classifier is always right on its training data. Only a separate test set gives an honest estimate.

2. Scale Matters

A classifier uses annual income (in dollars) and number of late payments. Before computing distances, both attributes should be converted to standard units.

Hide code cell source

 
### Solution
# True
# Otherwise income, measured in thousands of dollars, swamps late payments in the distance, and the classifier effectively uses one attribute.

3. Scaling With the Test Set

When converting attributes to standard units, it is fine to compute the mean and SD from the whole data set, including the test set.

Hide code cell source

 
### Solution
# False
# Compute the mean and SD from the training set only and apply them to the test set. Otherwise the test data influence the classifier, and the test is no longer a fair stand-in for new cases.

4. One Neighbor

With k = 1, a nearest-neighbor classifier predicts the class of the single training example closest to the new case.

Hide code cell source

 
### Solution
# True
# k is the number of neighbors that vote; with k = 1, the closest one decides.

5. High Accuracy

Only 2% of a store’s orders are fraudulent. A fraud classifier that is 98% accurate is therefore useful.

Hide code cell source

 
### Solution
# False
# Labeling every order "not fraud" is also 98% accurate and catches no fraud at all. Compare accuracy with that baseline, and check how many fraud cases are actually caught.

### 6. Distance Between Two Customers
# Each row of customers describes one customer in standard units.
#   1. Convert the rows for "Ana" and "Ben" to arrays with np.array(...loc[...]).
#   2. Compute the distance between them, rounded to 3 decimals.
# Print exactly:
# Distance: 1.414
import numpy as np
import pandas as pd

customers = pd.DataFrame({
    "visits_su": [0.5, -0.5, 1.2],
    "spend_su": [1.0, 0.0, -0.3],
}, index=["Ana", "Ben", "Cy"])

### Your code starts here.



### Your code ends here.

Hide code cell source

 
### Solution
import numpy as np
import pandas as pd

customers = pd.DataFrame({
    "visits_su": [0.5, -0.5, 1.2],
    "spend_su": [1.0, 0.0, -0.3],
}, index=["Ana", "Ben", "Cy"])

ana = np.array(customers.loc['Ana'])
ben = np.array(customers.loc['Ben'])
print("Distance:", round(np.sqrt(np.sum((ana - ben) ** 2)), 3))

Distance: 1.414

### 7. Will They Renew?
#   1. Find the 3 subscribers nearest to new_subscriber.
#   2. Print their labels (nearest first) and the majority prediction.
# Print exactly:
# Nearest labels: ['renew', 'cancel', 'renew']
# Prediction: renew
import numpy as np
import pandas as pd

subscribers = pd.DataFrame({
    "usage_su": [1.0, -0.8, 0.4, -1.2, 0.1, 0.9],
    "tenure_su": [0.6, -0.5, -0.2, 0.8, 0.3, -1.0],
    "label": ["renew", "cancel", "cancel", "cancel", "renew", "renew"],
})
new_subscriber = np.array([0.3, 0.1])

### Your code starts here.



### Your code ends here.

Hide code cell source

 
### Solution
import numpy as np
import pandas as pd

subscribers = pd.DataFrame({
    "usage_su": [1.0, -0.8, 0.4, -1.2, 0.1, 0.9],
    "tenure_su": [0.6, -0.5, -0.2, 0.8, 0.3, -1.0],
    "label": ["renew", "cancel", "cancel", "cancel", "renew", "renew"],
})
new_subscriber = np.array([0.3, 0.1])

attributes = subscribers[['usage_su', 'tenure_su']]
subscribers['distance'] = np.sqrt(((attributes - new_subscriber) ** 2).sum(axis=1))
nearest = subscribers.sort_values('distance').head(3)

print("Nearest labels:", nearest['label'].tolist())
print("Prediction:", nearest['label'].value_counts().idxmax())

Nearest labels: ['renew', 'cancel', 'renew']
Prediction: renew

### 8. Scaling the Test Set
#   1. Convert the test customers' monthly_spend to standard units using the
#      TRAINING mean and SD (np.std). Round each value to 2 decimals.
# Print exactly:
# Scaled test spend: [-1.06, 0.35, 1.41]
import numpy as np
import pandas as pd

train_spend = pd.Series([40, 60, 50, 70, 30])
test_spend = pd.Series([35, 55, 70])

### Your code starts here.



### Your code ends here.

Hide code cell source

 
### Solution
import numpy as np
import pandas as pd

train_spend = pd.Series([40, 60, 50, 70, 30])
test_spend = pd.Series([35, 55, 70])

mean = np.mean(train_spend)
sd = np.std(train_spend)
scaled = ((test_spend - mean) / sd).round(2)
print("Scaled test spend:", scaled.tolist())

Scaled test spend: [-1.06, 0.35, 1.41]

### 9. Accuracy and the Baseline
# A fraud classifier was checked on 20 test orders.
#   1. Compute the classifier's accuracy.
#   2. Compute the accuracy of always predicting "ok".
#   3. Count the fraud orders the classifier caught (actual and predicted are both "fraud").
# Print exactly:
# Accuracy: 0.9
# Always-ok accuracy: 0.9
# Fraud caught: 1 of 2
import numpy as np
import pandas as pd

orders = pd.DataFrame({
    "actual": ["ok"] * 18 + ["fraud"] * 2,
    "predicted": ["ok"] * 17 + ["fraud"] + ["ok", "fraud"],
})

### Your code starts here.



### Your code ends here.

Hide code cell source

 
### Solution
import numpy as np
import pandas as pd

orders = pd.DataFrame({
    "actual": ["ok"] * 18 + ["fraud"] * 2,
    "predicted": ["ok"] * 17 + ["fraud"] + ["ok", "fraud"],
})

fraud = orders['actual'] == 'fraud'
caught = (fraud & (orders['predicted'] == 'fraud')).sum()

print("Accuracy:", (orders['actual'] == orders['predicted']).mean())
print("Always-ok accuracy:", (orders['actual'] == 'ok').mean())
print("Fraud caught:", caught, "of", fraud.sum())

Accuracy: 0.9
Always-ok accuracy: 0.9
Fraud caught: 1 of 2

### 10. Choosing k
# The same test set was classified with k = 1, 3, and 5.
#   1. Compute the test accuracy for each k.
#   2. Print the k with the highest accuracy.
# Print exactly:
# k1: 0.625
# k3: 0.875
# k5: 0.625
# Best: k3
import numpy as np
import pandas as pd

results = pd.DataFrame({
    "actual": ["A", "B", "A", "A", "B", "B", "A", "B"],
    "k1": ["A", "A", "A", "B", "B", "A", "A", "B"],
    "k3": ["A", "B", "A", "A", "B", "A", "A", "B"],
    "k5": ["A", "B", "A", "A", "A", "A", "A", "A"],
})

### Your code starts here.



### Your code ends here.

Hide code cell source

 
### Solution
import numpy as np
import pandas as pd

results = pd.DataFrame({
    "actual": ["A", "B", "A", "A", "B", "B", "A", "B"],
    "k1": ["A", "A", "A", "B", "B", "A", "A", "B"],
    "k3": ["A", "B", "A", "A", "B", "A", "A", "B"],
    "k5": ["A", "B", "A", "A", "A", "A", "A", "A"],
})

accuracy = {}
for k in ['k1', 'k3', 'k5']:
    accuracy[k] = (results['actual'] == results[k]).mean()
    print(k + ":", accuracy[k])
print("Best:", max(accuracy, key=accuracy.get))

k1: 0.625
k3: 0.875
k5: 0.625
Best: k3