Clustering

16. Clustering#

This chapter introduces clustering as unsupervised learning. You will use k-means to group observations, interpret cluster centers, and evaluate whether the resulting groups make practical sense.

  • Clustering: Grouping observations by similarity when no labels are available, as in customer segmentation.

  • K-Means: An algorithm that repeatedly assigns points to the nearest cluster center and then moves each center to the mean of its points.

  • Feature Scaling: Because clusters are formed by distance, features on different scales must be standardized first or the largest-scale feature dominates.

  • Choosing k: The elbow method and the silhouette score help compare numbers of clusters, but they do not replace business judgment.

  • Interpreting Clusters: Cluster numbers have no meaning on their own; describe each group in business terms and decide what action it suggests.

  • Principal Component Analysis (optional): Summarizing many correlated features with a few new axes, so customers described by many columns can be plotted and their clusters seen in two dimensions.

Chapter Video

Video credit: StatQuest with Josh Starmer. (2018, May 23). StatQuest: K-means clustering. YouTube. Watch on YouTube (8:30).

This embedded video is provided for educational reference only. All rights are retained by the respective copyright holder.

Learning Objectives

By the end of this chapter, you will be able to:

  1. Explain clustering as grouping observations without known labels.

  2. Describe the k-means algorithm in terms of centers and assignments.

  3. Prepare numeric features so distance-based clusters are meaningful.

  4. Interpret cluster centers and cluster membership in an applied context.

  5. Evaluate clusters with metrics and domain judgment.

  6. Optional: Use principal component analysis to summarize many features, read its loadings, and plot clusters in two dimensions.

Chapter Flow

Glossary

#

Term

Explanation

1

Clustering

Grouping observations based on similarity.

2

Unsupervised learning

Learning patterns from data without known target labels.

3

Cluster

A group of observations that are similar to each other.

4

K-means

A clustering algorithm that partitions data into k groups.

5

Centroid

The center point of a k-means cluster.

6

Assignment step

The k-means step that assigns observations to nearest centroids.

7

Update step

The k-means step that recalculates centroids.

8

Distance metric

A rule for measuring how far observations are apart.

9

Euclidean distance

Straight-line distance between points.

10

Feature scaling

Transforming variables so their scales are comparable.

11

Standardization

Scaling values by subtracting a mean and dividing by a standard deviation.

12

Within-cluster variation

How spread out points are inside a cluster.

13

Silhouette score

A metric comparing within-cluster fit with separation from other clusters.

14

Elbow method

A heuristic for choosing k by inspecting improvement as k increases.

15

Initialization

The starting placement of cluster centers.

16

Convergence

The point when assignments and centers stop changing much.

17

Dimensionality

The number of features used to represent observations.

18

Segmentation

Using clusters to divide customers, products, or cases into groups.

19

Interpretability

How easily cluster results can be explained.

20

Domain judgment

Subject-matter reasoning used to decide whether clusters are useful.

21

Noisy feature

A variable that adds distraction rather than useful grouping information.

22

Dimensionality reduction

Replacing many features with a few new ones that keep most of the information.

23

Principal component analysis (PCA)

A method that finds new axes, in order of how much of the data’s spread they capture.

24

Principal component

One of the new axes found by PCA: a weighted combination of the original features.

25

Explained variance ratio

The share of the total variation in the data that one principal component captures.

26

Scree plot

A chart of the explained variance ratio of each component, used to choose how many to keep.

27

Loading

The weight of an original feature in a principal component.

28

Inertia

scikit-learn’s name for the within-cluster sum of squares: the total squared distance from each point to its own cluster center; lower means tighter clusters.

29

Log scale

Measuring values by their base-10 logarithm, so each tenfold increase is one step; used before standardizing spending that differs by factors of 10.

Lecture Slides