Clustering Techniques

What Is Clustering?

Clustering is an unsupervised learning technique used to group data points based on similarity. Unlike supervised learning, clustering does not use labelled data. Instead, it looks for natural patterns or groupings within the dataset.

For example, clustering might help a business identify customer segments or discover unexpected patterns in shopping habits.

Why Clustering?

Clustering groups similar items together when no labels exist. It helps you uncover the natural structure in data so you can describe it, summarise it, and act on it-without pre-defined categories.

What problem does it solve? Given lots of records (customers, images, events), how can we organise them into a small number of meaningful groups where members within a group are more alike than members across groups?

Find structure: Reveal hidden groupings (e.g. “regular small spenders” vs “rare big spenders”).
Summarise & explain: Turn thousands of points into a handful of easy-to-talk-about segments.
Target actions: Tailor emails, offers, or support by cluster; plan stock or resources by segment.
Spot oddities: Points far from any cluster can flag fraud, data errors, or rare cases.

Simple example: Using features like visit frequency, average basket size, and return rate, clustering can separate shoppers into groups such as loyal regulars, bargain hunters, and big occasional buyers-even though no one labelled them beforehand.

Note: Results depend on the features and their scale (e.g. £ vs counts). It’s common to standardise features and try a few cluster counts to find the most useful grouping.

Common Clustering Techniques

Technique	Description	Example Applications
K-Means Clustering	Groups data into `K` clusters based on distance from centroids (average positions).	Customer segmentation, product clustering.
Hierarchical Clustering	Builds a tree of clusters by merging or splitting them step by step.	Genetic analysis, document similarity.
DBSCAN	Groups densely packed points and treats points in sparse areas as noise.	Anomaly detection, geographic data clustering.

Common Clustering Techniques

K-Means Clustering

K-Means is a popular unsupervised learning algorithm that groups data into K clusters based on similarity. It works by finding cluster centres (called centroids) and assigning each data point to the nearest one.

How it works:

Step 1: Choose how many clusters you want - this is the value of K. (in the example below, K is 3)
Step 2: Randomly place K centroids in the data space.

K-Means Example, Step 2
I, Weston.pace, CC BY-SA 3.0, via Wikimedia Commons

The dataset contains a number of grey points, each representing an individual data entry. Since K = 3, three random points are chosen to act as the initial centroids - here marked in red, green, and blue. These centroids do not yet represent any meaningful group - they’re simply the starting guess.

Step 3: Assign each data point to its nearest centroid.

K-Means Example, Step 3
I, Weston.pace, CC BY-SA 3.0, via Wikimedia Commons

You now see clusters forming, marked by coloured backgrounds (red, green, and blue). The model now has three clusters - but the centroids are likely not in the optimal positions yet.

Step 4: Move the centroids to the average position of all the points in their group.
Step 5: Repeat steps 3 and 4 until the centroids stop moving significantly.

K-Means Example, Step 4
I, Weston.pace, CC BY-SA 3.0, via Wikimedia Commons

For each cluster, the algorithm calculates the mean position (centroid) of the assigned points. The red, green, and blue points shift to new locations - the average positions of their clusters. This is where K-Means gets its name: it finds the mean of each cluster and updates the centroid.

This process aims to reduce the total distance between each data point and its assigned centroid - so the clusters become as “tight” and distinct as possible.

Understanding Random Centroids in K-Means

Why Start with Random Centroids?

K-Means doesn’t start with any knowledge about what the clusters should look like. To begin, it places K centroids randomly in the feature space - like dropping pins on a map with no instructions. These act as temporary centres for the groups that will form.

This random starting point doesn’t decide the final clusters - it's just a guess that helps the algorithm begin the learning process.

So How Do Those Random Starts Help?

Once the centroids are placed, K-Means begins a process of refinement:

Step 1: Each data point is assigned to the nearest centroid.
Step 2: Each centroid is updated to the average position of the points assigned to it.
Step 3: The assignments and centroids are updated repeatedly until they stop changing.

This allows the centroids to “move” into positions that best represent the true structure of the data - like uncovering natural groups that were hidden in the dataset.

What if the Centroids Start in Bad Places, and the Randomness Leads to Bad Results?

Because the starting points are random, K-Means might not always find the best clustering on the first try. Some centroids could land in awkward spots that lead to poor groupings.

To solve this, most implementations of K-Means (including those in scikit-learn) run the algorithm several times with different random seeds and keep the result with the lowest total error. This error is usually based on how close each point is to its assigned centroid.

This technique is often controlled with the n_init parameter in code - usually set to 10 by default.

Example K-Means Scenarios

Customer Segmentation

A retail company uses K-Means to segment customers based on two features: average purchase value and number of store visits per month. By setting K = 3, the algorithm might group customers into:

Cluster A: Frequent, low-spending customers
Cluster B: Occasional, high-spending customers
Cluster C: Infrequent, low-spending customers

This helps tailor marketing strategies to different buyer types.

Classifying Plant Species

In a botany dataset with measurements like petal width and length, K-Means can be used to group flower samples into natural clusters. If K = 3, the algorithm may roughly group them into three species - even without the species labels - just by analysing the measurements.

Python Example: K-Means

Scenario: Grouping customers by purchasing behaviour.

                        # scikit-learn (imported as sklearn)

                        from sklearn.cluster import KMeans

                        import numpy as np

                        X = np.array([[200, 5], [250, 8], [180, 3], [300, 12], [400, 15]])

                        kmeans = KMeans(n_clusters=2, random_state=42)

                        kmeans.fit(X)

                        print("Cluster Labels:", kmeans.labels_)

Hierarchical Clustering

Hierarchical clustering builds a tree-like structure (called a dendrogram) that shows how individual data points group together at different levels of similarity. It starts by treating every point as its own cluster and gradually merges the closest pairs until everything is in one large group.

Unlike K-Means, this method doesn't require you to predefine the number of clusters - you can choose how many to keep by “cutting” the dendrogram at a specific height.

Step 1: Sample Dataset

We begin with 10 customers, each described by two features:

Spend: Total amount spent
Purchases: Number of purchases

Customer ID	Spend (£)	Purchases
0	200	5
1	220	6
2	210	5
3	400	15
4	410	16
5	390	14
6	100	2
7	90	2
8	105	3
9	205	5

Step 2: Visualising the Data

Each customer is a point in a 2D space, with Spend on one axis and Purchases on the other.

Plotted Data

Step 3: Hierarchical Clustering

To find patterns in the data, we use a method called hierarchical clustering - specifically, Ward’s method. It works by:

Starting with each customer as its own group.
Gradually merging the closest groups together based on how similar their spending and purchasing behaviour is.
Repeating this until all customers are part of one big cluster.

This process uses a distance matrix - a table that shows how far apart each customer is from the others.

Smaller values mean the customers are more alike and may be merged earlier. Larger values mean they are more different and will be merged later.

Ward’s method chooses which clusters to merge by looking for the combination that causes the smallest increase in overall variation within the groups. This helps keep clusters as compact and meaningful as possible.

Step 4: Dendrogram Output

The dendrogram shows how customers are clustered step-by-step:

Leaf nodes represent individual customers.
Vertical lines represent merge steps - the higher the line, the greater the dissimilarity at merge time.
A horizontal cut (not present in this graph) allows you to choose how many clusters you want to keep.

Dendrogram

Key Characteristics

Produces a nested structure that shows relationships between points and clusters.
No need to specify the number of clusters in advance.
Well suited for exploratory analysis and small-to-medium datasets.
Less efficient on very large datasets due to higher time complexity.

Example Hierarchical Clustering Scenarios

Customer Segmentation

An e-commerce team wants to understand how their customers naturally group based on behaviour - such as total amount spent and number of purchases.

They use hierarchical clustering to analyse their data:

Each customer starts as their own cluster.
The algorithm calculates distances between all customers based on their purchase behaviour.
It merges the two most similar customers into a cluster.
Then it continues to merge the next closest clusters - step-by-step - until all customers form one large cluster.
A dendrogram is created showing how and when each customer was grouped.

By slicing the dendrogram at a specific height, the team identifies 3 meaningful customer types:

High spenders with frequent purchases
Low spenders with infrequent activity
Medium spenders with regular habits

This helps them tailor promotions and email campaigns to each segment.

Document Clustering

A researcher wants to group a set of news articles into themes - without knowing how many categories exist.

They extract text-based features (like topic keywords or word frequencies), and use hierarchical clustering:

Each article is treated as its own cluster.
The algorithm calculates similarity between articles (e.g. cosine similarity based on word usage).
Articles with the most similar language are merged first.
Gradually, larger article groups are formed based on topic overlap.

The resulting dendrogram helps the researcher decide how to organise the articles:

One branch may group political news
Another may form around sports content
Another may contain science articles

This allows the researcher to sort and tag the articles even without pre-labels - purely based on content similarity.

Python Example: Hierarchical Clustering

Scenario: Grouping customers by spend and purchase behaviour using Ward’s method.

                        import numpy as np        # For numerical operations

                        import pandas as pd                # For handling the distance matrix

                        import matplotlib.pyplot as plt      # For plotting

                        from scipy.cluster.hierarchy import linkage, dendrogram  # For clustering + tree diagram

                        from scipy.spatial.distance import pdist, squareform          # For distance calculations

                        # Step 1: Define the data - 10 customers with [Spend (£), Purchases]

                        data = np.array([

                          [200, 5],

                          [220, 6],

                          [210, 5],

                          [400, 15],

                          [410, 16],

                          [390, 14],

                          [100, 2],

                          [90, 2],

                          [105, 3],

                          [205, 5]

                        ])

                        # Create IDs for labelling each customer

                        customer_ids = [f"C{i}" for i in range(len(data))]

                        # Step 2: Plot the customers on a scatterplot

                        plt.figure(figsize=(8, 6))

                        plt.scatter(data[:, 0], data[:, 1])

                        # Label each point with its Customer ID

                        for i, txt in enumerate(customer_ids):

                          plt.annotate(txt, (data[i, 0] + 2, data[i, 1] + 0.1))

                        plt.title("Customer Spend vs Purchases")

                        plt.xlabel("Spend (£)")

                        plt.ylabel("Purchases")

                        plt.grid(True)

                        plt.tight_layout()

                        plt.savefig("customer_scatter_labeled.png")

                        plt.close()

                        # Step 3: Compute the distance matrix

                        dist_matrix = pdist(data, metric='euclidean')

                        dist_df = pd.DataFrame(squareform(dist_matrix), columns=customer_ids, index=customer_ids)

                        print(np.round(dist_df, 1))

                        # Step 4: Ward linkage and dendrogram

                        linkage_matrix = linkage(data, method="ward")

                        plt.figure(figsize=(10, 6))

                        dendrogram(linkage_matrix, labels=customer_ids, leaf_rotation=0, leaf_font_size=10)

                        plt.title("Hierarchical Clustering Dendrogram")

                        plt.xlabel("Customer")

                        plt.ylabel("Dissimilarity")

                        plt.tight_layout()

                        plt.savefig("customer_dendrogram.png")

                        plt.close()

DBSCAN Clustering

DBSCAN (Density-Based Spatial Clustering of Applications with Noise) is an unsupervised learning algorithm that groups data points based on how closely packed they are.

Unlike K-Means or Hierarchical Clustering, DBSCAN does not try to form circular clusters or build a tree - instead, it finds groups of high density separated by areas of low density. It’s especially useful when the clusters have irregular shapes or when the number of clusters is unknown.

How it works:

Step 1: Choose two hyperparameters:
- eps: The maximum distance between two points to consider them part of the same cluster.
- min_samples: The minimum number of points required to form a dense region.
Step 2: The algorithm visits each point and checks how many neighbours are within eps distance.
Step 3: If the number of nearby points is at least min_samples, a new cluster is started and points are added to it.
Step 4: Points that don’t meet the density requirement and are not near any cluster are marked as noise (outliers).

DBSCAN Clustering
Chire. This file is licensed under the Creative Commons Attribution-Share Alike 3.0 Unported license.

Example 1: Customer Density

A shopping centre tracks footfall data and wants to identify popular customer gathering areas. Using DBSCAN, clusters are formed where many customers linger near stores - while passers-by in less popular zones are marked as noise.

Example 2: Geospatial Mapping

A conservation team uses GPS data to identify where herds of animals group together. DBSCAN groups the data into dense, irregular patches of movement, while occasional strays or outliers are ignored.

Key Characteristics

Aspect	Description
Cluster Shape	Can identify clusters of arbitrary shape, unlike K-Means which assumes circular boundaries.
Noise Handling	Labels low-density points as outliers, which are not assigned to any cluster.
No Need for K	Does not require you to pre-set the number of clusters. You define `eps` and `min_samples` instead.
Density Control	Flexible for finding clusters based on local density rather than strict distance.
Limitations	Can struggle with datasets that have varying densities or require fine-tuning of parameters.

Python Example: DBSCAN

Scenario: Identifying natural clusters of customer behaviour - without knowing how many clusters there are.

                        from sklearn.cluster import DBSCAN

                        import numpy as np

                        # Sample data: Spend (£) vs Number of Purchases

                        X = np.array([[200, 5], [210, 6], [215, 5], [100, 2], [105, 2], [95, 1], [400, 14], [405, 15], [410, 13], [300, 12]])

                        # Apply DBSCAN clustering

                        db = DBSCAN(eps=15, min_samples=2)

                        db.fit(X)

                        print("Cluster Labels:", db.labels_)

                        # -1 indicates an outlier

Choosing the Right Clustering Technique

K-Means: Best when clusters are round, equally sized, and the number of clusters is known.
Hierarchical: Suitable when you want to understand how data is grouped at multiple levels.
DBSCAN: Ideal for datasets with noise and clusters that are not evenly shaped.

Real-World Applications of Clustering

Customer Segmentation

Businesses use clustering to identify groups of customers with similar buying patterns - which helps with personalised marketing strategies.

Image Segmentation

In computer vision, clustering can group pixels by colour or texture, helping separate objects in an image.

Anomaly Detection

DBSCAN and other density-based methods help spot outliers - useful in fraud detection or fault monitoring systems.

Social Network Analysis

Clustering reveals communities or subgroups within a larger network of people or organisations.

Key Takeaways

Clustering groups data based on similarity - without using labelled outputs.
K-Means is efficient but needs a predefined number of clusters.
Hierarchical clustering builds a cluster tree - great for understanding relationships.
DBSCAN handles complex cluster shapes and can identify noise.
Each technique has strengths depending on the structure and size of the data.