Association Rule

What Is Association Rule Learning?

Association rule learning is a machine learning technique used to find interesting relationships between attributes in large datasets. It is widely used in applications where we want to discover co-occurrences or patterns in item sets, behaviours, or events.

Unlike clustering or classification, association rule learning does not group or label data - it identifies meaningful rules that highlight dependencies or correlations between variables.

Association Rule Mining

Association rule mining finds useful “if–then” patterns in transaction-style data. It answers questions like: “If a basket has bread and butter, then it often also has jam.”

Why it matters: It helps with recommendations (“you might also buy…”), product bundles, store layout, and spotting habits or oddities-without needing pre-made labels.

Key ideas

Support: How common a combination is (e.g. appears in 30% of baskets).
Confidence: When the left side occurs, how often the right side also appears (e.g. with bread+butter, jam shows up 70% of the time).
Lift: How much stronger the rule is than chance (>1 is helpful, ≈1 means it’s ordinary, <1 means it’s misleading).

Example (5 receipts)

Receipt	Items
1	bread, butter, jam
2	bread, milk
3	butter, jam
4	bread, butter
5	milk, eggs

Our example rule is: {bread, butter} → {jam} : or, If bread and butter are bought, then so is jam.

Support: in 1 out of 5 baskets (20%).
Confidence: of the 2 baskets that had bread+butter, 1 also had jam (50%).
Lift: compare that 50% with jam's base rate (in 2/5 baskets = 40%) → above chance (1.25).

Steps

Collect transactions: Lists of items per basket/order/session.
Tidy names: Make item names consistent (e.g. "cola 330ml" vs "cola can").
Find common groups: Keep item sets that appear often enough (minimum support).
Create rules: Turn those groups into "if (left) then (right)" suggestions.
Score & filter: Keep rules with good confidence and lift; drop weak or obvious ones.
Validate: Check on recent data, and A/B test if possible.
Act & monitor: Recommend pairs, bundle deals, adjust layout; re-run as habits change.

Note: Association shows co-occurrence, not cause. Use common sense and testing before making changes.

Measuring Rule Strength

Measure	Description
Support	How often the full rule (both A and B) appears in the dataset. Higher support means the rule is more commonly observed.
Support (Consequent)	How often the outcome of the rule (e.g. B) occurs on its own. Needed to calculate Lift.
Confidence	The likelihood that the rule is correct - i.e. how often B occurs when A has occurred. Example: If 80% of people who buy bread also buy butter, confidence is 80%.
Lift	Measures how much more likely A and B are to occur together compared to if they were independent. Calculated as: `Confidence / Support(B)`. A lift > 1 suggests a positive association.

Python Example: Association Rule Learning

Scenario: Identifying patterns in supermarket transactions to support product placement.

            # requires mlxtend

            from mlxtend.frequent_patterns import apriori, association_rules

            import pandas as pd

            # Sample dataset

            data = {'Milk': [1, 0, 1, 1, 0, 1, 1, 0, 1, 0],

            'Bread': [1, 1, 0, 1, 1, 1, 0, 1, 1, 1],

            'Butter': [0, 1, 1, 1, 0, 1, 1, 0, 1, 0]}

            df = pd.DataFrame(data)

            # Find frequent itemsets

            frequent_itemsets = apriori(df, min_support=0.5, use_colnames=True)

            # Generate rules

            rules = association_rules(frequent_itemsets, metric="confidence", min_threshold=0.6)

            print(rules[['antecedents', 'consequents', 'support', 'confidence', 'lift']])

Real-World Applications of Association Rule Learning

Lift Formula

            \[
            \text{Lift}(A \rightarrow B) = \frac{\text{Confidence}(A \rightarrow B)}{\text{Support}(B)}
            \]
        

This formula compares how likely B is to occur given A, against how likely B is to occur in general. A lift value:

> 1: A and B occur together more than expected (positive association)
= 1: A and B occur independently (no real association)
< 1: A reduces the likelihood of B (negative association)

Market Basket Analysis

Retailers use association rules to discover product combinations that are often bought together - e.g. {Milk} → {Bread} (Milk implies Bread. Or, if milk is bought, bread is likely to be bought as well.) - to optimise store layout and suggest add-on purchases.

For example, the data might reveal:

Support (Milk ∩ Bread): 0.46 - 46% of all transactions contain both milk and bread.
Support (Bread): 0.65 - Bread appears in 65% of transactions overall.
Confidence (Milk → Bread): 0.70 - 70% of customers who buy milk also buy bread.
Therefore, Lift: 0.70 / 0.65 = 1.08 - Milk buyers are slightly more likely to also buy bread than the average shopper.

Crime Analysis

By mining crime records, patterns can be revealed - such as {Vandalism} → {Theft} - showing that high vandalism areas often also experience theft.

For example, the data might reveal:

Support (Vandalism ∩ Theft): 0.32 - This pattern appears in 32% of crime reports.
Support (Theft): 0.47 - Theft appears in 47% of crime reports overall.
Confidence (Vandalism → Theft): 0.68 - In 68% of vandalism reports, theft also occurs.
Therefore, Lift: 0.68 / 0.47 = 1.45 - Theft is 1.45× more likely when vandalism is present.

Healthcare

Doctors and researchers use association rules to find links between symptoms, conditions, and treatments - e.g. {Hypertension, Diabetes} → {ACE Inhibitor}.

For example, the data might reveal:

Support (Conditions ∩ Treatment): 0.33 - 33% of all patients have both conditions and are prescribed this treatment.
Support (ACE Inhibitor): 0.44 - This treatment appears in 44% of all prescriptions.
Confidence (Conditions → Treatment): 0.75 - 75% of patients with both conditions are prescribed ACE inhibitors.
Therefore, Lift: 0.75 / 0.44 = 1.70 - Treatment is 1.7× more likely when both conditions are present.

Fraud Detection

Banks and insurers use association rule mining to uncover patterns like {Foreign IP, High Transaction} → {Fraud}.

Support (Pattern ∩ Fraud): 0.02 - This pattern shows up in 2% of cases.
Support (Fraud): 0.024 - Fraud occurs in 2.4% of all transactions.
Confidence (Pattern → Fraud): 0.85 - 85% of such transactions are fraudulent.
Therefore, Lift: 0.85 / 0.024 = 35.42 - The fraud risk is over 35× higher than average when this pattern appears.

Challenges of Association Rule Learning

Large Datasets: Processing all possible item combinations in big data can be computationally expensive.
Threshold Tuning: Setting the right support, confidence, and lift values requires experimentation.
Interpreting Results: Not all discovered rules are meaningful - some may be coincidental or unhelpful.

Key Takeaways

Association rule learning reveals relationships between items or events in large datasets.
It uses support, confidence, and lift to evaluate the strength of discovered rules.
Useful in retail, healthcare, crime prevention, and fraud detection.
Helps in uncovering insights that aren't immediately obvious - like dependencies or hidden correlations.