SERP Based Keyword Clustering: How It Works and When to Use It

SERP based keyword clustering is a way of grouping keywords by comparing the pages Google ranks for them. You pull the top 10 organic URLs for every keyword, count how many URLs each pair shares, and put keywords in the same group when the overlap passes a threshold. A common convention is 3 shared URLs out of 10. Keywords that share enough results can usually be targeted by one page, because Google already treats them as the same need.

It’s the closest thing we have to asking Google directly which searches belong together. It also costs money and time that meaning-based methods don’t. This guide walks through the method with tables you can read like a diagram, so you can judge the output of any tool instead of trusting it blindly.

What Does SERP Based Keyword Clustering Compare?

It compares ranking URLs, nothing else. Not the words in the keywords, not their meaning, not their volume. Two queries count as related only if Google shows the same pages for both.

Table 1 is an illustrative example I built for this post, not real ranking data. It shows the top 5 results for two keywords, with shared URLs marked.

Table 1: Two results pages side by side (illustrative)

Look at rows 2 to 4. The same three URLs appear for both queries, just in a different order. Position doesn’t matter to the basic method. A URL counts as shared if it appears anywhere in both top 10 lists.

Honestly, this is why I like SERP based keyword clustering. When I plan clusters for a site, I don’t need to argue with anyone about whether “best” changes the intent. Google has already answered. The answer is in the table.

How Do You Read a SERP Overlap Matrix?

With more than two keywords, you build a matrix. Every cell holds the number of URLs two keywords share in their top 10. Table 2 uses five illustrative keywords, listed from highest to lowest search volume.

Table 2: Shared top 10 URLs between five keywords (illustrative)

Read it row by row. A, B and C share 4 to 6 URLs with each other, so they form a tight block in the top-left corner. D shares 3 with A but only 1 or 2 with the others. E barely overlaps with anything.

That’s already a content plan. The A, B and C block is one page. D and E are probably their own pages. The interesting question is what happens to D, and that depends on the clustering mode you pick.

Why Is 3 Out of 10 the Usual Threshold?

It isn’t a rule. It’s a convention that many SEOs and tools start from, because three shared results is roughly where overlap stops looking like coincidence. Big sites like Wikipedia or a major retailer can appear in unrelated results pages, so one shared URL proves little. Three is harder to explain away.

Tools don’t agree on the number, and they don’t have to. Some default to a percentage of shared URLs, some let you set any count, and some compare deeper than the top 10. I listed each vendor’s documented default in my comparison of keyword clustering tools, so I won’t repeat it here.

My take: raise the threshold to 4 or 5 when you’re building individual pages and a wrong merge is expensive. Drop it to 2 or 3 when you’re sketching topic hubs. Then read a sample of the output yourself. No threshold replaces ten minutes of looking at real clusters, because the number only tells you how much two results pages overlap, while your eyes tell you whether the pages Google ranks are the kind of page you were planning to write.

Hard vs Soft Clustering: The Same Matrix, Two Results

This is where most confusion starts. Both modes use the same overlap counts from Table 2, with the same threshold of 3. They just ask different questions.

Soft clustering picks the highest-volume keyword as the head and compares every other keyword against the head only. Hard clustering requires every keyword in a group to meet the threshold with every other member, not just the head.

Table 3: Clusters from Table 2 at a threshold of 3

Look at the D column in Table 2 again. It clears the bar with A and fails it with B and C. Soft mode lets it in because it only checks A. Hard mode keeps it out because “free email marketing software” is a different job for Google: the results lean toward free plans, while the others lean toward paid comparisons.

Which is right? Depends on the job. For a single page, I’d trust hard mode. If D goes on the same page as A, B and C, you’re writing one article that tries to serve both paid and free intent, and it usually ranks worse for both. For a topic hub, soft mode is fine, since D can become a supporting page under the same hub.

The Detail That Breaks Most SERP Clusters

Two small data problems cause more bad clusters than any threshold choice.

The first is URL formatting. If one export lists https://www.site.com/page/ and another lists http://site.com/page, a naive comparison sees two different URLs. Before you count anything, normalize every URL:

• Lowercase the whole string.
• Drop the protocol and the “www” prefix.
• Strip trailing slashes.
• Remove query strings, tracking parameters and fragments.

When I audit a clustering export that looks oddly fragmented, this is the first thing I check. It’s boring. It’s also the culprit surprisingly often.

The second is that a results page is a snapshot. Google’s own Performance report help page notes that “Search results are specific to the time, place, device, and recent history of the person searching.” So pull every keyword on the same day, for the same country, language and device. If you mix a mobile pull from London with a desktop pull from New York, the overlap numbers stop meaning anything.

I also date every pull. Rankings shift, and a cluster built from results six months old can be quietly wrong today.

How Much Does SERP Clustering Cost?

The cost is in collecting data, not in the math. Every keyword needs one results page, so 2,000 keywords means 2,000 SERP requests. SERP data providers and clustering tools typically charge per request or per keyword, and you pay again every time you refresh.

The comparison step is almost free. Comparing every pair of 2,000 keywords means 1,999,000 pairs, which sounds scary, but it’s simple set math that runs in seconds on a laptop. So the budget question is always “how many keywords do I pull results for?”, never “how many comparisons can I afford?”

That’s why I rarely send a raw 20,000-row export straight to SERP based keyword clustering. In my experience, most of those rows never become pages anyway. Here’s the order I use to keep the bill small:

1. Remove duplicates, plurals and obvious junk with a simple lexical pass.
2. Group what’s left by meaning, which costs nothing.
3. Drop groups you’d never publish, such as off-topic or no-fit queries.
4. Pull results data only for the keywords in the groups that survive.
5. Run the SERP overlap check and let it split or merge those groups.

A Short Python Script for SERP Clustering

If you already have the results data, you can cluster it yourself. One warning first. Don’t scrape Google to get it. Google’s spam policies say that “scraping results for rank-checking purposes or other types of automated access to Google Search conducted without express permission” violates its policies and Terms of Service.

Instead, export the data from a SERP API provider or rank tracker whose terms allow it. You need two CSV files. The first has one row per ranking URL:

keyword,position,url
email marketing software,1,https://vendor-a.com/email-marketing
email marketing software,2,https://reviewsite-x.com/best-email-marketing
best email marketing software,1,https://reviewsite-x.com/best-email-marketing

The second has one row per keyword with its search volume, which decides the order and the head keyword:

keyword,volume
email marketing software,900
best email marketing software,500

The script below uses only the Python standard library, so there’s nothing to install. It needs Python 3.9 or newer. Set MODE to “hard” or “soft” and THRESHOLD to the number of shared URLs you want.

import csv
from collections import defaultdict
from urllib.parse import urlsplit

SERP_FILE = "serps.csv"        # columns: keyword, position, url
VOLUME_FILE = "volumes.csv"    # columns: keyword, volume
THRESHOLD = 3                  # shared URLs in the top 10
MODE = "hard"                  # "hard" or "soft"

def clean_url(url):
    parts = urlsplit(url.strip().lower())
    host = parts.netloc.removeprefix("www.")
    return host + parts.path.rstrip("/")

def load_serps(path):
    top10 = defaultdict(set)
    with open(path, newline="", encoding="utf-8") as f:
        for row in csv.DictReader(f):
            if int(row["position"]) <= 10:
                top10[row["keyword"].strip().lower()].add(clean_url(row["url"]))
    return top10

def load_volumes(path):
    volumes = {}
    with open(path, newline="", encoding="utf-8") as f:
        for row in csv.DictReader(f):
            volumes[row["keyword"].strip().lower()] = int(row["volume"] or 0)
    return volumes

def shared(a, b):
    return len(top10[a] & top10[b])

def soft_clusters(keywords):
    clusters, used = [], set()
    for head in keywords:
        if head in used:
            continue
        group = [head] + [k for k in keywords
                          if k not in used and k != head
                          and shared(head, k) >= THRESHOLD]
        used.update(group)
        clusters.append(group)
    return clusters

def hard_clusters(keywords):
    clusters = []
    for kw in keywords:
        for group in clusters:
            if all(shared(kw, member) >= THRESHOLD for member in group):
                group.append(kw)
                break
        else:
            clusters.append([kw])
    return clusters

top10 = load_serps(SERP_FILE)
volumes = load_volumes(VOLUME_FILE)
keywords = sorted(top10, key=lambda k: volumes.get(k, 0), reverse=True)

build = hard_clusters if MODE == "hard" else soft_clusters
clusters = build(keywords)

with open("clusters.csv", "w", newline="", encoding="utf-8") as f:
    writer = csv.writer(f)
    writer.writerow(["cluster_id", "head_keyword", "keyword", "volume", "shared_with_head"])
    for i, group in enumerate(clusters, start=1):
        head = group[0]
        for kw in group:
            writer.writerow([i, head, kw, volumes.get(kw, 0), shared(head, kw)])

print(len(keywords), "keywords ->", len(clusters), "clusters |", MODE, "mode, threshold", THRESHOLD)

I tested it on a small made-up file before publishing. A keyword that cleared the bar with the head but not with the second keyword stayed in the group in soft mode and split out in hard mode, the same behaviour as Table 3.

The clean_url function drops query strings and fragments, which handles the formatting problem above. Hard mode is greedy: it places each keyword in the first group where it fits every member, working from the highest volume down. That’s simple and predictable, and it’s the logic I want for page-level planning.

When Is Semantic Clustering Enough?

More often than people admit. If you’re grouping a raw list into rough topics for a topical map, meaning-based clustering is fast, free and good enough. My semantic keyword clustering Python tutorial does exactly that with embeddings and no SERP data at all.

Switch to SERP data when a wrong merge would cost you a page. That means commercial keywords, keywords with similar wording but different intent, and anything going straight into a brief. The best workflow uses both: semantic grouping to shrink the list cheaply, then SERP overlap to confirm the clusters you plan to publish. I compare the two approaches in more depth in keyword clustering vs semantic clustering.

After clustering, each group still needs an owner URL on your site. That’s a separate step called keyword mapping. And if your list is full of tiny long-tail phrases, read my guide to zero volume keywords before you pay to pull results for all of them.

What the Three Tables Proved

Table 1 showed that “related” in SERP clustering means shared ranking URLs, regardless of order. Table 2 showed that a simple count matrix already sketches your pages before any algorithm runs. Table 3 showed that the clustering mode, not just the threshold, decides whether an intent-shifted keyword like the “free” variant gets merged or split.

If you’d rather have someone else build and check the clusters, that’s part of our keyword research service, where every cluster is tied to one page before it goes into a content plan.

Frequently Asked Questions

What Is a Good SERP Overlap Threshold for Keyword Clustering?

Three shared URLs out of the top 10 is a common starting convention, not a Google rule. I use 4 or 5 for page-level clusters where a wrong merge is costly, and 2 or 3 for broad topic groups. Always review a sample of the output.

What Is the Difference Between Hard and Soft Keyword Clustering?

Soft clustering compares each keyword only with the group’s head keyword, usually the highest-volume one. Hard clustering requires every keyword to share enough URLs with every other keyword in the group. Hard mode gives smaller, tighter groups that map more safely to single pages.

Is SERP Based Clustering Better Than Semantic Clustering?

It’s more accurate for deciding what goes on one page, because it reflects what Google ranks. Semantic clustering is cheaper and faster for rough topic grouping. Many SEOs use semantic grouping first, then SERP overlap to confirm the clusters that matter.

Can I Scrape Google Results for SERP Clustering?

No. Google’s spam policies say that scraping results for rank checking or other automated access without permission violates its policies and Terms of Service. Use an export from a SERP API provider or rank tracker whose terms allow it.

Last updated: September 2026 by Mizanur Rahman