Search Intent Data (2025): Build a Versioned Intent Dataset Your Classifiers Can Trust
A practical how-to for turning raw queries into a governed, confidence-scored intent dataset you can join to pages, outcomes, and AI citations - then keep it from rotting.

Search Intent Data Is a Versioned Table, Not a Label You Assign Once
Most teams treat search intent as a one-off tagging exercise: export queries, paste them into a spreadsheet, manually label each as informational, commercial, or transactional, then move on. That output is stale within weeks and useless for automated classifiers that need to decide which page archetype to serve. A governed intent dataset is a versioned table with columns for taxonomy version, label, confidence score, timestamp, and locale - essentially the same discipline you would apply to a customer database. Search Console's Queries report gives you a free query-to-URL join that most teams ignore, according to Google's documentation.
Intent is a probability distribution over page archetypes, not a single tag. Store the probability and the confidence so your classifiers can hedge appropriately.
Prerequisites: What You Need Before You Classify a Single Query
Skip these and your dataset will be unjoinable and unusable downstream:
- A search performance export containing query, page URL, and clicks (Search Console or Bing Webmaster Tools).
- A canonical list of your own landing page URLs with their page archetype (category, product, article, comparison, location).
- An agreed intent taxonomy with written definitions - no more than 10-12 archetypes for most sites.
- A labelling environment (even a spreadsheet) with columns for taxonomy version, label, confidence, timestamp, and locale.
- Access to SERP data for a sample of queries (via API or manual capture) to validate classifier output.
Step 1 - Freeze an Intent Taxonomy With Written Definitions
Write one-sentence definitions for each archetype you will classify. For an automotive site, you might use Vehicle listing (transactional), Price research, Ownership question, and Comparison. Do not reuse the generic informational/commercial/transactional/navigational set - it is too coarse to drive page-type decisions. Publish this taxonomy as version 1.0 and treat any change as a new version; never overwrite old labels.
Step 2 - Join Search Console Queries to Your Own Landing URLs
The Search Console UI hides the query-to-URL mapping, but the API exposes it. Pull rows where the query and page pair occur together; store both. Check current row limits in Google's API reference. For each query, record which of your URLs already receives it and its click-through rate - that is first-party intent evidence.
Step 3 - Parse Modifiers and Entity Tokens Before Any Classifier Runs
Many queries contain modifiers that override a generic classifier. Strip them, classify the residual, then re-append. Example: “waterproof hiking boots size 12” → remove size token → classify “waterproof hiking boots” → re-attach size as a filter attribute. Here is a minimal parsing snippet:
import re
MODIFIER_PATTERNS = {
'price': r'under\s+[R$€£]?\d[\d\s,]*',
'size': r'size\s+\d+',
'vs': r'\bvs\b',
'near_me': r'near\s+me',
'how_to': r'^how\s+to'
}
def parse_modifiers(query):
tokens = {}
for name, pattern in MODIFIER_PATTERNS.items():
match = re.search(pattern, query, re.IGNORECASE)
if match:
tokens[name] = match.group(0)
return tokens
Step 4 - Classify Against the SERP, Store a Confidence Score
Use a combination of SERP shape (how many ads, whether a shopping carousel appears, whether featured snippets are present) and query morphology to assign a probability to each archetype. Output a table row per query with intent_label, intent_probability, and classifier_confidence. A simple table structure:
| query | taxonomy_version | intent_label | intent_probability | locale | timestamp |
|---|---|---|---|---|---|
| bakkie for sale under R250 000 | v1.2 | vehicle_listing | 0.92 | en-ZA | 2025-03-01 |
| is a bakkie good for a family | v1.2 | ownership_question | 0.87 | en-ZA | 2025-03-01 |
LLM classifiers tend to over-label confidently. Always store a probability and a separate confidence score so you can apply a threshold later.


